2024-09-21T02:32:17.884 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T02:32:17.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:32:18.010 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913435' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIj+3IAwpOLF2FjQ/e62bMDeHO2cYzjbGSM6aWVDZywtSmBMac1Ya6eCRbHkTBUxO0uqgQ4OFyAf4N174Pz25rk= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGzJUzdctNn4IxMVaSPzo/QjN93izX/SEzQLaRIXStI8VxTh/OyZIh/TgXiycAU7s/rNw6oW1+lFTh19nvyWLV0= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBODYSC0NSI9vuyoqUeeVct1Al73O+LyzUvapLPCnn1I8/lz99MyVMp3jlMlnXGPv4o19yg3lO1yfp9NPBzHVdq0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T02:32:18.011 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T02:32:18.011 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T02:32:18.011 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T02:32:18.013 INFO:teuthology.task.internal:Checking packages... 2024-09-21T02:32:18.033 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T02:32:18.033 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T02:32:18.033 INFO:teuthology.packaging:ref: None 2024-09-21T02:32:18.033 INFO:teuthology.packaging:tag: None 2024-09-21T02:32:18.033 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:32:18.033 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:32:18.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:32:18.184 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T02:32:18.185 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T02:32:18.191 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T02:32:18.191 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T02:32:18.197 INFO:teuthology.task.internal:Saving configuration 2024-09-21T02:32:18.217 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T02:32:18.223 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T02:32:18.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:26:50.053055', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIj+3IAwpOLF2FjQ/e62bMDeHO2cYzjbGSM6aWVDZywtSmBMac1Ya6eCRbHkTBUxO0uqgQ4OFyAf4N174Pz25rk='} 2024-09-21T02:32:18.268 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:26:50.056835', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGzJUzdctNn4IxMVaSPzo/QjN93izX/SEzQLaRIXStI8VxTh/OyZIh/TgXiycAU7s/rNw6oW1+lFTh19nvyWLV0='} 2024-09-21T02:32:18.293 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:26:50.055071', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBODYSC0NSI9vuyoqUeeVct1Al73O+LyzUvapLPCnn1I8/lz99MyVMp3jlMlnXGPv4o19yg3lO1yfp9NPBzHVdq0='} 2024-09-21T02:32:18.293 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T02:32:18.299 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T02:32:18.300 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T02:32:18.300 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T02:32:18.300 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T02:32:18.382 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33a401c160>, signals=[15]) 2024-09-21T02:32:18.382 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T02:32:18.388 INFO:teuthology.task.internal:Opening connections... 2024-09-21T02:32:18.388 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T02:32:18.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:32:18.463 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-09-21T02:32:18.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:32:18.538 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-09-21T02:32:18.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:32:18.609 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T02:32:18.622 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-21T02:32:18.640 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-21T02:32:18.640 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:32:18.697 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:32:18.698 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:32:18.698 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:32:18.698 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:32:18.698 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-21T02:32:18.729 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-09-21T02:32:18.747 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-09-21T02:32:18.748 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-09-21T02:32:18.806 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-09-21T02:32:18.806 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-09-21T02:32:18.806 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:32:18.807 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:32:18.808 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-09-21T02:32:18.829 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-09-21T02:32:18.849 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-09-21T02:32:18.849 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-09-21T02:32:18.908 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-09-21T02:32:18.908 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-09-21T02:32:18.908 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-09-21T02:32:18.908 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:32:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-09-21T02:32:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:32:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:32:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-09-21T02:32:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:32:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:32:18.910 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:32:18.910 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:32:18.910 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:32:18.910 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:32:18.910 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-09-21T02:32:18.937 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T02:32:18.948 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T02:32:18.959 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T02:32:18.960 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:32:18.962 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:32:18.964 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:32:18.981 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T02:32:18.987 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T02:32:18.987 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:32:19.019 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:32:19.022 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:32:19.143 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T02:32:19.221 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T02:32:19.240 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T02:32:19.241 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T02:32:19.241 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T02:32:19.241 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:32:19.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:32:19.258 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-21T02:32:19.315 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-21T02:32:19.315 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-21T02:32:19.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-21T02:32:22.908 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-21T02:32:24.909 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-21T02:32:29.276 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-21T02:32:30.852 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-21T02:32:32.097 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T02:32:37.703 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T02:32:39.676 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T02:32:39.676 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T02:32:39.770 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-21T02:32:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:32:39.786 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-21T02:32:39.787 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-21T02:32:39.787 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-21T02:32:40.029 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 319 kB/s | 36 kB 00:00 2024-09-21T02:32:42.721 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-09-21T02:32:44.096 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 7.0 MB/s | 29 MB 00:04 2024-09-21T02:32:44.490 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 7.6 MB/s | 35 MB 00:04 2024-09-21T02:32:44.490 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:32:44.490 INFO:teuthology.orchestra.run.smithi031.stdout:Total 17 MB/s | 79 MB 00:04 2024-09-21T02:32:45.207 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-21T02:32:45.591 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-21T02:32:45.591 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-21T02:32:46.569 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-21T02:32:46.569 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-21T02:32:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-21T02:32:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T02:32:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:32:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:32:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:32:57.917 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:32:57.961 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:33:03.101 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:33:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:33:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:33:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:33:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T02:33:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:33:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:33:43.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:33:43.153 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:33:43.153 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-21T02:33:43.153 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T02:33:43.154 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T02:33:43.154 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T02:33:43.154 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T02:33:43.154 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:33:43.154 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-21T02:33:43.510 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T02:33:43.602 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T02:33:43.602 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T02:33:43.602 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T02:33:43.603 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T02:33:43.603 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T02:33:43.603 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T02:33:43.603 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T02:33:43.603 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T02:33:43.660 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T02:33:43.661 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T02:33:43.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-21T02:33:43.698 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-21T02:33:43.698 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T02:33:43.699 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:33:43.700 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T02:33:43.701 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T02:33:43.701 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-508.el9.x86_64 2024-09-21T02:33:43.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T02:33:44.370 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-21T02:33:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T02:33:45.750 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-21T02:33:45.752 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-21T02:33:45.767 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.KiMPfwVxF0 2024-09-21T02:33:45.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KiMPfwVxF0 2024-09-21T02:33:45.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.KiMPfwVxF0 2024-09-21T02:33:45.970 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.KiMPfwVxF0 is 6KB 2024-09-21T02:33:45.990 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.KiMPfwVxF0 2024-09-21T02:33:46.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T02:33:46.074 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T02:33:46.075 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T02:33:46.075 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T02:33:46.075 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T02:33:46.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T02:33:46.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-21T02:34:16.231 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T02:34:16.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:34:16.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:34:34.607 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-21T02:34:43.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:34:43.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:35:01.999 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-21T02:35:14.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:35:14.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:35:17.075 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-21T02:35:32.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:35:32.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:35:32.386 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-21T02:35:32.732 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:35:32.732 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T02:35:32.732 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-21T02:35:32.749 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-508.el9.x86_64 2024-09-21T02:35:32.750 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T02:35:32.750 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T02:35:32.750 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T02:35:33.751 DEBUG:teuthology.parallel:result is None 2024-09-21T02:35:33.751 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T02:35:33.765 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T02:35:33.765 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:35:33.767 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:35:33.770 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:35:33.788 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T02:35:33.795 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T02:35:33.803 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T02:35:33.804 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:35:33.826 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:35:33.829 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:35:34.014 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T02:35:34.020 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T02:35:34.020 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:35:34.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:35:34.037 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:35:34.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:35:34.055 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:35:34.057 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:35:34.080 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:35:34.112 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:35:34.124 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:35:34.138 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:35:34.138 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:35:34.148 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:35:34.163 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:35:34.165 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T02:35:34.175 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T02:35:34.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:35:34.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:35:34.179 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:35:34.209 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T02:35:34.218 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T02:35:34.218 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:35:34.248 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:35:34.250 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:35:34.266 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:35:34.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:35:34.395 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:35:34.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:35:34.521 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:35:34.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:35:34.588 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:35:34.633 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:35:34.680 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:35:34.766 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:35:34.807 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:35:34.807 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:35:34.873 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:35:34.919 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:35:34.973 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:35:35.058 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:35:35.101 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:35:35.101 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:35:35.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-21T02:35:35.169 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-09-21T02:35:35.171 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-09-21T02:35:35.208 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:35:35.213 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:35:35.245 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:35:35.765 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T02:35:35.773 INFO:teuthology.task.internal:Starting timer... 2024-09-21T02:35:35.773 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T02:35:35.783 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T02:35:35.792 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T02:35:35.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-21T02:35:35.862 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-21T02:35:35.864 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-09-21T02:35:35.941 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-09-21T02:35:35.942 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-09-21T02:35:36.007 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-09-21T02:35:36.009 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T02:35:36.009 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-21T02:35:36.030 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-21T02:35:36.031 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-09-21T02:35:36.063 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-09-21T02:35:36.064 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-09-21T02:35:36.095 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-09-21T02:35:36.095 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-09-21T02:35:36.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:35:36.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:35:36.125 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:35:36.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:35:36.155 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:35:36.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:35:36.183 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T02:35:36.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:35:36.228 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:35:36.268 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:35:36.300 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T02:35:36.312 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T02:35:36.315 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T02:35:36.419 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T02:35:36.429 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T02:35:36.430 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi165.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T02:39:31.733 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-09-21T02:39:31.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:39:31.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:39:31.810 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-21T02:39:31.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T02:39:31.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T02:39:31.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:39:31.964 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-21T02:39:32.042 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-21T02:39:32.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-21T02:39:32.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:39:32.116 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-21T02:39:32.202 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-21T02:39:32.203 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T02:39:32.213 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T02:39:32.213 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:39:32.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:39:32.216 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:39:32.216 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:39:32.219 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:39:32.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:39:32.247 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:39:32.248 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:39:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:39:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:39:32.273 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:39:32.288 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:39:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-09-21T02:39:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-21T02:39:32.300 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-09-21T02:39:32.306 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-21T02:39:32.313 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-09-21T02:39:32.314 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:39:32.322 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:39:32.324 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-09-21T02:39:32.328 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:39:32.338 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:39:32.338 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:39:32.351 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:39:32.383 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:39:32.386 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:39:32.386 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-09-21T02:39:32.386 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.386 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.386 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.386 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.397 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:39:32.401 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:39:32.401 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-21T02:39:32.401 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.401 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.406 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:39:32.410 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:39:32.410 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-21T02:39:32.410 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.410 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.410 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.410 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:39:32.411 INFO:teuthology.run_tasks:Running task install... 2024-09-21T02:39:32.421 DEBUG:teuthology.task.install:project ceph 2024-09-21T02:39:32.421 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T02:39:32.422 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T02:39:32.422 INFO:teuthology.task.install:Using flavor: default 2024-09-21T02:39:32.427 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T02:39:32.427 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T02:39:32.427 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:39:32.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:39:32.429 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:39:32.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:39:32.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:39:32.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:39:32.587 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:39:32.587 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:39:32.598 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:39:32.598 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:39:32.601 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:39:32.601 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:39:32.695 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:39:32.695 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:39:32.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:39:32.704 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:39:32.704 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:39:32.704 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:39:32.709 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:39:32.709 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:39:32.709 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:39:32.724 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:39:32.725 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:39:32.731 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:39:32.731 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:39:32.735 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:39:32.735 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:39:32.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:39:32.796 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:39:32.801 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:39:32.871 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:39:32.872 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:39:32.876 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:39:32.936 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-09-21T02:39:32.936 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-21T02:39:32.938 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-09-21T02:39:32.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-21T02:39:32.939 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-09-21T02:39:32.941 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-09-21T02:39:33.164 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-09-21T02:39:33.165 INFO:teuthology.orchestra.run.smithi165.stdout:45 files removed 2024-09-21T02:39:33.181 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-21T02:39:33.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:39:33.193 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:39:33.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:39:33.732 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-21T02:39:33.747 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-21T02:39:33.766 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 329 kB/s | 84 kB 00:00 2024-09-21T02:39:34.040 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T02:39:34.049 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 102 kB/s | 24 kB 00:00 2024-09-21T02:39:34.083 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 78 kB/s | 24 kB 00:00 2024-09-21T02:39:34.308 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 8.7 kB/s | 1.9 kB 00:00 2024-09-21T02:39:34.316 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T02:39:34.359 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T02:39:36.142 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-21T02:39:36.150 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.3 MB 00:01 2024-09-21T02:39:37.099 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-21T02:39:37.951 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-21T02:39:40.692 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 20 MB 00:02 2024-09-21T02:39:42.727 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.5 MB 00:00 2024-09-21T02:39:44.345 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 19 kB 00:00 2024-09-21T02:39:44.568 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 20 MB 00:07 2024-09-21T02:39:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T02:39:46.109 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-21T02:39:47.626 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-21T02:39:48.398 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T02:39:48.885 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T02:39:50.553 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-21T02:39:50.963 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-21T02:39:51.357 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T02:39:53.257 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:39:53.258 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:39:53.269 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:39:53.270 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:39:53.271 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:39:53.272 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:39:53.274 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:39:53.276 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:39:53.277 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:39:53.344 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:39:53.375 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:39:53.376 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:39:53.377 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:39:53.378 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:39:53.379 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:39:53.380 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:39:53.380 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:39:53.380 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:39:53.381 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:39:53.382 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:39:53.383 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:39:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:39:53.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:39:53.386 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:39:53.387 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:39:53.388 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout:Install 149 Packages 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 8 Packages 2024-09-21T02:39:53.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:39:53.390 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 228 M 2024-09-21T02:39:53.390 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-21T02:39:53.823 INFO:teuthology.orchestra.run.smithi031.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-21T02:39:53.861 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-21T02:39:54.081 INFO:teuthology.orchestra.run.smithi031.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.3 MB/s | 873 kB 00:00 2024-09-21T02:39:54.190 INFO:teuthology.orchestra.run.smithi031.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T02:39:54.215 INFO:teuthology.orchestra.run.smithi031.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.1 MB/s | 148 kB 00:00 2024-09-21T02:39:54.299 INFO:teuthology.orchestra.run.smithi031.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 21 MB/s | 2.3 MB 00:00 2024-09-21T02:39:54.416 INFO:teuthology.orchestra.run.smithi031.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-09-21T02:39:54.574 INFO:teuthology.orchestra.run.smithi031.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.0 MB 00:00 2024-09-21T02:39:54.791 INFO:teuthology.orchestra.run.smithi031.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T02:39:54.833 INFO:teuthology.orchestra.run.smithi031.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 610 kB/s | 25 kB 00:00 2024-09-21T02:39:55.175 INFO:teuthology.orchestra.run.smithi031.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 17 MB/s | 10 MB 00:00 2024-09-21T02:39:55.242 INFO:teuthology.orchestra.run.smithi031.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T02:39:55.326 INFO:teuthology.orchestra.run.smithi031.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 324 kB/s | 27 kB 00:00 2024-09-21T02:39:55.534 INFO:teuthology.orchestra.run.smithi031.stdout:(13/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 16 MB/s | 18 MB 00:01 2024-09-21T02:39:55.559 INFO:teuthology.orchestra.run.smithi031.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 136 kB/s | 32 kB 00:00 2024-09-21T02:39:55.743 INFO:teuthology.orchestra.run.smithi031.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 3.6 MB/s | 766 kB 00:00 2024-09-21T02:39:55.768 INFO:teuthology.orchestra.run.smithi031.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 760 kB/s | 158 kB 00:00 2024-09-21T02:39:55.794 INFO:teuthology.orchestra.run.smithi031.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T02:39:55.835 INFO:teuthology.orchestra.run.smithi031.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 7.4 MB/s | 507 kB 00:00 2024-09-21T02:39:55.911 INFO:teuthology.orchestra.run.smithi031.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 601 kB/s | 45 kB 00:00 2024-09-21T02:39:56.013 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:39:56.014 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:39:56.025 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:39:56.026 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:39:56.027 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:39:56.028 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:39:56.028 INFO:teuthology.orchestra.run.smithi031.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 23 MB/s | 5.3 MB 00:00 2024-09-21T02:39:56.029 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:39:56.032 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:39:56.032 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:39:56.053 INFO:teuthology.orchestra.run.smithi031.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.0 MB/s | 148 kB 00:00 2024-09-21T02:39:56.087 INFO:teuthology.orchestra.run.smithi031.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T02:39:56.095 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:39:56.108 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:39:56.109 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:39:56.110 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:39:56.111 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:39:56.112 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:39:56.113 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:39:56.114 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:39:56.115 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:39:56.116 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:39:56.117 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:39:56.118 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:39:56.119 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout:Install 149 Packages 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 8 Packages 2024-09-21T02:39:56.120 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:39:56.121 INFO:teuthology.orchestra.run.smithi031.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T02:39:56.121 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 228 M 2024-09-21T02:39:56.121 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-21T02:39:56.137 INFO:teuthology.orchestra.run.smithi031.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T02:39:56.163 INFO:teuthology.orchestra.run.smithi031.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T02:39:56.188 INFO:teuthology.orchestra.run.smithi031.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T02:39:56.247 INFO:teuthology.orchestra.run.smithi031.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.8 MB/s | 170 kB 00:00 2024-09-21T02:39:56.305 INFO:teuthology.orchestra.run.smithi031.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 22 MB/s | 3.1 MB 00:00 2024-09-21T02:39:56.330 INFO:teuthology.orchestra.run.smithi031.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 410 kB/s | 34 kB 00:00 2024-09-21T02:39:56.356 INFO:teuthology.orchestra.run.smithi031.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T02:39:56.755 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T02:39:56.789 INFO:teuthology.orchestra.run.smithi031.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 17 MB/s | 7.4 MB 00:00 2024-09-21T02:39:56.865 INFO:teuthology.orchestra.run.smithi031.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.8 MB/s | 292 kB 00:00 2024-09-21T02:39:57.054 INFO:teuthology.orchestra.run.smithi178.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 55 kB/s | 6.6 kB 00:00 2024-09-21T02:39:57.157 INFO:teuthology.orchestra.run.smithi031.stdout:(33/157): ceph-mgr-dashboard-19.3.0-5120.g18432 17 MB/s | 14 MB 00:00 2024-09-21T02:39:57.182 INFO:teuthology.orchestra.run.smithi031.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 156 kB/s | 49 kB 00:00 2024-09-21T02:39:57.207 INFO:teuthology.orchestra.run.smithi031.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 330 kB/s | 16 kB 00:00 2024-09-21T02:39:57.246 INFO:teuthology.orchestra.run.smithi178.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.5 MB/s | 873 kB 00:00 2024-09-21T02:39:57.346 INFO:teuthology.orchestra.run.smithi178.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.5 MB/s | 148 kB 00:00 2024-09-21T02:39:57.421 INFO:teuthology.orchestra.run.smithi178.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T02:39:57.513 INFO:teuthology.orchestra.run.smithi178.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 17 MB/s | 1.5 MB 00:00 2024-09-21T02:39:57.558 INFO:teuthology.orchestra.run.smithi031.stdout:(36/157): ceph-test-19.3.0-5120.g18432218.el9.x 15 MB/s | 40 MB 00:02 2024-09-21T02:39:57.571 INFO:teuthology.orchestra.run.smithi178.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 10 MB/s | 2.3 MB 00:00 2024-09-21T02:39:57.583 INFO:teuthology.orchestra.run.smithi031.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 712 kB/s | 285 kB 00:00 2024-09-21T02:39:57.616 INFO:teuthology.orchestra.run.smithi031.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 1.9 MB/s | 784 kB 00:00 2024-09-21T02:39:57.642 INFO:teuthology.orchestra.run.smithi031.stdout:(39/157): cephfs-top-19.3.0-5120.g18432218.el9. 220 kB/s | 18 kB 00:00 2024-09-21T02:39:57.764 INFO:teuthology.orchestra.run.smithi178.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.0 MB 00:00 2024-09-21T02:39:58.156 INFO:teuthology.orchestra.run.smithi178.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 16 MB/s | 20 MB 00:01 2024-09-21T02:39:58.197 INFO:teuthology.orchestra.run.smithi178.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 609 kB/s | 25 kB 00:00 2024-09-21T02:39:58.267 INFO:teuthology.orchestra.run.smithi031.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 117 kB/s | 80 kB 00:00 2024-09-21T02:39:58.375 INFO:teuthology.orchestra.run.smithi031.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 260 kB/s | 191 kB 00:00 2024-09-21T02:39:58.431 INFO:teuthology.orchestra.run.smithi178.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 16 MB/s | 10 MB 00:00 2024-09-21T02:39:58.481 INFO:teuthology.orchestra.run.smithi178.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T02:39:58.510 INFO:teuthology.orchestra.run.smithi031.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 898 kB/s | 801 kB 00:00 2024-09-21T02:39:58.648 INFO:teuthology.orchestra.run.smithi178.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 17 MB/s | 18 MB 00:01 2024-09-21T02:39:58.674 INFO:teuthology.orchestra.run.smithi178.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 141 kB/s | 27 kB 00:00 2024-09-21T02:39:58.698 INFO:teuthology.orchestra.run.smithi178.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T02:39:58.741 INFO:teuthology.orchestra.run.smithi178.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T02:39:58.765 INFO:teuthology.orchestra.run.smithi178.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T02:39:58.791 INFO:teuthology.orchestra.run.smithi178.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T02:39:58.816 INFO:teuthology.orchestra.run.smithi178.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 9.9 MB/s | 507 kB 00:00 2024-09-21T02:39:58.875 INFO:teuthology.orchestra.run.smithi178.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 773 kB/s | 45 kB 00:00 2024-09-21T02:39:58.875 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:39:58.877 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:39:58.890 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:39:58.891 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:39:58.892 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:39:58.893 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:39:58.895 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:39:58.897 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:39:58.898 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:39:58.950 INFO:teuthology.orchestra.run.smithi178.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T02:39:58.961 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:39:58.976 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:39:58.977 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:39:58.978 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:39:58.979 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:39:58.980 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:39:58.981 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:39:58.982 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:39:58.983 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:39:58.984 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:39:58.985 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:39:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:39:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout:Install 149 Packages 2024-09-21T02:39:58.988 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 8 Packages 2024-09-21T02:39:58.989 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:39:58.989 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 228 M 2024-09-21T02:39:58.990 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-21T02:39:59.025 INFO:teuthology.orchestra.run.smithi178.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 23 MB/s | 5.3 MB 00:00 2024-09-21T02:39:59.050 INFO:teuthology.orchestra.run.smithi178.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.6 MB/s | 161 kB 00:00 2024-09-21T02:39:59.067 INFO:teuthology.orchestra.run.smithi031.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 317 kB/s | 253 kB 00:00 2024-09-21T02:39:59.084 INFO:teuthology.orchestra.run.smithi178.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T02:39:59.110 INFO:teuthology.orchestra.run.smithi178.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T02:39:59.144 INFO:teuthology.orchestra.run.smithi178.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.6 MB/s | 99 kB 00:00 2024-09-21T02:39:59.168 INFO:teuthology.orchestra.run.smithi178.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T02:39:59.176 INFO:teuthology.orchestra.run.smithi031.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 162 kB/s | 106 kB 00:00 2024-09-21T02:39:59.227 INFO:teuthology.orchestra.run.smithi178.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T02:39:59.286 INFO:teuthology.orchestra.run.smithi178.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 22 MB/s | 3.1 MB 00:00 2024-09-21T02:39:59.311 INFO:teuthology.orchestra.run.smithi178.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 410 kB/s | 34 kB 00:00 2024-09-21T02:39:59.327 INFO:teuthology.orchestra.run.smithi031.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-21T02:39:59.336 INFO:teuthology.orchestra.run.smithi178.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T02:39:59.623 INFO:teuthology.orchestra.run.smithi165.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-09-21T02:39:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 195 kB/s | 135 kB 00:00 2024-09-21T02:39:59.770 INFO:teuthology.orchestra.run.smithi178.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 17 MB/s | 7.4 MB 00:00 2024-09-21T02:39:59.837 INFO:teuthology.orchestra.run.smithi178.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T02:39:59.877 INFO:teuthology.orchestra.run.smithi031.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 178 kB/s | 125 kB 00:00 2024-09-21T02:39:59.923 INFO:teuthology.orchestra.run.smithi178.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 573 kB/s | 49 kB 00:00 2024-09-21T02:40:00.006 INFO:teuthology.orchestra.run.smithi165.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.2 MB/s | 873 kB 00:00 2024-09-21T02:40:00.057 INFO:teuthology.orchestra.run.smithi165.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.9 MB/s | 148 kB 00:00 2024-09-21T02:40:00.063 INFO:teuthology.orchestra.run.smithi178.stdout:(34/157): ceph-mgr-dashboard-19.3.0-5120.g18432 19 MB/s | 14 MB 00:00 2024-09-21T02:40:00.088 INFO:teuthology.orchestra.run.smithi178.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 100 kB/s | 16 kB 00:00 2024-09-21T02:40:00.094 INFO:teuthology.orchestra.run.smithi031.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 280 kB/s | 215 kB 00:00 2024-09-21T02:40:00.121 INFO:teuthology.orchestra.run.smithi178.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T02:40:00.172 INFO:teuthology.orchestra.run.smithi178.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 9.2 MB/s | 784 kB 00:00 2024-09-21T02:40:00.197 INFO:teuthology.orchestra.run.smithi178.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 244 kB/s | 18 kB 00:00 2024-09-21T02:40:00.224 INFO:teuthology.orchestra.run.smithi165.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 6.1 MB/s | 5.5 MB 00:00 2024-09-21T02:40:00.374 INFO:teuthology.orchestra.run.smithi165.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 10 MB/s | 1.5 MB 00:00 2024-09-21T02:40:00.499 INFO:teuthology.orchestra.run.smithi165.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 5.2 MB/s | 2.3 MB 00:00 2024-09-21T02:40:00.552 INFO:teuthology.orchestra.run.smithi031.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-21T02:40:00.602 INFO:teuthology.orchestra.run.smithi031.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 818 kB/s | 689 kB 00:00 2024-09-21T02:40:00.661 INFO:teuthology.orchestra.run.smithi031.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-21T02:40:00.674 INFO:teuthology.orchestra.run.smithi165.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-21T02:40:00.705 INFO:teuthology.orchestra.run.smithi178.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T02:40:00.720 INFO:teuthology.orchestra.run.smithi031.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T02:40:00.770 INFO:teuthology.orchestra.run.smithi031.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-21T02:40:00.829 INFO:teuthology.orchestra.run.smithi031.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 991 kB/s | 943 kB 00:00 2024-09-21T02:40:00.879 INFO:teuthology.orchestra.run.smithi031.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-21T02:40:00.946 INFO:teuthology.orchestra.run.smithi031.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-21T02:40:01.108 INFO:teuthology.orchestra.run.smithi165.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 11 MB/s | 20 MB 00:01 2024-09-21T02:40:01.146 INFO:teuthology.orchestra.run.smithi031.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 802 kB/s | 160 kB 00:00 2024-09-21T02:40:01.150 INFO:teuthology.orchestra.run.smithi165.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 606 kB/s | 25 kB 00:00 2024-09-21T02:40:01.172 INFO:teuthology.orchestra.run.smithi178.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 80 kB/s | 80 kB 00:01 2024-09-21T02:40:01.213 INFO:teuthology.orchestra.run.smithi031.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-21T02:40:01.238 INFO:teuthology.orchestra.run.smithi031.stdout:(59/157): libnbd-1.20.2-2.el9.x86_64.rpm 475 kB/s | 171 kB 00:00 2024-09-21T02:40:01.264 INFO:teuthology.orchestra.run.smithi031.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-09-21T02:40:01.275 INFO:teuthology.orchestra.run.smithi165.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 17 MB/s | 10 MB 00:00 2024-09-21T02:40:01.331 INFO:teuthology.orchestra.run.smithi031.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T02:40:01.342 INFO:teuthology.orchestra.run.smithi165.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T02:40:01.372 INFO:teuthology.orchestra.run.smithi031.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-21T02:40:01.409 INFO:teuthology.orchestra.run.smithi165.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 404 kB/s | 27 kB 00:00 2024-09-21T02:40:01.431 INFO:teuthology.orchestra.run.smithi178.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 649 kB/s | 801 kB 00:01 2024-09-21T02:40:01.440 INFO:teuthology.orchestra.run.smithi031.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-21T02:40:01.456 INFO:teuthology.orchestra.run.smithi178.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 260 kB/s | 191 kB 00:00 2024-09-21T02:40:01.476 INFO:teuthology.orchestra.run.smithi165.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 476 kB/s | 32 kB 00:00 2024-09-21T02:40:01.490 INFO:teuthology.orchestra.run.smithi031.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T02:40:01.557 INFO:teuthology.orchestra.run.smithi031.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T02:40:01.568 INFO:teuthology.orchestra.run.smithi165.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 8.2 MB/s | 766 kB 00:00 2024-09-21T02:40:01.599 INFO:teuthology.orchestra.run.smithi031.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-21T02:40:01.632 INFO:teuthology.orchestra.run.smithi031.stdout:(67/157): libtool-2.4.6-46.el9.x86_64.rpm 1.4 MB/s | 581 kB 00:00 2024-09-21T02:40:01.652 INFO:teuthology.orchestra.run.smithi165.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T02:40:01.727 INFO:teuthology.orchestra.run.smithi165.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.7 MB/s | 127 kB 00:00 2024-09-21T02:40:01.819 INFO:teuthology.orchestra.run.smithi165.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 5.4 MB/s | 507 kB 00:00 2024-09-21T02:40:01.931 INFO:teuthology.orchestra.run.smithi178.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-21T02:40:01.949 INFO:teuthology.orchestra.run.smithi031.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-09-21T02:40:02.003 INFO:teuthology.orchestra.run.smithi165.stdout:(18/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 12 MB/s | 18 MB 00:01 2024-09-21T02:40:02.075 INFO:teuthology.orchestra.run.smithi031.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-21T02:40:02.100 INFO:teuthology.orchestra.run.smithi165.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 465 kB/s | 45 kB 00:00 2024-09-21T02:40:02.131 INFO:teuthology.orchestra.run.smithi178.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-21T02:40:02.150 INFO:teuthology.orchestra.run.smithi031.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-09-21T02:40:02.192 INFO:teuthology.orchestra.run.smithi031.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T02:40:02.212 INFO:teuthology.orchestra.run.smithi165.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.3 MB 00:00 2024-09-21T02:40:02.237 INFO:teuthology.orchestra.run.smithi165.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.1 MB/s | 148 kB 00:00 2024-09-21T02:40:02.262 INFO:teuthology.orchestra.run.smithi165.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.1 MB/s | 161 kB 00:00 2024-09-21T02:40:02.288 INFO:teuthology.orchestra.run.smithi165.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.2 MB/s | 321 kB 00:00 2024-09-21T02:40:02.313 INFO:teuthology.orchestra.run.smithi165.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T02:40:02.317 INFO:teuthology.orchestra.run.smithi031.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-21T02:40:02.355 INFO:teuthology.orchestra.run.smithi165.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T02:40:02.380 INFO:teuthology.orchestra.run.smithi165.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T02:40:02.407 INFO:teuthology.orchestra.run.smithi178.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-21T02:40:02.425 INFO:teuthology.orchestra.run.smithi031.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-09-21T02:40:02.431 INFO:teuthology.orchestra.run.smithi165.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T02:40:02.531 INFO:teuthology.orchestra.run.smithi165.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 18 MB/s | 3.1 MB 00:00 2024-09-21T02:40:02.556 INFO:teuthology.orchestra.run.smithi165.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 274 kB/s | 34 kB 00:00 2024-09-21T02:40:02.581 INFO:teuthology.orchestra.run.smithi165.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T02:40:02.690 INFO:teuthology.orchestra.run.smithi178.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 178 kB/s | 135 kB 00:00 2024-09-21T02:40:02.816 INFO:teuthology.orchestra.run.smithi178.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 183 kB/s | 125 kB 00:00 2024-09-21T02:40:02.874 INFO:teuthology.orchestra.run.smithi178.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-21T02:40:02.916 INFO:teuthology.orchestra.run.smithi178.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-21T02:40:02.950 INFO:teuthology.orchestra.run.smithi178.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 652 kB/s | 22 kB 00:00 2024-09-21T02:40:02.992 INFO:teuthology.orchestra.run.smithi178.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-09-21T02:40:03.023 INFO:teuthology.orchestra.run.smithi165.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 21 MB/s | 40 MB 00:01 2024-09-21T02:40:03.025 INFO:teuthology.orchestra.run.smithi178.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-21T02:40:03.042 INFO:teuthology.orchestra.run.smithi178.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-21T02:40:03.074 INFO:teuthology.orchestra.run.smithi165.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.8 MB/s | 292 kB 00:00 2024-09-21T02:40:03.077 INFO:teuthology.orchestra.run.smithi178.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 548 kB/s | 18 kB 00:00 2024-09-21T02:40:03.118 INFO:teuthology.orchestra.run.smithi178.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-21T02:40:03.143 INFO:teuthology.orchestra.run.smithi178.stdout:(56/157): python3-urllib3-1.26.5-6.el9.noarch.r 292 kB/s | 215 kB 00:00 2024-09-21T02:40:03.193 INFO:teuthology.orchestra.run.smithi178.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-21T02:40:03.207 INFO:teuthology.orchestra.run.smithi165.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T02:40:03.236 INFO:teuthology.orchestra.run.smithi178.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T02:40:03.241 INFO:teuthology.orchestra.run.smithi165.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 296 kB/s | 49 kB 00:00 2024-09-21T02:40:03.262 INFO:teuthology.orchestra.run.smithi165.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 302 kB/s | 16 kB 00:00 2024-09-21T02:40:03.269 INFO:teuthology.orchestra.run.smithi178.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T02:40:03.288 INFO:teuthology.orchestra.run.smithi165.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 6.0 MB/s | 285 kB 00:00 2024-09-21T02:40:03.311 INFO:teuthology.orchestra.run.smithi178.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-21T02:40:03.321 INFO:teuthology.orchestra.run.smithi165.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 13 MB/s | 784 kB 00:00 2024-09-21T02:40:03.347 INFO:teuthology.orchestra.run.smithi165.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 314 kB/s | 18 kB 00:00 2024-09-21T02:40:03.428 INFO:teuthology.orchestra.run.smithi178.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-21T02:40:03.428 INFO:teuthology.orchestra.run.smithi178.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-09-21T02:40:03.429 INFO:teuthology.orchestra.run.smithi178.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T02:40:03.504 INFO:teuthology.orchestra.run.smithi178.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T02:40:03.522 INFO:teuthology.orchestra.run.smithi165.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 15 MB/s | 14 MB 00:00 2024-09-21T02:40:03.529 INFO:teuthology.orchestra.run.smithi178.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T02:40:03.554 INFO:teuthology.orchestra.run.smithi178.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T02:40:03.589 INFO:teuthology.orchestra.run.smithi165.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 299 kB/s | 80 kB 00:00 2024-09-21T02:40:03.647 INFO:teuthology.orchestra.run.smithi165.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T02:40:03.673 INFO:teuthology.orchestra.run.smithi165.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T02:40:03.706 INFO:teuthology.orchestra.run.smithi165.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-21T02:40:03.731 INFO:teuthology.orchestra.run.smithi165.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T02:40:03.757 INFO:teuthology.orchestra.run.smithi165.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T02:40:03.782 INFO:teuthology.orchestra.run.smithi165.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T02:40:03.807 INFO:teuthology.orchestra.run.smithi165.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T02:40:03.841 INFO:teuthology.orchestra.run.smithi165.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-09-21T02:40:03.913 INFO:teuthology.orchestra.run.smithi178.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-09-21T02:40:03.983 INFO:teuthology.orchestra.run.smithi165.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-09-21T02:40:04.017 INFO:teuthology.orchestra.run.smithi165.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-09-21T02:40:04.042 INFO:teuthology.orchestra.run.smithi165.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-21T02:40:04.075 INFO:teuthology.orchestra.run.smithi165.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-21T02:40:04.101 INFO:teuthology.orchestra.run.smithi165.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-21T02:40:04.126 INFO:teuthology.orchestra.run.smithi165.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 33 kB 00:00 2024-09-21T02:40:04.151 INFO:teuthology.orchestra.run.smithi165.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T02:40:04.168 INFO:teuthology.orchestra.run.smithi165.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-09-21T02:40:04.202 INFO:teuthology.orchestra.run.smithi165.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T02:40:04.277 INFO:teuthology.orchestra.run.smithi165.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-21T02:40:04.303 INFO:teuthology.orchestra.run.smithi165.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T02:40:04.328 INFO:teuthology.orchestra.run.smithi165.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-09-21T02:40:04.355 INFO:teuthology.orchestra.run.smithi178.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-09-21T02:40:04.361 INFO:teuthology.orchestra.run.smithi165.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T02:40:04.387 INFO:teuthology.orchestra.run.smithi165.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T02:40:04.409 INFO:teuthology.orchestra.run.smithi031.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 1.7 MB/s | 4.9 MB 00:02 2024-09-21T02:40:04.412 INFO:teuthology.orchestra.run.smithi165.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T02:40:04.446 INFO:teuthology.orchestra.run.smithi165.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-21T02:40:04.471 INFO:teuthology.orchestra.run.smithi165.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-21T02:40:04.496 INFO:teuthology.orchestra.run.smithi165.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-21T02:40:04.522 INFO:teuthology.orchestra.run.smithi165.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T02:40:04.526 INFO:teuthology.orchestra.run.smithi031.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 526 kB/s | 61 kB 00:00 2024-09-21T02:40:04.613 INFO:teuthology.orchestra.run.smithi178.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-09-21T02:40:04.630 INFO:teuthology.orchestra.run.smithi165.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-21T02:40:04.672 INFO:teuthology.orchestra.run.smithi165.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-21T02:40:04.689 INFO:teuthology.orchestra.run.smithi178.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-21T02:40:04.714 INFO:teuthology.orchestra.run.smithi165.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-21T02:40:04.730 INFO:teuthology.orchestra.run.smithi178.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T02:40:04.739 INFO:teuthology.orchestra.run.smithi165.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T02:40:04.829 INFO:teuthology.orchestra.run.smithi165.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-21T02:40:04.865 INFO:teuthology.orchestra.run.smithi165.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-21T02:40:04.890 INFO:teuthology.orchestra.run.smithi165.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.4 MB/s | 87 kB 00:00 2024-09-21T02:40:04.901 INFO:teuthology.orchestra.run.smithi031.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-21T02:40:04.941 INFO:teuthology.orchestra.run.smithi165.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T02:40:05.016 INFO:teuthology.orchestra.run.smithi165.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-21T02:40:05.022 INFO:teuthology.orchestra.run.smithi178.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.3 MB/s | 4.9 MB 00:01 2024-09-21T02:40:05.041 INFO:teuthology.orchestra.run.smithi165.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T02:40:05.073 INFO:teuthology.orchestra.run.smithi178.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-09-21T02:40:05.075 INFO:teuthology.orchestra.run.smithi165.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-21T02:40:05.100 INFO:teuthology.orchestra.run.smithi165.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-21T02:40:05.115 INFO:teuthology.orchestra.run.smithi178.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 954 kB/s | 87 kB 00:00 2024-09-21T02:40:05.125 INFO:teuthology.orchestra.run.smithi165.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-21T02:40:05.151 INFO:teuthology.orchestra.run.smithi165.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T02:40:05.157 INFO:teuthology.orchestra.run.smithi178.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T02:40:05.176 INFO:teuthology.orchestra.run.smithi165.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-21T02:40:05.257 INFO:teuthology.orchestra.run.smithi178.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T02:40:05.268 INFO:teuthology.orchestra.run.smithi165.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T02:40:05.276 INFO:teuthology.orchestra.run.smithi031.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 662 kB/s | 248 kB 00:00 2024-09-21T02:40:05.293 INFO:teuthology.orchestra.run.smithi165.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T02:40:05.319 INFO:teuthology.orchestra.run.smithi165.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T02:40:05.352 INFO:teuthology.orchestra.run.smithi165.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T02:40:05.369 INFO:teuthology.orchestra.run.smithi165.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-21T02:40:05.374 INFO:teuthology.orchestra.run.smithi178.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-21T02:40:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-21T02:40:05.403 INFO:teuthology.orchestra.run.smithi165.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T02:40:05.416 INFO:teuthology.orchestra.run.smithi178.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-21T02:40:05.428 INFO:teuthology.orchestra.run.smithi165.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T02:40:05.458 INFO:teuthology.orchestra.run.smithi178.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 462 kB/s | 19 kB 00:00 2024-09-21T02:40:05.487 INFO:teuthology.orchestra.run.smithi165.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-21T02:40:05.493 INFO:teuthology.orchestra.run.smithi031.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-21T02:40:05.512 INFO:teuthology.orchestra.run.smithi165.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-09-21T02:40:05.533 INFO:teuthology.orchestra.run.smithi178.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-21T02:40:05.537 INFO:teuthology.orchestra.run.smithi165.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T02:40:05.571 INFO:teuthology.orchestra.run.smithi165.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-09-21T02:40:05.583 INFO:teuthology.orchestra.run.smithi178.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-21T02:40:05.596 INFO:teuthology.orchestra.run.smithi165.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-21T02:40:05.621 INFO:teuthology.orchestra.run.smithi165.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-09-21T02:40:05.647 INFO:teuthology.orchestra.run.smithi165.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T02:40:05.672 INFO:teuthology.orchestra.run.smithi165.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-21T02:40:05.697 INFO:teuthology.orchestra.run.smithi165.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-09-21T02:40:05.723 INFO:teuthology.orchestra.run.smithi165.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T02:40:05.748 INFO:teuthology.orchestra.run.smithi165.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T02:40:05.773 INFO:teuthology.orchestra.run.smithi165.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T02:40:05.785 INFO:teuthology.orchestra.run.smithi031.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 590 kB/s | 172 kB 00:00 2024-09-21T02:40:05.799 INFO:teuthology.orchestra.run.smithi165.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T02:40:05.824 INFO:teuthology.orchestra.run.smithi165.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-21T02:40:05.849 INFO:teuthology.orchestra.run.smithi165.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-21T02:40:05.891 INFO:teuthology.orchestra.run.smithi165.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-21T02:40:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-21T02:40:05.950 INFO:teuthology.orchestra.run.smithi165.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-09-21T02:40:05.975 INFO:teuthology.orchestra.run.smithi165.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 794 kB/s | 19 kB 00:00 2024-09-21T02:40:06.009 INFO:teuthology.orchestra.run.smithi165.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-21T02:40:06.034 INFO:teuthology.orchestra.run.smithi165.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T02:40:06.059 INFO:teuthology.orchestra.run.smithi165.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T02:40:06.085 INFO:teuthology.orchestra.run.smithi165.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T02:40:06.110 INFO:teuthology.orchestra.run.smithi165.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T02:40:06.169 INFO:teuthology.orchestra.run.smithi165.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T02:40:06.211 INFO:teuthology.orchestra.run.smithi165.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T02:40:06.236 INFO:teuthology.orchestra.run.smithi165.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T02:40:06.262 INFO:teuthology.orchestra.run.smithi165.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T02:40:06.287 INFO:teuthology.orchestra.run.smithi165.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-21T02:40:06.312 INFO:teuthology.orchestra.run.smithi165.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T02:40:06.379 INFO:teuthology.orchestra.run.smithi165.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T02:40:06.405 INFO:teuthology.orchestra.run.smithi165.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-21T02:40:06.417 INFO:teuthology.orchestra.run.smithi178.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-21T02:40:06.430 INFO:teuthology.orchestra.run.smithi165.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T02:40:06.455 INFO:teuthology.orchestra.run.smithi165.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T02:40:06.500 INFO:teuthology.orchestra.run.smithi178.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-21T02:40:06.584 INFO:teuthology.orchestra.run.smithi178.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T02:40:06.606 INFO:teuthology.orchestra.run.smithi165.stdout:(123/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-21T02:40:06.639 INFO:teuthology.orchestra.run.smithi165.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 787 kB/s | 144 kB 00:00 2024-09-21T02:40:06.642 INFO:teuthology.orchestra.run.smithi178.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T02:40:06.664 INFO:teuthology.orchestra.run.smithi165.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 770 kB/s | 18 kB 00:00 2024-09-21T02:40:06.690 INFO:teuthology.orchestra.run.smithi165.stdout:(126/157): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-09-21T02:40:06.701 INFO:teuthology.orchestra.run.smithi178.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-09-21T02:40:06.715 INFO:teuthology.orchestra.run.smithi165.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T02:40:06.740 INFO:teuthology.orchestra.run.smithi165.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T02:40:06.766 INFO:teuthology.orchestra.run.smithi165.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T02:40:06.791 INFO:teuthology.orchestra.run.smithi165.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T02:40:06.816 INFO:teuthology.orchestra.run.smithi165.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T02:40:06.841 INFO:teuthology.orchestra.run.smithi165.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T02:40:06.843 INFO:teuthology.orchestra.run.smithi178.stdout:(87/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-21T02:40:06.868 INFO:teuthology.orchestra.run.smithi178.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-09-21T02:40:06.875 INFO:teuthology.orchestra.run.smithi165.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T02:40:06.886 INFO:teuthology.orchestra.run.smithi031.stdout:(82/157): python3-babel-2.9.1-2.el9.noarch.rpm 1.3 MB/s | 6.0 MB 00:04 2024-09-21T02:40:06.900 INFO:teuthology.orchestra.run.smithi165.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-21T02:40:06.926 INFO:teuthology.orchestra.run.smithi165.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T02:40:06.927 INFO:teuthology.orchestra.run.smithi178.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-21T02:40:06.961 INFO:teuthology.orchestra.run.smithi178.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-21T02:40:06.967 INFO:teuthology.orchestra.run.smithi165.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-09-21T02:40:06.993 INFO:teuthology.orchestra.run.smithi165.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-09-21T02:40:06.994 INFO:teuthology.orchestra.run.smithi178.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T02:40:07.019 INFO:teuthology.orchestra.run.smithi165.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 609 kB/s | 31 kB 00:00 2024-09-21T02:40:07.019 INFO:teuthology.orchestra.run.smithi178.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-21T02:40:07.044 INFO:teuthology.orchestra.run.smithi165.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T02:40:07.061 INFO:teuthology.orchestra.run.smithi178.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T02:40:07.069 INFO:teuthology.orchestra.run.smithi165.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-21T02:40:07.095 INFO:teuthology.orchestra.run.smithi165.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T02:40:07.103 INFO:teuthology.orchestra.run.smithi178.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T02:40:07.120 INFO:teuthology.orchestra.run.smithi165.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T02:40:07.187 INFO:teuthology.orchestra.run.smithi178.stdout:(95/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-21T02:40:07.229 INFO:teuthology.orchestra.run.smithi178.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 866 kB/s | 36 kB 00:00 2024-09-21T02:40:07.240 INFO:teuthology.orchestra.run.smithi165.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-09-21T02:40:07.254 INFO:teuthology.orchestra.run.smithi178.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-21T02:40:07.261 INFO:teuthology.orchestra.run.smithi031.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 668 kB/s | 250 kB 00:00 2024-09-21T02:40:07.279 INFO:teuthology.orchestra.run.smithi178.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-21T02:40:07.320 INFO:teuthology.orchestra.run.smithi165.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 448 kB/s | 90 kB 00:00 2024-09-21T02:40:07.321 INFO:teuthology.orchestra.run.smithi178.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T02:40:07.346 INFO:teuthology.orchestra.run.smithi165.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 918 kB/s | 230 kB 00:00 2024-09-21T02:40:07.363 INFO:teuthology.orchestra.run.smithi165.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-21T02:40:07.380 INFO:teuthology.orchestra.run.smithi178.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-21T02:40:07.396 INFO:teuthology.orchestra.run.smithi165.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T02:40:07.397 INFO:teuthology.orchestra.run.smithi178.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-21T02:40:07.422 INFO:teuthology.orchestra.run.smithi165.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-09-21T02:40:07.439 INFO:teuthology.orchestra.run.smithi178.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T02:40:07.464 INFO:teuthology.orchestra.run.smithi178.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-09-21T02:40:07.472 INFO:teuthology.orchestra.run.smithi165.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T02:40:07.539 INFO:teuthology.orchestra.run.smithi178.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-21T02:40:07.590 INFO:teuthology.orchestra.run.smithi178.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T02:40:07.623 INFO:teuthology.orchestra.run.smithi178.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T02:40:07.648 INFO:teuthology.orchestra.run.smithi165.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T02:40:07.648 INFO:teuthology.orchestra.run.smithi178.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-21T02:40:07.690 INFO:teuthology.orchestra.run.smithi165.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-09-21T02:40:07.690 INFO:teuthology.orchestra.run.smithi178.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-21T02:40:07.732 INFO:teuthology.orchestra.run.smithi178.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 626 kB/s | 26 kB 00:00 2024-09-21T02:40:07.736 INFO:teuthology.orchestra.run.smithi031.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 468 kB/s | 222 kB 00:00 2024-09-21T02:40:07.739 INFO:teuthology.orchestra.run.smithi165.stdout:(152/157): librbd1-19.3.0-5120.g18432218.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-21T02:40:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T02:40:07.765 INFO:teuthology.orchestra.run.smithi165.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 545 kB/s | 41 kB 00:00 2024-09-21T02:40:07.791 INFO:teuthology.orchestra.run.smithi178.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T02:40:07.825 INFO:teuthology.orchestra.run.smithi178.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-21T02:40:07.832 INFO:teuthology.orchestra.run.smithi165.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 9.8 MB/s | 1.8 MB 00:00 2024-09-21T02:40:07.857 INFO:teuthology.orchestra.run.smithi165.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-21T02:40:07.892 INFO:teuthology.orchestra.run.smithi178.stdout:(113/157): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-21T02:40:07.899 INFO:teuthology.orchestra.run.smithi165.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-21T02:40:07.925 INFO:teuthology.orchestra.run.smithi178.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T02:40:07.942 INFO:teuthology.orchestra.run.smithi178.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-21T02:40:07.949 INFO:teuthology.orchestra.run.smithi165.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-21T02:40:07.956 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:40:07.956 INFO:teuthology.orchestra.run.smithi165.stdout:Total 25 MB/s | 228 MB 00:08 2024-09-21T02:40:07.959 INFO:teuthology.orchestra.run.smithi178.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T02:40:07.977 INFO:teuthology.orchestra.run.smithi178.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 857 kB/s | 14 kB 00:00 2024-09-21T02:40:07.978 INFO:teuthology.orchestra.run.smithi031.stdout:(85/157): postgresql-server-13.16-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-09-21T02:40:08.001 INFO:teuthology.orchestra.run.smithi178.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T02:40:08.012 INFO:teuthology.orchestra.run.smithi031.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 280 kB/s | 77 kB 00:00 2024-09-21T02:40:08.035 INFO:teuthology.orchestra.run.smithi178.stdout:(119/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-21T02:40:08.061 INFO:teuthology.orchestra.run.smithi178.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T02:40:08.086 INFO:teuthology.orchestra.run.smithi178.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T02:40:08.095 INFO:teuthology.orchestra.run.smithi031.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-09-21T02:40:08.111 INFO:teuthology.orchestra.run.smithi178.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T02:40:08.137 INFO:teuthology.orchestra.run.smithi178.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T02:40:08.162 INFO:teuthology.orchestra.run.smithi178.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T02:40:08.187 INFO:teuthology.orchestra.run.smithi178.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-21T02:40:08.220 INFO:teuthology.orchestra.run.smithi178.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 694 kB/s | 23 kB 00:00 2024-09-21T02:40:08.246 INFO:teuthology.orchestra.run.smithi178.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-21T02:40:08.271 INFO:teuthology.orchestra.run.smithi178.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-21T02:40:08.313 INFO:teuthology.orchestra.run.smithi178.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-09-21T02:40:08.338 INFO:teuthology.orchestra.run.smithi178.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T02:40:08.355 INFO:teuthology.orchestra.run.smithi178.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-21T02:40:08.381 INFO:teuthology.orchestra.run.smithi178.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T02:40:08.387 INFO:teuthology.orchestra.run.smithi031.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-21T02:40:08.414 INFO:teuthology.orchestra.run.smithi178.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-21T02:40:08.439 INFO:teuthology.orchestra.run.smithi178.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-21T02:40:08.467 INFO:teuthology.orchestra.run.smithi178.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T02:40:08.491 INFO:teuthology.orchestra.run.smithi178.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-21T02:40:08.505 INFO:teuthology.orchestra.run.smithi031.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 547 kB/s | 269 kB 00:00 2024-09-21T02:40:08.516 INFO:teuthology.orchestra.run.smithi178.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-09-21T02:40:08.541 INFO:teuthology.orchestra.run.smithi178.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T02:40:08.566 INFO:teuthology.orchestra.run.smithi178.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T02:40:08.591 INFO:teuthology.orchestra.run.smithi178.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T02:40:08.606 INFO:teuthology.orchestra.run.smithi031.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-09-21T02:40:08.684 INFO:teuthology.orchestra.run.smithi178.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-09-21T02:40:08.705 INFO:teuthology.orchestra.run.smithi031.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-09-21T02:40:08.709 INFO:teuthology.orchestra.run.smithi178.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 636 kB/s | 74 kB 00:00 2024-09-21T02:40:08.734 INFO:teuthology.orchestra.run.smithi178.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T02:40:08.759 INFO:teuthology.orchestra.run.smithi178.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T02:40:08.785 INFO:teuthology.orchestra.run.smithi178.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-21T02:40:08.810 INFO:teuthology.orchestra.run.smithi178.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-21T02:40:08.835 INFO:teuthology.orchestra.run.smithi178.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-21T02:40:08.885 INFO:teuthology.orchestra.run.smithi178.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T02:40:08.888 INFO:teuthology.orchestra.run.smithi031.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 557 kB/s | 279 kB 00:00 2024-09-21T02:40:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-21T02:40:09.086 INFO:teuthology.orchestra.run.smithi178.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T02:40:09.114 INFO:teuthology.orchestra.run.smithi031.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-21T02:40:09.161 INFO:teuthology.orchestra.run.smithi178.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T02:40:09.191 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-21T02:40:09.344 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-21T02:40:09.344 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-21T02:40:09.498 INFO:teuthology.orchestra.run.smithi031.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 576 kB/s | 221 kB 00:00 2024-09-21T02:40:09.598 INFO:teuthology.orchestra.run.smithi031.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-09-21T02:40:09.698 INFO:teuthology.orchestra.run.smithi031.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-09-21T02:40:10.086 INFO:teuthology.orchestra.run.smithi178.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:00 2024-09-21T02:40:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 663 kB/s | 304 kB 00:00 2024-09-21T02:40:10.173 INFO:teuthology.orchestra.run.smithi178.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-09-21T02:40:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-09-21T02:40:10.465 INFO:teuthology.orchestra.run.smithi031.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 479 kB/s | 96 kB 00:00 2024-09-21T02:40:10.666 INFO:teuthology.orchestra.run.smithi031.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-21T02:40:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 69 kB/s | 41 kB 00:00 2024-09-21T02:40:10.808 INFO:teuthology.orchestra.run.smithi031.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 316 kB/s | 45 kB 00:00 2024-09-21T02:40:10.883 INFO:teuthology.orchestra.run.smithi031.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-21T02:40:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 330 kB/s | 249 kB 00:00 2024-09-21T02:40:11.025 INFO:teuthology.orchestra.run.smithi031.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-09-21T02:40:11.075 INFO:teuthology.orchestra.run.smithi031.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T02:40:11.117 INFO:teuthology.orchestra.run.smithi031.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-21T02:40:11.142 INFO:teuthology.orchestra.run.smithi031.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-21T02:40:11.181 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-21T02:40:11.181 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-21T02:40:11.218 INFO:teuthology.orchestra.run.smithi031.stdout:(108/157): python3-numpy-1.20.1-5.el9.x86_64.rp 980 kB/s | 5.1 MB 00:05 2024-09-21T02:40:11.260 INFO:teuthology.orchestra.run.smithi031.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 627 kB/s | 26 kB 00:00 2024-09-21T02:40:11.285 INFO:teuthology.orchestra.run.smithi031.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T02:40:11.310 INFO:teuthology.orchestra.run.smithi031.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T02:40:11.344 INFO:teuthology.orchestra.run.smithi031.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-21T02:40:11.428 INFO:teuthology.orchestra.run.smithi031.stdout:(113/157): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-21T02:40:11.461 INFO:teuthology.orchestra.run.smithi031.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-21T02:40:11.486 INFO:teuthology.orchestra.run.smithi031.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T02:40:11.529 INFO:teuthology.orchestra.run.smithi031.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-09-21T02:40:11.554 INFO:teuthology.orchestra.run.smithi031.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 478 kB/s | 32 kB 00:00 2024-09-21T02:40:11.579 INFO:teuthology.orchestra.run.smithi031.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T02:40:11.612 INFO:teuthology.orchestra.run.smithi178.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 1.5 MB/s | 1.4 MB 00:00 2024-09-21T02:40:11.613 INFO:teuthology.orchestra.run.smithi031.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T02:40:11.646 INFO:teuthology.orchestra.run.smithi031.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-09-21T02:40:11.680 INFO:teuthology.orchestra.run.smithi031.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T02:40:11.705 INFO:teuthology.orchestra.run.smithi031.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T02:40:11.730 INFO:teuthology.orchestra.run.smithi031.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T02:40:11.756 INFO:teuthology.orchestra.run.smithi031.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T02:40:11.781 INFO:teuthology.orchestra.run.smithi031.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-21T02:40:11.806 INFO:teuthology.orchestra.run.smithi031.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 921 kB/s | 23 kB 00:00 2024-09-21T02:40:11.831 INFO:teuthology.orchestra.run.smithi031.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-21T02:40:11.857 INFO:teuthology.orchestra.run.smithi031.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-21T02:40:11.907 INFO:teuthology.orchestra.run.smithi031.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-09-21T02:40:11.929 INFO:teuthology.orchestra.run.smithi178.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:00 2024-09-21T02:40:11.932 INFO:teuthology.orchestra.run.smithi031.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T02:40:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T02:40:11.983 INFO:teuthology.orchestra.run.smithi031.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T02:40:12.008 INFO:teuthology.orchestra.run.smithi031.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-21T02:40:12.050 INFO:teuthology.orchestra.run.smithi031.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-21T02:40:12.075 INFO:teuthology.orchestra.run.smithi031.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-09-21T02:40:12.101 INFO:teuthology.orchestra.run.smithi031.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T02:40:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-21T02:40:12.151 INFO:teuthology.orchestra.run.smithi031.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-21T02:40:12.176 INFO:teuthology.orchestra.run.smithi031.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T02:40:12.202 INFO:teuthology.orchestra.run.smithi031.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T02:40:12.227 INFO:teuthology.orchestra.run.smithi031.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T02:40:12.262 INFO:teuthology.orchestra.run.smithi178.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.1 MB/s | 16 MB 00:05 2024-09-21T02:40:12.268 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:40:12.269 INFO:teuthology.orchestra.run.smithi178.stdout:Total 14 MB/s | 228 MB 00:16 2024-09-21T02:40:12.402 INFO:teuthology.orchestra.run.smithi031.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 426 kB/s | 74 kB 00:00 2024-09-21T02:40:12.469 INFO:teuthology.orchestra.run.smithi031.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T02:40:12.494 INFO:teuthology.orchestra.run.smithi031.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 863 kB/s | 230 kB 00:00 2024-09-21T02:40:12.520 INFO:teuthology.orchestra.run.smithi031.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-21T02:40:12.562 INFO:teuthology.orchestra.run.smithi031.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-09-21T02:40:12.587 INFO:teuthology.orchestra.run.smithi031.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T02:40:12.646 INFO:teuthology.orchestra.run.smithi031.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-21T02:40:12.838 INFO:teuthology.orchestra.run.smithi031.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T02:40:12.929 INFO:teuthology.orchestra.run.smithi031.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T02:40:13.492 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-21T02:40:13.645 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-21T02:40:13.645 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-21T02:40:13.945 INFO:teuthology.orchestra.run.smithi031.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-09-21T02:40:13.987 INFO:teuthology.orchestra.run.smithi031.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-21T02:40:14.504 INFO:teuthology.orchestra.run.smithi031.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 73 kB/s | 41 kB 00:00 2024-09-21T02:40:14.755 INFO:teuthology.orchestra.run.smithi031.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 324 kB/s | 249 kB 00:00 2024-09-21T02:40:15.189 INFO:teuthology.orchestra.run.smithi031.stdout:(155/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.5 MB/s | 16 MB 00:06 2024-09-21T02:40:15.455 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-21T02:40:15.456 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-21T02:40:15.505 INFO:teuthology.orchestra.run.smithi031.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:01 2024-09-21T02:40:15.905 INFO:teuthology.orchestra.run.smithi031.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-09-21T02:40:15.912 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:40:15.912 INFO:teuthology.orchestra.run.smithi031.stdout:Total 10 MB/s | 228 MB 00:22 2024-09-21T02:40:17.185 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-21T02:40:17.358 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-21T02:40:17.358 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-21T02:40:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-21T02:40:17.694 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:40:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:40:17.928 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:40:17.962 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:40:17.996 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:40:18.039 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:40:18.381 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:40:18.497 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:40:18.615 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:40:18.648 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:40:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:40:18.769 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:40:18.798 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:40:18.955 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:40:19.009 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:40:19.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:40:19.143 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:40:19.171 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:40:19.233 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:40:19.255 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:40:19.289 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:40:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:40:19.610 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:40:19.697 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-21T02:40:19.697 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-21T02:40:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:40:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:20.252 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:40:20.285 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:40:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:40:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:40:20.437 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:40:20.473 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:40:20.516 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:40:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:40:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:40:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:40:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:40:20.739 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:40:20.774 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:40:20.855 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:40:20.910 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:40:20.953 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:40:20.982 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:40:21.052 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:40:21.097 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:40:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:40:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:40:21.365 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:40:21.775 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-21T02:40:21.882 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:40:21.899 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:40:21.961 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:40:21.991 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:40:22.021 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:40:22.034 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:40:22.072 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:40:22.113 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:40:22.276 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:40:22.294 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:40:22.325 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:40:22.350 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:40:22.375 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:40:22.413 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:40:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:40:22.479 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:40:22.543 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:40:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:40:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:40:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:40:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:40:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:40:22.758 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:40:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:40:22.884 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:40:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:40:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:40:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:40:23.010 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:40:23.035 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:40:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:40:23.127 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:40:23.156 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:40:23.185 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:40:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:40:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:40:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:40:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:40:23.337 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:40:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:40:23.395 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:40:23.423 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:40:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:40:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:40:23.513 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:40:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:40:23.541 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:40:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:40:23.604 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:40:23.622 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:40:23.625 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:40:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:40:23.662 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:40:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:40:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:40:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:40:23.785 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:40:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:40:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:40:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:40:23.950 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:40:23.952 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:40:23.981 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:40:24.076 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:40:24.099 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:40:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:40:24.202 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:24.275 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:24.487 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:24.596 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:40:24.629 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:40:24.664 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:40:24.728 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:40:24.773 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:40:24.805 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:40:24.848 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:40:24.857 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:40:24.881 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:40:24.905 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:40:24.909 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:40:24.938 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:40:24.977 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:40:25.030 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:40:25.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:40:25.070 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:40:25.093 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:40:25.105 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:40:25.138 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:40:25.186 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:40:25.219 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:40:25.225 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:40:25.254 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:40:25.258 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:40:25.283 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:40:25.351 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:40:25.398 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:40:25.443 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:40:25.471 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:40:25.472 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:40:25.550 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:40:25.605 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:40:25.658 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:40:26.176 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:40:26.192 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:40:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-21T02:40:26.253 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:40:26.283 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:40:26.325 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:40:26.337 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:40:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:40:26.363 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:40:26.405 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:40:26.448 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:40:26.554 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:40:26.608 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:40:26.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:40:26.699 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:40:26.711 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:40:26.744 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:40:26.764 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:40:26.800 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:40:26.836 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:40:26.868 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:40:26.904 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:40:26.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:40:26.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:40:27.030 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:40:27.073 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:40:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:40:27.208 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:40:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:40:27.269 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:40:27.299 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:40:27.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:40:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:40:27.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:40:27.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:40:27.481 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:40:27.506 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:40:27.521 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:40:27.531 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:40:27.550 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:40:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:40:27.580 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:40:27.606 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:40:27.644 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:40:27.672 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:40:27.705 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:40:27.764 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:40:27.783 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:40:27.821 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:40:27.822 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:40:27.850 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:40:27.850 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:40:27.884 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:40:27.923 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:40:27.954 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:40:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:40:27.984 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:40:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:40:28.019 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:40:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:40:28.052 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:40:28.055 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:40:28.076 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:40:28.085 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:40:28.093 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:40:28.125 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:40:28.128 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:40:28.168 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:40:28.215 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:40:28.267 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:40:28.281 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:40:28.298 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:40:28.373 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:40:28.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:40:28.410 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:40:28.446 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:40:28.449 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:40:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:40:28.677 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:28.919 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:40:28.955 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:40:29.018 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:40:29.026 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:40:29.057 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:40:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:40:29.165 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:40:29.167 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:40:29.208 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:40:29.215 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:40:29.248 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:40:29.293 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:40:29.343 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:40:29.346 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:40:29.391 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:40:29.403 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:40:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:40:29.448 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:40:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:40:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:40:29.534 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:40:29.567 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:40:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:40:29.634 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:40:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:40:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:40:29.751 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:40:29.790 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:40:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:40:29.851 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:40:29.867 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:40:29.895 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:40:29.931 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:40:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:40:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:40:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:40:30.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:40:30.342 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:40:30.506 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:40:30.620 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:40:30.905 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:40:30.990 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:40:31.023 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:40:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:40:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:40:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:40:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:40:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:40:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:40:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:40:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:40:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:40:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:40:31.836 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:40:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:40:31.918 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:40:31.945 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:40:31.987 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:40:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:40:32.110 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:40:32.148 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:40:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:40:32.248 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:40:32.249 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:40:32.249 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:40:32.249 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:32.283 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:40:32.312 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:40:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:40:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:40:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:40:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:40:32.449 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:40:32.480 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:40:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:40:32.516 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:40:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:40:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:40:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:40:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:40:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:40:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:40:32.932 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:40:32.961 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:40:32.994 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:40:33.013 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:40:33.034 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:40:33.045 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:40:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:40:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:40:33.116 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:40:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:40:33.146 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:40:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:40:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:40:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:40:33.363 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:40:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:40:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:40:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:40:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:40:34.423 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:40:34.475 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:40:34.511 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:40:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:40:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:40:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:40:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:40:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:40:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:40:35.074 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:40:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:40:35.401 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:35.651 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:40:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:40:36.218 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:36.254 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:40:36.336 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:40:36.337 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:40:36.337 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:40:36.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:36.363 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:40:37.328 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:40:37.411 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:40:37.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:40:37.472 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:40:37.815 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:40:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:40:37.969 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:40:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:40:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:40:38.560 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:40:38.596 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:40:38.650 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:40:38.680 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:40:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:41.152 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:40:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:40:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:40:41.879 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:40:41.879 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:40:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:40:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-09-21T02:40:44.127 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-09-21T02:40:44.128 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:44.305 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:40:44.322 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:40:44.322 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:44.322 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:40:44.322 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:45.917 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:40:45.944 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:40:45.944 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:45.944 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:40:45.944 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:40:45.945 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:40:45.945 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:46.168 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:40:46.191 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:40:46.191 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:46.191 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:40:46.191 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:40:46.191 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:40:46.192 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:46.660 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:40:46.684 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:40:46.684 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:46.684 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:40:46.684 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:40:46.684 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:40:46.684 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:46.799 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:40:46.855 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:40:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:40:47.095 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:40:47.153 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:40:47.222 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:40:47.310 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:40:47.451 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:40:47.495 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:40:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:40:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:40:47.998 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:40:48.029 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:40:48.061 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:40:48.119 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:40:48.160 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:40:48.408 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:40:48.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-09-21T02:40:48.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-09-21T02:40:48.409 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-09-21T02:40:48.409 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-09-21T02:40:48.409 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-09-21T02:40:48.409 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-09-21T02:40:48.409 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-09-21T02:40:48.409 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:48.581 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:40:48.597 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:40:48.597 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:48.597 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:40:48.598 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:50.193 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:40:50.219 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:40:50.219 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:50.220 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:40:50.220 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:40:50.220 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:40:50.220 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:50.453 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:40:50.477 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:40:50.477 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:50.477 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:40:50.477 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:40:50.477 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:40:50.477 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:50.953 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:40:50.977 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:40:50.977 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:50.977 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:40:50.977 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:40:50.977 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:40:50.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:51.075 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:40:51.123 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:40:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:40:51.369 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:40:51.394 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:40:51.412 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:40:51.459 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:40:51.481 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:40:51.578 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:40:51.717 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:40:51.754 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:40:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:40:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:40:52.275 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:40:52.304 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:40:52.336 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:40:52.403 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:40:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:40:52.435 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:40:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:40:52.575 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:40:52.752 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:40:52.777 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:40:52.802 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:40:52.802 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:52.802 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:40:52.802 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:40:52.802 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:40:52.802 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:52.853 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:40:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:40:52.965 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:40:53.027 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:40:53.993 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:40:54.023 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:40:54.047 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:40:54.047 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:54.047 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:40:54.047 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:40:54.047 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:40:54.047 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:54.090 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:40:54.114 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:40:54.114 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:54.114 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:40:54.114 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:54.165 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:40:54.189 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:40:54.189 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:54.189 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:40:54.189 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:40:54.189 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:40:54.189 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:54.246 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:40:54.246 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-21T02:40:54.246 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-21T02:40:54.246 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-21T02:40:54.246 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-21T02:40:54.246 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-21T02:40:54.247 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-21T02:40:54.247 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-21T02:40:54.247 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:40:54.423 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:40:54.440 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:40:54.440 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:54.440 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:40:54.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:40:54.508 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:40:54.532 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:40:54.532 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:54.532 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:40:54.532 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:40:54.532 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:40:54.532 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:40:55.593 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:40:55.649 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:40:56.093 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:40:56.122 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:40:56.122 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:56.123 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:40:56.123 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:40:56.123 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:40:56.123 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:40:56.369 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:40:56.394 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:40:56.394 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:56.395 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:40:56.395 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:40:56.395 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:40:56.395 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:40:56.607 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:40:56.641 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:40:56.754 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:40:56.887 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:40:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:40:56.912 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:56.912 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:40:56.912 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:40:56.912 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:40:56.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:40:56.925 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:40:56.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:40:56.975 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:40:56.975 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:56.975 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:40:56.975 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:40:56.975 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:40:56.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:40:57.027 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:40:57.038 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:40:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:40:57.137 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:40:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:40:57.183 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:40:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:40:57.413 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:40:57.496 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:40:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:40:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:40:57.796 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:40:57.828 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:40:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:40:58.142 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:40:58.171 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:40:58.194 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:40:58.194 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:58.194 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:40:58.194 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:40:58.195 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:40:58.195 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:58.238 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:40:58.261 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:40:58.261 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:58.261 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:40:58.261 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:40:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:40:58.315 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:40:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:40:58.337 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:40:58.337 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:58.337 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:40:58.337 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:40:58.337 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:40:58.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:58.406 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:40:58.435 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:40:58.521 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:40:58.562 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:40:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:40:58.643 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:40:58.664 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:40:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:40:58.688 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:40:58.688 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:40:58.688 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:40:58.688 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:40:58.688 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:40:58.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:40:58.857 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:40:58.910 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:40:58.939 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:40:58.969 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:40:59.024 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:40:59.683 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:40:59.722 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:40:59.743 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:41:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:41:00.747 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:41:00.825 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:41:00.875 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:41:00.899 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:41:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:41:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:41:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:41:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:41:01.084 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:41:01.109 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:41:01.118 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:01.118 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:41:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:41:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:41:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:41:02.685 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:41:02.726 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:41:02.766 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:41:02.799 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:41:02.826 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:41:02.955 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:41:02.991 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:41:03.009 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:41:03.056 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:41:03.086 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:41:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:41:03.124 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:41:03.171 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:41:03.291 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:41:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:41:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:41:03.394 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:41:03.394 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:41:03.394 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:41:03.394 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:41:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:41:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:41:03.567 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:41:03.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:41:03.786 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:03.818 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:03.838 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:04.655 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:41:04.674 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:41:04.688 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:41:04.713 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:41:04.713 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:41:04.713 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:41:04.714 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:41:04.714 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:41:04.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:04.755 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:41:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:41:04.780 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:41:04.780 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:41:04.780 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:04.831 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:41:04.839 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:41:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:41:04.858 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:41:04.858 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:41:04.858 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:41:04.858 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:41:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:04.880 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:41:04.962 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:41:05.000 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:41:05.026 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:41:05.051 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:41:05.051 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:41:05.136 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:41:05.137 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:41:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:41:05.212 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:41:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:41:05.215 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:41:05.216 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:41:05.216 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:41:05.216 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:41:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:05.236 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:41:05.242 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:05.242 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:05.422 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:41:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:41:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:41:07.491 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:41:07.492 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:41:07.493 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:41:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:41:07.495 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:41:07.496 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:41:07.497 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:41:07.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:41:07.499 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:41:07.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:41:07.501 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:41:07.502 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:41:07.503 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:41:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:41:09.511 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:41:09.560 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:41:09.592 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:41:09.619 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:41:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:41:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:41:09.929 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:41:09.958 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:41:09.996 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:41:10.276 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:10.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:41:10.276 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-09-21T02:41:10.276 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:10.276 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.277 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.278 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:41:10.279 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:41:10.280 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:41:10.281 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:41:10.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:41:10.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:41:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:41:10.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:41:10.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:41:10.287 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:41:10.288 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-21T02:41:10.656 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:10.695 DEBUG:teuthology.parallel:result is None 2024-09-21T02:41:10.702 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:10.723 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:41:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:41:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:41:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:41:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:41:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:41:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:41:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:41:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:41:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:41:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:41:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:41:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:41:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:41:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:41:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:41:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:41:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:41:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:41:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:41:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:41:11.859 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:41:11.901 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:41:11.934 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:41:11.960 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:41:11.960 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:41:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:41:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:41:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:41:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:41:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:41:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:41:12.407 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:41:12.475 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:13.985 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:13.986 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.987 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.988 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.989 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:41:13.990 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:41:13.991 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:41:13.992 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:41:13.993 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:41:13.994 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.995 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:41:13.996 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:41:13.997 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:41:13.998 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:41:13.998 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:41:13.998 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-21T02:41:14.395 DEBUG:teuthology.parallel:result is None 2024-09-21T02:41:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:41:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:41:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:41:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:41:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:41:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:41:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:41:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:41:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:41:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:41:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:41:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:41:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:41:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:41:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:41:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:41:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:41:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:41:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:41:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:41:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:41:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:41:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:41:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:41:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:41:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:41:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:41:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:41:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:41:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:41:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:41:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:41:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:41:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:41:21.758 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-21T02:41:22.179 DEBUG:teuthology.parallel:result is None 2024-09-21T02:41:22.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:41:22.327 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:41:22.349 INFO:teuthology.orchestra.run.smithi031.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:41:22.350 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:41:22.350 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:41:22.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:41:22.496 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:41:22.522 INFO:teuthology.orchestra.run.smithi165.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:41:22.523 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:41:22.523 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:41:22.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:41:22.659 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:41:22.686 INFO:teuthology.orchestra.run.smithi178.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:41:22.686 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:41:22.686 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:41:22.689 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T02:41:22.689 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:41:22.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:41:22.720 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:41:22.720 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:41:22.753 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:41:22.753 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:41:22.785 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T02:41:22.786 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:41:22.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:41:22.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:41:22.881 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:41:22.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:41:22.912 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:41:22.981 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:41:22.981 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:41:23.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:41:23.080 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T02:41:23.080 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:41:23.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:41:23.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:41:23.177 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:41:23.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:41:23.209 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:41:23.277 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:41:23.277 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:41:23.307 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:41:23.375 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T02:41:23.375 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:41:23.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:41:23.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:41:23.472 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:41:23.472 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:41:23.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:41:23.562 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:41:23.563 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:41:23.592 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:41:23.660 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T02:41:23.762 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T02:41:23.762 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:41:23.763 INFO:tasks.cephadm:Cluster fsid is fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:41:23.763 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T02:41:23.763 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.165', 'mon.c': '172.21.15.178'} 2024-09-21T02:41:23.763 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-09-21T02:41:23.763 INFO:tasks.cephadm:First mgr is x 2024-09-21T02:41:23.763 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T02:41:23.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-09-21T02:41:23.795 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-09-21T02:41:23.821 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-09-21T02:41:23.851 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T02:41:23.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:41:23.995 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T02:41:24.138 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:41:24.140 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T02:41:24.140 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T02:41:24.140 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:24.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 02:41 /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:24.693 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.245 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 02:41 /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.246 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.851 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 02:41 /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.852 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.872 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.891 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T02:41:25.925 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T02:41:25.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T02:41:25.928 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T02:41:25.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T02:41:27.447 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:41:27.571 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:41:27.686 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:42:00.775 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-21T02:42:00.776 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:42:00.776 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T02:42:00.776 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-09-21T02:42:00.776 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T02:42:00.776 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-21T02:42:00.776 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-21T02:42:01.551 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-21T02:42:01.551 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:42:01.552 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T02:42:01.552 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-09-21T02:42:01.552 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T02:42:01.552 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-21T02:42:01.552 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T02:42:01.606 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:42:01.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-09-21T02:42:01.678 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-09-21T02:42:01.711 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-09-21T02:42:01.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-09-21T02:42:01.765 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-09-21T02:42:01.793 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-09-21T02:42:01.817 INFO:tasks.cephadm:Writing seed config... 2024-09-21T02:42:01.818 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T02:42:01.819 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T02:42:01.820 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T02:42:01.821 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:42:01.821 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T02:42:01.837 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fdd37b14-77c2-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 2024-09-21T02:42:01.837 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T02:42:01.880 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T02:42:01.923 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T02:42:01.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:42:02.135 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:42:02.135 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-09-21T02:42:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T02:42:02.135 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-09-21T02:42:02.159 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-21T02:42:02.159 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-09-21T02:42:02.159 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-09-21T02:42:02.167 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T02:42:02.167 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T02:42:02.174 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T02:42:02.174 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T02:42:02.181 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-21T02:42:02.188 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-21T02:42:02.188 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-21T02:42:02.188 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-09-21T02:42:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-21T02:42:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T02:42:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-09-21T02:42:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-09-21T02:42:02.218 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T02:42:02.218 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T02:42:02.224 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T02:42:02.224 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T02:42:02.231 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-21T02:42:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-21T02:42:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-21T02:42:02.238 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-09-21T02:42:02.238 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:02.238 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 139797902805360 on /run/cephadm/fdd37b14-77c2-11ef-baf5-efdc52797490.lock 2024-09-21T02:42:02.238 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 139797902805360 acquired on /run/cephadm/fdd37b14-77c2-11ef-baf5-efdc52797490.lock 2024-09-21T02:42:02.238 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-09-21T02:42:02.238 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-09-21T02:42:02.239 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-21T02:42:02.242 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-09-21T02:42:02.242 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-09-21T02:42:02.245 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T02:42:02.245 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T02:42:02.245 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1786sec hoplimit 64 pref medium 2024-09-21T02:42:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T02:42:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T02:42:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T02:42:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T02:42:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-21T02:42:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T02:42:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-21T02:42:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-21T02:42:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T02:42:02.249 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T02:42:02.249 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T02:42:02.693 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T02:42:03.613 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T02:42:03.614 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T02:42:03.614 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T02:42:04.595 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-09-21T02:42:04.595 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-09-21T02:42:05.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQB8Mu5m9oGCOBAAkVp3z2Gq7ZHPDNODLjjGHQ== 2024-09-21T02:42:06.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQB9Mu5mywiXMhAAp6v31ViJbinjVGCPtvvnpg== 2024-09-21T02:42:07.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQB+Mu5m6dDNKhAAsLVCW/uknbgi3HGHxwrQ6w== 2024-09-21T02:42:07.404 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-09-21T02:42:08.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:42:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-09-21T02:42:15.671 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-09-21T02:42:16.189 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T02:42:16.576 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T02:42:17.080 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490.target. 2024-09-21T02:42:17.081 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490.target. 2024-09-21T02:42:17.837 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a 2024-09-21T02:42:17.837 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T02:42:18.261 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490.target.wants/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@.service. 2024-09-21T02:42:18.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 podman[30825]: 2024-09-21 02:42:18.409714679 +0000 UTC m=+0.027678540 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:42:18.807 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:42:18.807 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T02:42:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:42:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T02:42:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T02:42:18.814 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T02:42:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-09-21T02:42:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-09-21T02:42:18.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 podman[30825]: 2024-09-21 02:42:18.700809324 +0000 UTC m=+0.318773173 container init 87ccec974ed0361b6e04c4bfa5515cf10661f9f2e896ad9ce233ac3826f063af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T02:42:18.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 podman[30825]: 2024-09-21 02:42:18.703908674 +0000 UTC m=+0.321872522 container start 87ccec974ed0361b6e04c4bfa5515cf10661f9f2e896ad9ce233ac3826f063af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T02:42:18.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 ceph-mon[30843]: mkfs fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:18.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 ceph-mon[30843]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T02:42:18.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 bash[30825]: 87ccec974ed0361b6e04c4bfa5515cf10661f9f2e896ad9ce233ac3826f063af 2024-09-21T02:42:18.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:18 smithi031 systemd[1]: Started Ceph mon.a for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:42:19.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T02:42:19.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:19.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T02:42:19.854 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.546234s) 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-09-21T02:42:19.855 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: monmap epoch 1 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: last_changed 2024-09-21T02:42:07.738925+0000 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: min_mon_release 19 (squid) 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: election_strategy: 1 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: fsmap 2024-09-21T02:42:19.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T02:42:19.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: mgrmap e1: no daemons active 2024-09-21T02:42:19.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:19 smithi031 ceph-mon[30843]: from='client.? 172.21.15.31:0/3738778364' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T02:42:21.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:20 smithi031 ceph-mon[30843]: from='client.? 172.21.15.31:0/2899375402' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T02:42:21.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:20 smithi031 ceph-mon[30843]: from='client.? 172.21.15.31:0/2899375402' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T02:42:21.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T02:42:21.020 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-09-21T02:42:22.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:21 smithi031 ceph-mon[30843]: from='client.? 172.21.15.31:0/1741149346' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:42:22.083 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-09-21T02:42:22.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:22 smithi031 systemd[1]: Stopping Ceph mon.a for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:42:22.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:22 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a[30839]: 2024-09-21T02:42:22.365+0000 7fbca5efd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:42:22.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:22 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a[30839]: 2024-09-21T02:42:22.365+0000 7fbca5efd640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T02:42:22.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:22 smithi031 podman[31087]: 2024-09-21 02:42:22.529850085 +0000 UTC m=+0.275456706 container died 87ccec974ed0361b6e04c4bfa5515cf10661f9f2e896ad9ce233ac3826f063af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T02:42:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:22 smithi031 podman[31087]: 2024-09-21 02:42:22.907163275 +0000 UTC m=+0.652769883 container cleanup 87ccec974ed0361b6e04c4bfa5515cf10661f9f2e896ad9ce233ac3826f063af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T02:42:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:22 smithi031 bash[31087]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a 2024-09-21T02:42:23.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:23 smithi031 podman[31100]: 2024-09-21 02:42:23.208057966 +0000 UTC m=+0.670848790 container remove 87ccec974ed0361b6e04c4bfa5515cf10661f9f2e896ad9ce233ac3826f063af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:42:23.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:23 smithi031 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T02:42:23.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:23 smithi031 systemd[1]: Stopped Ceph mon.a for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:42:23.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:23 smithi031 systemd[1]: Starting Ceph mon.a for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:42:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:23 smithi031 podman[31189]: 2024-09-21 02:42:23.458230829 +0000 UTC m=+0.016556949 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:42:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:24 smithi031 podman[31189]: 2024-09-21 02:42:24.425450826 +0000 UTC m=+0.983776929 container create ecef9b21124f577615900b8fe09289a90c81ead6b1de62e2fa7ac7f53c59cfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 podman[31189]: 2024-09-21 02:42:26.352094664 +0000 UTC m=+2.910420782 container init ecef9b21124f577615900b8fe09289a90c81ead6b1de62e2fa7ac7f53c59cfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 podman[31189]: 2024-09-21 02:42:26.35621407 +0000 UTC m=+2.914540174 container start ecef9b21124f577615900b8fe09289a90c81ead6b1de62e2fa7ac7f53c59cfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: pidfile_write: ignore empty --pid-file 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: load: jerasure load: lrc 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: RocksDB version: 7.9.2 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Git sha 0 2024-09-21T02:42:26.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: DB SUMMARY 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: DB Session ID: MKYG6013KY1WG2O118GZ 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: CURRENT file: CURRENT 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95400 ; 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.error_if_exists: 0 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.create_if_missing: 0 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.paranoid_checks: 1 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T02:42:26.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.env: 0x55d918ab4d60 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.info_log: 0x55d91d655360 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.statistics: (nil) 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.use_fsync: 0 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_log_file_size: 0 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T02:42:26.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.allow_fallocate: 1 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.use_direct_reads: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.db_log_dir: 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.wal_dir: 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T02:42:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.write_buffer_manager: 0x55d91d659d60 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.unordered_write: 0 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T02:42:26.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T02:42:26.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T02:42:26.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.row_cache: None 2024-09-21T02:42:26.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.wal_filter: None 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.two_write_queues: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.wal_compression: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.atomic_flush: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T02:42:26.841 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.log_readahead_size: 0 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_background_jobs: 2 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_background_compactions: -1 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_subcompactions: 1 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T02:42:26.842 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_open_files: -1 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_background_flushes: -1 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Compression algorithms supported: 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kZSTD supported: 0 2024-09-21T02:42:26.843 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kXpressCompression supported: 0 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kBZip2Compression supported: 0 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kLZ4Compression supported: 1 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kZlibCompression supported: 1 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: kSnappyCompression supported: 1 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.merge_operator: 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_filter: None 2024-09-21T02:42:26.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d91d654fa0) 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-09-21T02:42:26.845 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x55d919619610 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-09-21T02:42:26.846 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T02:42:26.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression: NoCompression 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.num_levels: 7 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T02:42:26.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T02:42:26.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T02:42:26.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T02:42:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.inplace_update_support: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.bloom_locality: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.max_successive_merges: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.ttl: 2592000 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T02:42:26.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.enable_blob_files: false 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.min_blob_size: 0 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T02:42:26.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7b7ca1cd-ea3f-4e73-b1b4-fba5bf0b9ab2 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886546403149, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886546405079, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91587, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89628, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11738, "raw_average_key_size": 48, "raw_value_size": 82969, "raw_average_value_size": 340, "num_data_blocks": 12, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726886546, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7b7ca1cd-ea3f-4e73-b1b4-fba5bf0b9ab2", "db_session_id": "MKYG6013KY1WG2O118GZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886546405206, "job": 1, "event": "recovery_finished"} 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d91d678a80 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: DB pointer 0x55d91e41e000 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 91.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 91.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-21T02:42:26.855 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 17.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 17.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:42:26.856 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x55d919619610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???) e1 preinit fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).mds e1 new map 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).mds e1 print_map 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: btime 2024-09-21T02:42:18:754388+0000 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:42:26.857 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: monmap epoch 1 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: last_changed 2024-09-21T02:42:07.738925+0000 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: min_mon_release 19 (squid) 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: election_strategy: 1 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:42:26.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: fsmap 2024-09-21T02:42:26.859 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T02:42:26.859 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:26 smithi031 ceph-mon[31209]: mgrmap e1: no daemons active 2024-09-21T02:42:27.268 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T02:42:27.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:27 smithi031 bash[31189]: ecef9b21124f577615900b8fe09289a90c81ead6b1de62e2fa7ac7f53c59cfa9 2024-09-21T02:42:27.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:27 smithi031 systemd[1]: Started Ceph mon.a for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:42:28.697 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T02:42:28.718 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:42:28.718 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-09-21T02:42:28.719 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T02:42:28.719 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T02:42:28.978 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x 2024-09-21T02:42:28.978 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T02:42:29.210 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490.target.wants/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@.service. 2024-09-21T02:42:29.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:29 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/123112506' entity='client.admin' 2024-09-21T02:42:29.334 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:29 smithi031 systemd[1]: Starting Ceph mgr.x for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:42:29.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:29 smithi031 podman[31419]: 2024-09-21 02:42:29.346773367 +0000 UTC m=+0.016769796 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:42:30.240 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:29 smithi031 podman[31419]: 2024-09-21 02:42:29.945950958 +0000 UTC m=+0.615947407 container create 1b306d23b1662a15555937d3c9c6076c353bd3b04f5f0a0b8d9e22a4762bbc34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T02:42:30.240 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:30 smithi031 podman[31419]: 2024-09-21 02:42:30.165232913 +0000 UTC m=+0.835229319 container init 1b306d23b1662a15555937d3c9c6076c353bd3b04f5f0a0b8d9e22a4762bbc34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T02:42:30.241 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:30 smithi031 podman[31419]: 2024-09-21 02:42:30.168537165 +0000 UTC m=+0.838533573 container start 1b306d23b1662a15555937d3c9c6076c353bd3b04f5f0a0b8d9e22a4762bbc34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:42:30.256 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:42:30.256 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T02:42:30.264 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:42:30.264 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T02:42:30.264 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T02:42:30.264 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T02:42:30.273 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:42:30.273 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T02:42:30.281 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:42:30.281 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T02:42:30.281 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T02:42:30.281 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T02:42:30.281 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-09-21T02:42:30.281 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-09-21T02:42:30.491 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:30 smithi031 bash[31419]: 1b306d23b1662a15555937d3c9c6076c353bd3b04f5f0a0b8d9e22a4762bbc34 2024-09-21T02:42:30.491 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:30 smithi031 systemd[1]: Started Ceph mgr.x for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:42:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:30 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2175784112' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T02:42:31.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:31.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:42:31.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:42:31.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T02:42:31.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T02:42:31.402 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T02:42:31.403 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T02:42:31.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T02:42:31.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:31.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T02:42:31.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T02:42:31.404 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T02:42:31.408 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T02:42:18:754388+0000", 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T02:42:31.409 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T02:42:18.755241+0000", 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:31.410 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T02:42:31.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:42:31.411 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-09-21T02:42:34.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:33 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/247648709' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T02:42:34.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T02:42:34.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T02:42:34.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T02:42:34.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T02:42:34.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T02:42:34.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T02:42:34.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T02:42:18:754388+0000", 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T02:42:34.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T02:42:18.755241+0000", 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:42:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: Activating manager daemon x 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: mgrmap e2: x(active, starting, since 0.00348195s) 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: Manager daemon x is now available 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:42:35.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' 2024-09-21T02:42:35.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:42:35.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' 2024-09-21T02:42:35.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:35 smithi031 ceph-mon[31209]: from='mgr.14100 172.21.15.31:0/2288201885' entity='mgr.x' 2024-09-21T02:42:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:37 smithi031 ceph-mon[31209]: mgrmap e3: x(active, since 1.00766s) 2024-09-21T02:42:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:37 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4157812770' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T02:42:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:42:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:42:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T02:42:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T02:42:37.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T02:42:37.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T02:42:37.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T02:42:37.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T02:42:37.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T02:42:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T02:42:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T02:42:18:754388+0000", 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T02:42:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T02:42:18.755241+0000", 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:42:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-09-21T02:42:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:38 smithi031 ceph-mon[31209]: mgrmap e4: x(active, since 2s) 2024-09-21T02:42:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4221317493' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T02:42:38.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:38.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T02:42:38.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:38.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T02:42:38.981 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T02:42:38.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T02:42:38.982 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-09-21T02:42:40.031 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:39 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3632201028' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T02:42:40.031 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:39 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ignoring --setuser ceph since I am not root 2024-09-21T02:42:40.031 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:39 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ignoring --setgroup ceph since I am not root 2024-09-21T02:42:41.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:40 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3632201028' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T02:42:41.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:40 smithi031 ceph-mon[31209]: mgrmap e5: x(active, since 4s) 2024-09-21T02:42:41.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-21T02:42:41.608 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-09-21T02:42:41.915 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:41 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3603636542' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: Active manager daemon x restarted 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: Activating manager daemon x 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: mgrmap e6: x(active, starting, since 0.0039422s) 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:42:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:45 smithi031 ceph-mon[31209]: Manager daemon x is now available 2024-09-21T02:42:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:42:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T02:42:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T02:42:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:42:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-09-21T02:42:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying orchestrator module is enabled... 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: mgrmap e7: x(active, since 1.00531s) 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: Found migration_current of "None". Setting to last migration. 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:42:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:42:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:42:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:46 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:42:47.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:42:47.471+0000 7fde4457d640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:02:42:47] ENGINE Error in HTTPServer.serve 2024-09-21T02:42:47.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: Traceback (most recent call last): 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._connections.run(self.expiration_interval) 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._run(expiration_interval) 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: s = self.context.wrap_socket( 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:42:47.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: return self.sslsocket_class._create( 2024-09-21T02:42:47.836 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:42:47.836 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self.do_handshake() 2024-09-21T02:42:47.836 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:42:47.836 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._sslobj.do_handshake() 2024-09-21T02:42:47.836 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:42:47.836 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:42:48.340 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: mgrmap e8: x(active, since 2s) 2024-09-21T02:42:48.340 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: [21/Sep/2024:02:42:47] ENGINE Bus STARTING 2024-09-21T02:42:48.340 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2550799311' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T02:42:48.340 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: [21/Sep/2024:02:42:47] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: [21/Sep/2024:02:42:47] ENGINE Error in HTTPServer.serve 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:42:48.341 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:42:48.342 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: [21/Sep/2024:02:42:47] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T02:42:48.343 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: [21/Sep/2024:02:42:47] ENGINE Bus STARTED 2024-09-21T02:42:48.343 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:48 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:42:48.669 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T02:42:48.670 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-09-21T02:42:49.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:49 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2550799311' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T02:42:49.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:49 smithi031 ceph-mon[31209]: mgrmap e9: x(active, since 3s) 2024-09-21T02:42:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:50 smithi031 ceph-mon[31209]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:42:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:50 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:50 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:42:51.181 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T02:42:51.181 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-09-21T02:42:51.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:51 smithi031 ceph-mon[31209]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:42:52.558 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: Generating public/private rsa key pair. 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: Your identification has been saved in /tmp/tmpnbn6xojr/key 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: Your public key has been saved in /tmp/tmpnbn6xojr/key.pub 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: The key fingerprint is: 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: SHA256:wJOKpiF5qfJ4887ID5Kiee25vSjigEC4wSQctfLvCjY ceph-fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: The key's randomart image is: 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: +---[RSA 3072]----+ 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |ooo. | 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |=. .. . | 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |o+ . = | 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |.+oo . o | 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |* =.. S | 2024-09-21T02:42:52.559 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |+B . | 2024-09-21T02:42:52.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |XE.. . | 2024-09-21T02:42:52.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |B*B+o+ | 2024-09-21T02:42:52.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: |==+O@oo. | 2024-09-21T02:42:52.560 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:42:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: +----[SHA256]-----+ 2024-09-21T02:42:53.421 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:53 smithi031 ceph-mon[31209]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:42:53.421 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:53 smithi031 ceph-mon[31209]: Generating ssh key... 2024-09-21T02:42:53.421 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:53 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:53.421 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:53 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:53.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:42:53.976 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:42:53.976 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T02:42:53.984 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-09-21T02:42:54.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:54 smithi031 ceph-mon[31209]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:42:55.292 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:55 smithi031 ceph-mon[31209]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:42:56.459 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:56 smithi031 ceph-mon[31209]: Deploying cephadm binary to smithi031 2024-09-21T02:42:59.102 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-09-21T02:42:59.102 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-09-21T02:42:59.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:59 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:42:59.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:59 smithi031 ceph-mon[31209]: Added host smithi031 2024-09-21T02:42:59.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:42:59 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:00.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:00 smithi031 ceph-mon[31209]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:00.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:00 smithi031 ceph-mon[31209]: Saving service mon spec with placement count:5 2024-09-21T02:43:00.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:00 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:43:00.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:00 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:43:01.230 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T02:43:01.230 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-09-21T02:43:02.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:02 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:43:02.565 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:02 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:43:03.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T02:43:03.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:03 smithi031 ceph-mon[31209]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:03.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:03 smithi031 ceph-mon[31209]: Saving service mgr spec with placement count:2 2024-09-21T02:43:03.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:03 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:43:03.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:03 smithi031 ceph-mon[31209]: from='mgr.14118 172.21.15.31:0/1710293916' entity='mgr.x' 2024-09-21T02:43:05.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:04 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/337986866' entity='client.admin' 2024-09-21T02:43:06.270 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:06 smithi031 ceph-mon[31209]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:06.270 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:06 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2944161039' entity='client.admin' 2024-09-21T02:43:06.531 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-09-21T02:43:08.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:08 smithi031 ceph-mon[31209]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:08.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:08 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3445289525' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T02:43:08.334 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ignoring --setuser ceph since I am not root 2024-09-21T02:43:08.334 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ignoring --setgroup ceph since I am not root 2024-09-21T02:43:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:09 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3445289525' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T02:43:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:09 smithi031 ceph-mon[31209]: mgrmap e10: x(active, since 23s) 2024-09-21T02:43:09.952 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:43:09.952 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T02:43:09.952 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T02:43:09.952 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T02:43:09.953 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T02:43:09.953 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:43:09.953 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-21T02:43:09.953 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 10... 2024-09-21T02:43:10.201 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:10 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1305244407' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: Active manager daemon x restarted 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: Activating manager daemon x 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: mgrmap e11: x(active, starting, since 0.00398516s) 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:43:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:43:13.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:13 smithi031 ceph-mon[31209]: Manager daemon x is now available 2024-09-21T02:43:14.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:14 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:14.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:14 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:43:14.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:14 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:43:14.647 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:14 smithi031 ceph-mon[31209]: mgrmap e12: x(active, since 1.00787s) 2024-09-21T02:43:14.932 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:43:14.932 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T02:43:14.933 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T02:43:14.933 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:43:14.933 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 10 is available 2024-09-21T02:43:14.933 INFO:teuthology.orchestra.run.smithi031.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T02:43:15.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-mon[31209]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T02:43:15.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-mon[31209]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T02:43:15.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:43:15.617+0000 7fa87bb4f640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:02:43:15] ENGINE Error in HTTPServer.serve 2024-09-21T02:43:15.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: Traceback (most recent call last): 2024-09-21T02:43:15.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:43:15.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._connections.run(self.expiration_interval) 2024-09-21T02:43:15.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:43:15.834 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._run(expiration_interval) 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: s = self.context.wrap_socket( 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: return self.sslsocket_class._create( 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self.do_handshake() 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._sslobj.do_handshake() 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:43:15.835 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:43:15 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:43:16.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: mgrmap e13: x(active, since 2s) 2024-09-21T02:43:16.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: [21/Sep/2024:02:43:15] ENGINE Bus STARTING 2024-09-21T02:43:16.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:16.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: [21/Sep/2024:02:43:15] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T02:43:16.646 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: [21/Sep/2024:02:43:15] ENGINE Error in HTTPServer.serve 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-21T02:43:16.647 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: [21/Sep/2024:02:43:15] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: [21/Sep/2024:02:43:15] ENGINE Bus STARTED 2024-09-21T02:43:16.648 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:16 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:17.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUVP7cahzEzuFjJzv330bhVajvF2MwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAyNDMxNloXDTM0\nMDkyMjAyNDMxNlowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjMxMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAulPNIp4Hqz674BFRRpw1PFccnG0zUnBEIqOi\naRyqpxvqFKe0N5Q5c8hP1hgyaj9Ym2gv1QcYORQ1XKu0aPj9li+ffL7660D4Id4G\nJuzGBDxyDINqorkt02VcA/CiyzOHi4B05qISNuCCloquysJP3p4o8petQruXVXXL\nsFi7JXPlval+yUVXMgHQ0inNfm2FBI6kGn+Fe0EDT7wOmRkTiqGUuxe1Yh8DS1gw\nEnZmJXZxLQKq1495krvIoQvueZex8apBB+MvVqq1bnStiOdagDGIcDmoe1haqVS/\nevP/x2dRVwMndrODEDLQKaFDYQolfB3cmfLj/WZBp/T0b+vQ4g6HzPA4Zyu2GU9f\ntN/IVa39sGEKoAwrXqWwZZ2XwMjQK9zp0IUkfasiKbveiwwF+LXkgM/TdO2ZukDP\nDVc+t+ijcFul8SL0GvQ0OtBOJrIYkuQY+oPbAarVR6hh1a6EcS6HbcPrqmDNkFuo\nWnDn/79UXGSmU2H3pc66eSwjAipi3mrptqRTUJQkv+b0SzFSeWseM2MdSUfTF3kd\nHhapIAmSRB9UGW/JM8/CzsST6Y8LA8gYfBeAP1eQFa2qv8ts9YRolIlD6HfeLwLE\nQK3A6CHKrFzjKof5P33vcw+DYkfND/snx5EyYOelrMWp9+jUn1JuCWjp1v7tnjV5\nzkt40GECAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDMxghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPHzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAB\ntoWRhOLlB+Vmpok0EnGkT9/UJ56qbX6mbW4J7ckzP/hQ9OtFo3QQDSY3uLWYkai4\n8l0Xv5lYLWceKp/cXAJBBSYmPZER08nwARv9zG5U6GWe3eOXIOM4RdYgt3l9Ex1L\njn94xkJoqm1LRUZ8+OReaC+Njm9WU1ljsmDCkhrZ+2QWIApb/G/fRfcwtS/kevKG\nOAJSENSqp2tsGKh5Iq2tSFx2k3X4x851hyxD9nZi/frRigg6d0DGaxsgc61gGFoA\n9zx8fumqH43rmpRUK8yNYnhKLIpe6lAJvXo12vTbpQbCHB3IoQN2IFWNv8QaNuVz\nrOtzeKLvC3LMGB61e38ejTn4WkcVntnW3NRvSx0gSe5fOAHxoJH6NSd+jqB8qGtD\nftgsq4kFPMl4+MYYwhdGUocJqu0RHqYOyTeXh7QOGGp1ZKJgmLJo0ABtML5kcvDA\nBP8M1yBm1B3q//XHUr5PrNDCUrMsJoDbBLWChEHKs3hg2yhNSgqU9dDYawNzq2Dc\nOFgEmVq+GOPgs/QY4/hT00YC0niHCiZ4vee053jhNL+ganFQ9yCrw36+AdrtqFnY\nBOX+HVytB4z1mnpT3fFtxJygLnsScZJHHI3BkApLTLmCR6BL179zyUcWYVaWOS+L\njUyGxiCd/aySeBKjxvurM9S6YF1PAJtZqQR9I+zn3g==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAulPNIp4Hqz674BFRRpw1PFccnG0zUnBEIqOiaRyqpxvqFKe0\nN5Q5c8hP1hgyaj9Ym2gv1QcYORQ1XKu0aPj9li+ffL7660D4Id4GJuzGBDxyDINq\norkt02VcA/CiyzOHi4B05qISNuCCloquysJP3p4o8petQruXVXXLsFi7JXPlval+\nyUVXMgHQ0inNfm2FBI6kGn+Fe0EDT7wOmRkTiqGUuxe1Yh8DS1gwEnZmJXZxLQKq\n1495krvIoQvueZex8apBB+MvVqq1bnStiOdagDGIcDmoe1haqVS/evP/x2dRVwMn\ndrODEDLQKaFDYQolfB3cmfLj/WZBp/T0b+vQ4g6HzPA4Zyu2GU9ftN/IVa39sGEK\noAwrXqWwZZ2XwMjQK9zp0IUkfasiKbveiwwF+LXkgM/TdO2ZukDPDVc+t+ijcFul\n8SL0GvQ0OtBOJrIYkuQY+oPbAarVR6hh1a6EcS6HbcPrqmDNkFuoWnDn/79UXGSm\nU2H3pc66eSwjAipi3mrptqRTUJQkv+b0SzFSeWseM2MdSUfTF3kdHhapIAmSRB9U\nGW/JM8/CzsST6Y8LA8gYfBeAP1eQFa2qv8ts9YRolIlD6HfeLwLEQK3A6CHKrFzj\nKof5P33vcw+DYkfND/snx5EyYOelrMWp9+jUn1JuCWjp1v7tnjV5zkt40GECAwEA\nAQKCAgAGKLj82PPF8a1CKzBgQM+drqolHZL3DvuhmCkeh7JjExAkyKUIup8/jYfe\nEC9dGw+ca3rmnspqhe7RoQy19O0PptlYb/Out8IRVIZ9iObEGGoTXHtiOkl31b+P\nPcgT0XXnOELPpjHV8IvrMABhIWqEdalC2PKtjwfh7PfjshCAR27OWyj0EZUKnna+\n9l9y5hSMjFkuVbgdf+OnzL6VdE0UuwybNp2/SuklTMVMsZnkyR/VQ2OkUSYioeRw\nNmBr/TCw8qty+EvbSzSeHGs+vZmx58aWdxbI+lMHio6dy4JRSWqt5qPKT7AYZqWX\nirDF0hEKeO45hIbhcceKfBVhvGN2LFMpaLrFhWpf3wIEb8ht7O+ZjgYsPxeoRn3c\ncUpsznGl4NPOiZxTY9vndbEONqB5X5ZUpJU805xpMW0IivO+SRtAsxLVlhjEYXeO\nKkGNtZlpUbTmb/ocmFrAlbbFKkGPz8eNavCFKzH2DpN9KMRH+7VfaOpwozD3K4Mx\n/4YFE5wMl4eQao2Dq2TAqW15P67jSpOzqj0WH55rEWRPIrRD6GHoKzSjriUAbOz1\n7q3wRT18ez4W6pt8ByRuIFvafP63R5zde3c6dbcnN5vVr0hWpXs8Sb6GbKdkgcQd\ngMHbRSaDRdIQzErTrfZxh9eodjMYkRj8iSzK95qfiZaZv9ewFQKCAQEA7jXmVK8V\nU6nkyqsqTHpGIqawo23m89+Q2eqvpcU2hvu1nzTNeBUfYz7/cP7PEYTZorAS5OLn\nhY8+4dhffwUpKmtiY6GHXJ3DDDGW98L5BHyPE5eR/BoHQNn8FrNC+4zdsTbDFcXf\nelZs7sLJQqOLZn8tT9hWFu6glz/B5oD6Abcd4e3LaQ/y4vkPugPGSSgLRdEuwM/j\nG69g4xz0XPq+LbF620CjJyohy0XeYAppBHqgyUMWXqjwvnB3NTvmaQXCjjw8DuHk\nLAmP6vJ4XnWgXjXfD5mK8bW1KoBD6mPzGXHT/06XAnOuL16IzccZ19Y1zGzRVUFu\nY0S+2VVPN/zmSwKCAQEAyD4AJUOluVXU82f9+Wvf4LdUeZ3fbhmx48jUmJdnNwXH\nw1FttgXV7YeffPUYYx+hIsJ7/YvSR3nm4yvcNYwDebHBWtMi4TMIRSMuSXh40Uh6\nFzmpL1Xs7TrM1WRV/rVQIWm8OtGuQepJc+IXtYW3CWEL1PhZCcW84NsBUf/eSn4d\nJyiYJIr9yWKryCmELekVimDNkqd7cFbrMeddAqoEMKswarOZP62JxhZDqvGJ7zxm\nN9fcNKNQprLUYb8RD7KttBkShT/ftXuRLUf/o8jJk9qM+rUZwXUnUYJqFqnPfY6q\n/JiiUKF5O/xbrB2aNovGB37M6kJAdiEdknxCIDHogwKCAQEAtqkiG+G9ys/GHxqt\nbsN8MGn9nPRTWsIeQtvRIJ5fN5agKJrGy8P2s5XFLpJVmity72twsY7E1geHwoq2\nf6IndN2vLpLYR066GAmWqBob9TLGS9G4IcabtUDyKz3+qQHVpWRGp9V5jkfjdBpt\nVeoupMP0rPtI2IDXJ+O9/dD85H/ULEC577BJIX62bn8id0p7imfjH/rJPZt4Ae9b\n+khTuJoViTbBZPDLxSGoaaDR8ptN28bTynZlVMLMxkdOYU4HaCYML6VL3/DdKN4R\nKu2rsVJB36ZvgvVBHHwKmBtYeDRuyaEr7rdh4/Trlk+MI5Og4Ua4JlXsHbbDWcgu\n3NHSjQKCAQAke0dDgsOatIyLnBKlcoXjRwNY9O5T/NJwEZLKCit/1snCR/+Y/g8v\nq1nfRjgfNxAAaTZ1xxeUdh0enksMZmnrrfrwNS9RoB6Nmg9Ay3YXiwDTYmn0ec8k\nmWXgKtuQADCNJdppTPJWh/zB5Bs1sxrZ1ph1QJnYMfJrdPsUmW1CXR+qlrbBUaFT\nI2rwmL/ALYsDHwfVBDZ1DSDkgqZradZBl67GgdoVw+Qp9rx1VHQw5ewpmGZCIM78\nz+/prwnItmD3Quog5LY5Nj4I5ekpSdeP3G1676u1l0aGjyAM6PsogOmeA6Du17xi\ntWNuRCZocaaujud+dnuQChB9M65Id09JAoIBAQDkNjmXWUb72zFdrg0yZw4EDqjl\nVnSGJxClVUTaM6Qhl5dnUGHRuXRAbYBMotNythWrclB+a/hoCImQpge5QkUnfzhP\nPeL+VHnN2a0OZbgK2YALnmJff4DNZZGPWwoMfVhogmw5hMtJlh9brgv9DEObcU3u\nY7ky8aZp1xwgjDbdQnjY8ZwVe7h6ehCAL5ih6i84m4oSlGjbgjrWEOAx39RfoEu4\niJyKecKlBqV2x+hvSQGTC89HSvmoabsp5QKyC0q/nRGrqYBn81aZiOo19v+P7GCZ\nAO/9wC1duk/IMdNgMLLnjg7q72+j3tnvo/3hp/0T5a1OutWHSBHuTGVskJps\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T02:43:17.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:17 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:18.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T02:43:19.292 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:18 smithi031 ceph-mon[31209]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:19.292 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:18 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:20.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T02:43:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-09-21T02:43:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:20 smithi031 ceph-mon[31209]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:20 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:21.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$YvS2tppWOVrYx0PIHRSBuu19mT65osPyLAEsJltxGgQyW1IfMW0Ee", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726886601, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T02:43:21.999 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-09-21T02:43:22.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:22 smithi031 ceph-mon[31209]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:22.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:22 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:23.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T02:43:23.248 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:43:23.248 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T02:43:23.256 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:43:23.256 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T02:43:23.256 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T02:43:23.256 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: Password: kvb7uztnf9 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:23.258 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config directory 2024-09-21T02:43:23.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:23 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/120387569' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T02:43:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:24.603 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-09-21T02:43:24.642 INFO:tasks.cephadm:Fetching config... 2024-09-21T02:43:24.642 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:43:24.642 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T02:43:24.658 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T02:43:24.658 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:43:24.658 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T02:43:24.714 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T02:43:24.714 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:43:24.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T02:43:24.779 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T02:43:24.779 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:43:24.779 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T02:43:24.835 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T02:43:24.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T02:43:24.912 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:24.923 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T02:43:24.965 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:24.978 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T02:43:25.018 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhmW5FOebtjqseJaZ6PfpVsYBlKhBueXlTOf94aLYeP4D0mk4R3zccY3dcuyoEPB0YOOnciAaB2+jETUk6mwg92cgVzJ+ftK68+1Ulx+sZeBH05PFGrG3ZBW/I5HtJsXz9HZYsmoCBrc/Vw4xgM9xVTIoneuvS9kgpw6XsuStY9RofpYvbVndBDIt3MdCe4MSS8sl2C6PfdF52vYogkN7pjpcOiXHWYfUgaQrLsJO2osUvv/xxiutyyqgHG4snyLUABtFj1hmGG4fta4S1Q6CVCoPZd0dYDHWvlBM+GYbt/YMnOKKrkjKVJhHCHgHxmUq7vXqkS66VllXq1cvXcUI5i1RC5yYXUCCeQfoilhOat6hZLZyZvtRzUpamRzfOhspo1MvD5o6pKoycKn8IrEitfr6LZOTBu9+tk+eftnFH5JITBsirA0quHZAquzf5YxnGr233B9PCG1l1c3F2bSBpWHK+UG2BLUF1FZoePI547HuTxx7+vNrM4kXvmlB0i4E= ceph-fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:25.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T02:43:25.193 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:25 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1290377567' entity='client.admin' 2024-09-21T02:43:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:26.615 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T02:43:26.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T02:43:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:27.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:26 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1042647894' entity='client.admin' 2024-09-21T02:43:27.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:26 smithi031 ceph-mon[31209]: mgrmap e14: x(active, since 12s) 2024-09-21T02:43:28.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:28.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:28.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:28.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:28.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:28.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:28.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:28.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:28.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:28.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:28 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:28.839 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-09-21T02:43:28.839 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:43:28.839 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-09-21T02:43:28.858 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:43:28.858 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:28.917 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-09-21T02:43:28.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch host add smithi165 2024-09-21T02:43:29.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:29 smithi031 ceph-mon[31209]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:29.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:29 smithi031 ceph-mon[31209]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T02:43:29.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:29 smithi031 ceph-mon[31209]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:29.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:29 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:29.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:29 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:29.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:29 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:29.612 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:30.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:30 smithi031 ceph-mon[31209]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:30.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:30 smithi031 ceph-mon[31209]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:31.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:31 smithi031 ceph-mon[31209]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:32 smithi031 ceph-mon[31209]: Deploying cephadm binary to smithi165 2024-09-21T02:43:34.003 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-09-21T02:43:34.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T02:43:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:35.328 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:35 smithi031 ceph-mon[31209]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:35.328 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:35 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:35.328 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:35 smithi031 ceph-mon[31209]: Added host smithi165 2024-09-21T02:43:35.328 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:35 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:35.328 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:35 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:35.328 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:35 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:35.624 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:35.625 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-09-21T02:43:36.245 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-09-21T02:43:36.245 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:43:36.246 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-09-21T02:43:36.265 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:43:36.265 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:36.322 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-09-21T02:43:36.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch host add smithi178 2024-09-21T02:43:36.575 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:36 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:36 smithi031 ceph-mon[31209]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:36 smithi031 ceph-mon[31209]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:38.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:38 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: Deploying cephadm binary to smithi178 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:40.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:39 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:40 smithi031 ceph-mon[31209]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:41.154 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-09-21T02:43:41.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T02:43:41.979 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:42.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:42 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:42.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:42 smithi031 ceph-mon[31209]: Added host smithi178 2024-09-21T02:43:42.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:42 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:42.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:42 smithi031 ceph-mon[31209]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:42.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:42 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:42.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:42 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:43:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-09-21T02:43:43.170 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T02:43:43.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T02:43:43.428 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:43:43.771 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:43 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:43.771 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:43 smithi031 ceph-mon[31209]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:43:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-21T02:43:44.782 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:44 smithi031 ceph-mon[31209]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:44.783 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:44 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/964701498' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T02:43:45.099 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-09-21T02:43:45.099 INFO:tasks.cephadm:Adding mon.b on smithi165 2024-09-21T02:43:45.099 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-09-21T02:43:45.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c' 2024-09-21T02:43:45.397 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:45.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/964701498' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T02:43:45.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:43:45.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:45.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:45.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:45.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:45.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:45.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:45.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:45 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:46.084 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-09-21T02:43:46.692 DEBUG:teuthology.orchestra.run.smithi165:mon.b> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T02:43:46.695 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T02:43:46.697 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:43:46.697 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:43:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-21T02:43:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:46 smithi031 ceph-mon[31209]: Deploying daemon mon.c on smithi178 2024-09-21T02:43:48.737 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:43:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:48 smithi031 ceph-mon[31209]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:49.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:43:49.585 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","modified":"2024-09-21T02:42:07.738925Z","created":"2024-09-21T02:42:07.738925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T02:43:49.585 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-21T02:43:49.877 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:49 smithi178 systemd[1]: Starting Ceph mon.c for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:43:50.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:49 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/2492111534' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:43:50.420 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 podman[34740]: 2024-09-21 02:43:50.31099982 +0000 UTC m=+0.106703679 container create cd588d01e868c640bb855b4847552caa5f39aabe5bb903ed3edeb0a06b324376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T02:43:50.420 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 podman[34740]: 2024-09-21 02:43:50.219848493 +0000 UTC m=+0.015552316 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:43:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 podman[34740]: 2024-09-21 02:43:50.504989917 +0000 UTC m=+0.300693730 container init cd588d01e868c640bb855b4847552caa5f39aabe5bb903ed3edeb0a06b324376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T02:43:50.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 podman[34740]: 2024-09-21 02:43:50.508286769 +0000 UTC m=+0.303990582 container start cd588d01e868c640bb855b4847552caa5f39aabe5bb903ed3edeb0a06b324376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T02:43:50.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T02:43:50.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T02:43:50.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pidfile_write: ignore empty --pid-file 2024-09-21T02:43:50.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: load: jerasure load: lrc 2024-09-21T02:43:50.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: RocksDB version: 7.9.2 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Git sha 0 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: DB SUMMARY 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: DB Session ID: DAN0H3E86EQDTYBXZMRT 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: CURRENT file: CURRENT 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T02:43:50.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.error_if_exists: 0 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.create_if_missing: 0 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.paranoid_checks: 1 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T02:43:50.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.env: 0x55e521438d60 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.info_log: 0x55e5263af360 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.statistics: (nil) 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.use_fsync: 0 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_log_file_size: 0 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T02:43:50.732 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.allow_fallocate: 1 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.use_direct_reads: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.db_log_dir: 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.wal_dir: 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T02:43:50.733 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.write_buffer_manager: 0x55e5263b3d60 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T02:43:50.734 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.unordered_write: 0 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.row_cache: None 2024-09-21T02:43:50.735 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.wal_filter: None 2024-09-21T02:43:50.736 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T02:43:50.736 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.two_write_queues: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.wal_compression: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.atomic_flush: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.log_readahead_size: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T02:43:50.737 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_background_jobs: 2 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_background_compactions: -1 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_subcompactions: 1 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T02:43:50.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_open_files: -1 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_background_flushes: -1 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Compression algorithms supported: 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kZSTD supported: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kXpressCompression supported: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kBZip2Compression supported: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T02:43:50.739 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kLZ4Compression supported: 1 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kZlibCompression supported: 1 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: kSnappyCompression supported: 1 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.merge_operator: 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_filter: None 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T02:43:50.740 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5263aefa0) 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_type: 0 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_index_type: 0 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_shortening: 1 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: checksum: 4 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: no_block_cache: 0 2024-09-21T02:43:50.741 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache: 0x55e522373610 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_options: 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: capacity : 536870912 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_shard_bits : 4 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: strict_capacity_limit : 0 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_compressed: (nil) 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: persistent_cache: (nil) 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size: 4096 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size_deviation: 10 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_restart_interval: 16 2024-09-21T02:43:50.742 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_block_restart_interval: 1 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: metadata_block_size: 4096 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: partition_filters: 0 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: use_delta_encoding: 1 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: filter_policy: bloomfilter 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: whole_key_filtering: 1 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: verify_compression: 0 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: format_version: 5 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_index_compression: 1 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_align: 0 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: max_auto_readahead_size: 262144 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: prepopulate_block_cache: 0 2024-09-21T02:43:50.743 INFO:journalctl@ceph.mon.c.smithi178.stdout: initial_auto_readahead_size: 8192 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression: NoCompression 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.num_levels: 7 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T02:43:50.744 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:43:50.745 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T02:43:50.746 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T02:43:50.747 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T02:43:50.748 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.inplace_update_support: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.bloom_locality: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.max_successive_merges: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.ttl: 2592000 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T02:43:50.749 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.enable_blob_files: false 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.min_blob_size: 0 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T02:43:50.750 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eb32166c-2e08-492c-b0ab-b30cffd0bf87 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886630545168, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886630545615, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726886630, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eb32166c-2e08-492c-b0ab-b30cffd0bf87", "db_session_id": "DAN0H3E86EQDTYBXZMRT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886630545709, "job": 1, "event": "recovery_finished"} 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e5263d2a80 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: DB pointer 0x55e527178000 2024-09-21T02:43:50.751 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** DB Stats ** 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:43:50.752 INFO:journalctl@ceph.mon.c.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T02:43:50.753 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache BinnedLRUCache@0x55e522373610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-21T02:43:50.754 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: starting mon.c rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(???) e0 preinit fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout: e1 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout: btime 2024-09-21T02:42:18:754388+0000 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:43:50.755 INFO:journalctl@ceph.mon.c.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:43:50.756 INFO:journalctl@ceph.mon.c.smithi178.stdout: legacy client fscid: -1 2024-09-21T02:43:50.756 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-21T02:43:50.756 INFO:journalctl@ceph.mon.c.smithi178.stdout: No filesystems configured 2024-09-21T02:43:50.756 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T02:43:50.756 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T02:43:50.756 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:43:50.757 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:43:50.758 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.758 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.758 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:43:50.758 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:50.758 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.758 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-21T02:43:50.759 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Deploying cephadm binary to smithi178 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.760 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Added host smithi178 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.761 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.762 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:43:50.762 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.762 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/964701498' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T02:43:50.762 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/964701498' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T02:43:50.762 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:43:50.762 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:50.763 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-21T02:43:50.764 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:50.764 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.764 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:50.764 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:50.764 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.764 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.765 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.765 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:50.765 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-21T02:43:50.765 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.765 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:50.765 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:50.766 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:50.766 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:50.766 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:43:50.766 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:50.766 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: Deploying daemon mon.c on smithi178 2024-09-21T02:43:50.767 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:50.767 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/2492111534' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:43:50.767 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 ceph-mon[34759]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T02:43:50.767 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 bash[34740]: cd588d01e868c640bb855b4847552caa5f39aabe5bb903ed3edeb0a06b324376 2024-09-21T02:43:50.767 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:50 smithi178 systemd[1]: Started Ceph mon.c for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:43:51.149 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:43:51.149 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:43:51.403 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:43:55.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: mon.a calling monitor election 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: mon.c calling monitor election 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:55.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: monmap epoch 2 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: last_changed 2024-09-21T02:43:50.574051+0000 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: min_mon_release 19 (squid) 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: election_strategy: 1 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: fsmap 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: mgrmap e14: x(active, since 42s) 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: overall HEALTH_OK 2024-09-21T02:43:55.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:55.981 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:55.981 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:55.981 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:43:55.981 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:55 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:56.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:43:56.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: mon.a calling monitor election 2024-09-21T02:43:56.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:56.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: mon.c calling monitor election 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: monmap epoch 2 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: last_changed 2024-09-21T02:43:50.574051+0000 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: min_mon_release 19 (squid) 2024-09-21T02:43:56.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: election_strategy: 1 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: fsmap 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: mgrmap e14: x(active, since 42s) 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: overall HEALTH_OK 2024-09-21T02:43:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:56.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:56.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:56.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:43:56.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:55 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:56.625 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:43:56.625 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","modified":"2024-09-21T02:43:50.574051Z","created":"2024-09-21T02:42:07.738925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T02:43:56.625 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-21T02:43:56.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:56 smithi178 ceph-mon[34759]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:56.928 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:56 smithi178 ceph-mon[34759]: Deploying daemon mon.b on smithi165 2024-09-21T02:43:56.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:56 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:56.929 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:56 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/2591959119' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:43:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:56 smithi031 ceph-mon[31209]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:56 smithi031 ceph-mon[31209]: Deploying daemon mon.b on smithi165 2024-09-21T02:43:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:56 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:56 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/2591959119' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:43:58.299 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:43:58.299 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:43:58.549 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:43:58.965 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:43:58 smithi178 ceph-mon[34759]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:43:58 smithi031 ceph-mon[31209]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:43:59.240 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","modified":"2024-09-21T02:43:50.574051Z","created":"2024-09-21T02:42:07.738925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T02:43:59.240 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-21T02:43:59.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 podman[34800]: 2024-09-21 02:43:59.293334772 +0000 UTC m=+0.337822131 container start b9f87f32b7ceb1f234a3cd5c5a955c52834ca8bd06fa8c9f8335db293d84b6d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T02:43:59.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T02:43:59.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pidfile_write: ignore empty --pid-file 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: load: jerasure load: lrc 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: RocksDB version: 7.9.2 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Git sha 0 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: DB SUMMARY 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: DB Session ID: WU30X553NO2CWJ8YJ9GO 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: CURRENT file: CURRENT 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T02:43:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.error_if_exists: 0 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.create_if_missing: 0 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.paranoid_checks: 1 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.env: 0x5607daf73d60 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.info_log: 0x5607e0b7f360 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.statistics: (nil) 2024-09-21T02:43:59.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.use_fsync: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_log_file_size: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.allow_fallocate: 1 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.use_direct_reads: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.db_log_dir: 2024-09-21T02:43:59.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.wal_dir: 2024-09-21T02:43:59.695 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T02:43:59.696 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.write_buffer_manager: 0x5607e0b83d60 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.unordered_write: 0 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T02:43:59.697 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.row_cache: None 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.wal_filter: None 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.two_write_queues: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.wal_compression: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.atomic_flush: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.log_readahead_size: 0 2024-09-21T02:43:59.698 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_background_jobs: 2 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_background_compactions: -1 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_subcompactions: 1 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T02:43:59.699 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T02:43:59.700 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T02:43:59.700 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_open_files: -1 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T02:43:59.701 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_background_flushes: -1 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Compression algorithms supported: 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kZSTD supported: 0 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kXpressCompression supported: 0 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kBZip2Compression supported: 0 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kLZ4Compression supported: 1 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kZlibCompression supported: 1 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: kSnappyCompression supported: 1 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T02:43:59.702 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.merge_operator: 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_filter: None 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5607e0b7efa0) 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout: cache_index_and_filter_blocks: 1 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout: pin_top_level_index_and_filter: 1 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout: index_type: 0 2024-09-21T02:43:59.703 INFO:journalctl@ceph.mon.b.smithi165.stdout: data_block_index_type: 0 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: index_shortening: 1 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: checksum: 4 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: no_block_cache: 0 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache: 0x5607dcb43610 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache_name: BinnedLRUCache 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache_options: 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: capacity : 536870912 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: num_shard_bits : 4 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: strict_capacity_limit : 0 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: high_pri_pool_ratio: 0.000 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache_compressed: (nil) 2024-09-21T02:43:59.704 INFO:journalctl@ceph.mon.b.smithi165.stdout: persistent_cache: (nil) 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_size: 4096 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_size_deviation: 10 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_restart_interval: 16 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: index_block_restart_interval: 1 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: metadata_block_size: 4096 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: partition_filters: 0 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: use_delta_encoding: 1 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: filter_policy: bloomfilter 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: whole_key_filtering: 1 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: verify_compression: 0 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: read_amp_bytes_per_bit: 0 2024-09-21T02:43:59.705 INFO:journalctl@ceph.mon.b.smithi165.stdout: format_version: 5 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout: enable_index_compression: 1 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_align: 0 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout: max_auto_readahead_size: 262144 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout: prepopulate_block_cache: 0 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout: initial_auto_readahead_size: 8192 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression: NoCompression 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T02:43:59.706 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.num_levels: 7 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T02:43:59.708 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T02:43:59.709 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T02:43:59.710 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.inplace_update_support: 0 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T02:43:59.711 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.bloom_locality: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.max_successive_merges: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.ttl: 2592000 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T02:43:59.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.enable_blob_files: false 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.min_blob_size: 0 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T02:43:59.713 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 230aa02f-0ab9-4b48-a34b-64b3a25693c2 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886639330358, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886639330776, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726886639, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "230aa02f-0ab9-4b48-a34b-64b3a25693c2", "db_session_id": "WU30X553NO2CWJ8YJ9GO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726886639330879, "job": 1, "event": "recovery_finished"} 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5607e0ba2a80 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: DB pointer 0x5607e1948000 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** DB Stats ** 2024-09-21T02:43:59.714 INFO:journalctl@ceph.mon.b.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** Compaction Stats [default] ** 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:59.715 INFO:journalctl@ceph.mon.b.smithi165.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** Compaction Stats [default] ** 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T02:43:59.716 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: Block cache BinnedLRUCache@0x5607dcb43610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: using public_addr v2:172.21.15.165:0/0 -> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: starting mon.b rank -1 at public addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] at bind addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(???) e0 preinit fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:59.717 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: e1 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: btime 2024-09-21T02:42:18:754388+0000 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: legacy client fscid: -1 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout: No filesystems configured 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T02:43:59.718 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.719 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Deploying cephadm binary to smithi178 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Added host smithi178 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/964701498' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/964701498' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:43:59.721 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.722 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Deploying daemon mon.c on smithi178 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/2492111534' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:43:59.723 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.a calling monitor election 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.c calling monitor election 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: monmap epoch 2 2024-09-21T02:43:59.724 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: last_changed 2024-09-21T02:43:50.574051+0000 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: min_mon_release 19 (squid) 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: election_strategy: 1 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: fsmap 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mgrmap e14: x(active, since 42s) 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: overall HEALTH_OK 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:43:59.725 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: Deploying daemon mon.b on smithi165 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/2591959119' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 ceph-mon[34818]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 bash[34800]: b9f87f32b7ceb1f234a3cd5c5a955c52834ca8bd06fa8c9f8335db293d84b6d5 2024-09-21T02:43:59.726 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:43:59 smithi165 systemd[1]: Started Ceph mon.b for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:44:00.906 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:44:00.907 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:44:01.161 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:44:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:44:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: mon.a calling monitor election 2024-09-21T02:44:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: mon.c calling monitor election 2024-09-21T02:44:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: monmap epoch 3 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: last_changed 2024-09-21T02:43:59.357767+0000 2024-09-21T02:44:04.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: min_mon_release 19 (squid) 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: election_strategy: 1 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: fsmap 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: mgrmap e14: x(active, since 51s) 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: overall HEALTH_OK 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:04 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.761 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: mon.a calling monitor election 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: mon.c calling monitor election 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:04.762 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: monmap epoch 3 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: last_changed 2024-09-21T02:43:59.357767+0000 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: min_mon_release 19 (squid) 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: election_strategy: 1 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: fsmap 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:44:04.763 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: mgrmap e14: x(active, since 51s) 2024-09-21T02:44:04.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: overall HEALTH_OK 2024-09-21T02:44:04.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.764 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:04 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:04.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:44:04.880 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","modified":"2024-09-21T02:43:59.357767Z","created":"2024-09-21T02:42:07.738925Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T02:44:04.880 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-09-21T02:44:05.665 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T02:44:05.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T02:44:05.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:05 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:05.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:05 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/3629582828' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:44:05.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:05 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:05.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:05 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:05.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:05 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:05.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:05 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:05.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:05 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/3629582828' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:44:05.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:05 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:05.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:05 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:05.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:05 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:05.919 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:06.632 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:06.632 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-21T02:44:06.632 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:06.632 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-21T02:44:06.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: mon.a calling monitor election 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: mon.c calling monitor election 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: monmap epoch 3 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: last_changed 2024-09-21T02:43:59.357767+0000 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: min_mon_release 19 (squid) 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: election_strategy: 1 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: fsmap 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:44:06.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: mgrmap e14: x(active, since 51s) 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: overall HEALTH_OK 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/3629582828' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:06.694 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:06 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:07.331 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T02:44:07.331 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:44:07.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T02:44:07.400 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:44:07.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:07.469 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:44:07.469 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T02:44:07.497 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:44:07.498 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:07.564 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:44:07.564 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: mon.b calling monitor election 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: Reconfiguring daemon mon.a on smithi031 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: mon.b calling monitor election 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: mon.c calling monitor election 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: mon.a calling monitor election 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: monmap epoch 3 2024-09-21T02:44:07.569 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: last_changed 2024-09-21T02:43:59.357767+0000 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: min_mon_release 19 (squid) 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: election_strategy: 1 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: fsmap 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: mgrmap e14: x(active, since 53s) 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: overall HEALTH_OK 2024-09-21T02:44:07.570 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:07 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3505184264' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:07.596 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:44:07.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:07.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: mon.b calling monitor election 2024-09-21T02:44:07.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T02:44:07.661 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: Reconfiguring daemon mon.a on smithi031 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: mon.b calling monitor election 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: mon.c calling monitor election 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: mon.a calling monitor election 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: monmap epoch 3 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: last_changed 2024-09-21T02:43:59.357767+0000 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: min_mon_release 19 (squid) 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: election_strategy: 1 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-21T02:44:07.662 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: fsmap 2024-09-21T02:44:07.663 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:44:07.663 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: mgrmap e14: x(active, since 53s) 2024-09-21T02:44:07.663 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: overall HEALTH_OK 2024-09-21T02:44:07.663 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:07 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3505184264' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:07.665 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T02:44:07.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T02:44:07.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: mon.b calling monitor election 2024-09-21T02:44:07.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T02:44:07.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: Reconfiguring daemon mon.a on smithi031 2024-09-21T02:44:07.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: mon.b calling monitor election 2024-09-21T02:44:07.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: mon.c calling monitor election 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: mon.a calling monitor election 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: monmap epoch 3 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: last_changed 2024-09-21T02:43:59.357767+0000 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: created 2024-09-21T02:42:07.738925+0000 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: min_mon_release 19 (squid) 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: election_strategy: 1 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: fsmap 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: mgrmap e14: x(active, since 53s) 2024-09-21T02:44:07.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: overall HEALTH_OK 2024-09-21T02:44:07.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:07 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3505184264' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:08.254 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:08.470 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:08.470 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:08.471 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:08.471 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:08.471 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:08.471 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:08.471 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:08 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:08.627 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:08.628 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:08.628 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:08.628 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:08.628 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:08.628 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:08.628 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:08 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:08 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:09.800 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:44:09.800 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: Reconfiguring daemon mon.b on smithi165 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/716468189' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:09.801 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:09 smithi165 ceph-mon[34818]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:10.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:44:10.056 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: Reconfiguring daemon mon.b on smithi165 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/716468189' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:10.057 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:09 smithi178 ceph-mon[34759]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:10.084 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ignoring --setuser ceph since I am not root 2024-09-21T02:44:10.084 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ignoring --setgroup ceph since I am not root 2024-09-21T02:44:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:44:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: Reconfiguring daemon mon.b on smithi165 2024-09-21T02:44:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/716468189' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T02:44:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' 2024-09-21T02:44:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:09 smithi031 ceph-mon[31209]: from='mgr.14152 172.21.15.31:0/2377382228' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:10.593 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-09-21T02:44:10.594 INFO:tasks.cephadm:Adding mgr.y on smithi165 2024-09-21T02:44:10.594 INFO:tasks.cephadm:Adding mgr.z on smithi178 2024-09-21T02:44:10.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi031=x;smithi165=y;smithi178=z' 2024-09-21T02:44:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:10 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/716468189' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T02:44:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:10 smithi031 ceph-mon[31209]: mgrmap e15: x(active, since 56s) 2024-09-21T02:44:10.852 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:44:10.967 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:10 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/716468189' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T02:44:10.967 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:10 smithi178 ceph-mon[34759]: mgrmap e15: x(active, since 56s) 2024-09-21T02:44:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:10 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/716468189' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T02:44:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:10 smithi165 ceph-mon[34818]: mgrmap e15: x(active, since 56s) 2024-09-21T02:44:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: Active manager daemon x restarted 2024-09-21T02:44:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: Activating manager daemon x 2024-09-21T02:44:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T02:44:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: mgrmap e16: x(active, starting, since 0.00501626s) 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:44:15.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:15 smithi165 ceph-mon[34818]: Manager daemon x is now available 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: Active manager daemon x restarted 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: Activating manager daemon x 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: mgrmap e16: x(active, starting, since 0.00501626s) 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:44:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:44:15.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:44:15.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:44:15.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:44:15.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:15 smithi178 ceph-mon[34759]: Manager daemon x is now available 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: Active manager daemon x restarted 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: Activating manager daemon x 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: mgrmap e16: x(active, starting, since 0.00501626s) 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:44:15.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:44:15.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:44:15.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:44:15.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:44:15.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:15 smithi031 ceph-mon[31209]: Manager daemon x is now available 2024-09-21T02:44:16.068 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-09-21T02:44:16.367 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:16.367 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:44:16.368 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:44:16.368 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:16 smithi178 ceph-mon[34759]: mgrmap e17: x(active, since 1.01021s) 2024-09-21T02:44:16.368 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:16.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:16.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:44:16.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:44:16.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:16 smithi031 ceph-mon[31209]: mgrmap e17: x(active, since 1.01021s) 2024-09-21T02:44:16.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:16.438 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:16.439 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:44:16.439 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:44:16.439 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:16 smithi165 ceph-mon[34818]: mgrmap e17: x(active, since 1.01021s) 2024-09-21T02:44:16.439 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:16.679 DEBUG:teuthology.orchestra.run.smithi165:mgr.y> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T02:44:16.681 DEBUG:teuthology.orchestra.run.smithi178:mgr.z> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T02:44:16.684 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T02:44:16.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T02:44:16.687 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T02:44:16.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T02:44:16.730 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T02:44:16.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T02:44:16.774 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T02:44:16.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T02:44:16.817 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T02:44:16.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T02:44:16.860 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T02:44:16.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T02:44:16.904 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T02:44:16.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T02:44:16.947 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T02:44:16.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T02:44:16.992 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T02:44:16.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T02:44:17.037 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T02:44:17.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T02:44:17.093 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T02:44:17.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T02:44:17.109 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:17.111 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T02:44:17.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T02:44:17.160 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T02:44:17.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T02:44:17.161 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:17.215 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T02:44:17.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T02:44:17.263 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T02:44:17.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T02:44:17.314 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T02:44:17.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T02:44:17.363 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T02:44:17.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T02:44:17.413 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T02:44:17.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T02:44:17.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:44:17.349+0000 7f08e1730640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:02:44:17] ENGINE Error in HTTPServer.serve 2024-09-21T02:44:17.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: Traceback (most recent call last): 2024-09-21T02:44:17.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:44:17.457 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._connections.run(self.expiration_interval) 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._run(expiration_interval) 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: s = self.context.wrap_socket( 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: return self.sslsocket_class._create( 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:44:17.458 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self.do_handshake() 2024-09-21T02:44:17.459 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:44:17.459 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: self._sslobj.do_handshake() 2024-09-21T02:44:17.459 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:44:17.459 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:44:17.477 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T02:44:17.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T02:44:17.532 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T02:44:17.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T02:44:17.581 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T02:44:17.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T02:44:17.640 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T02:44:17.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T02:44:17.690 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T02:44:17.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T02:44:17.746 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T02:44:17.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T02:44:17.805 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T02:44:17.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T02:44:17.863 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T02:44:17.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T02:44:17.915 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T02:44:17.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T02:44:17.975 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T02:44:17.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T02:44:17.975 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:18.022 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T02:44:18.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: Saving service mgr spec with placement smithi031=x;smithi165=y;smithi178=z;count:3 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:18.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.052 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T02:44:18.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T02:44:18.115 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T02:44:18.115 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi031') 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: Saving service mgr spec with placement smithi031=x;smithi165=y;smithi178=z;count:3 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:18.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: Saving service mgr spec with placement smithi031=x;smithi165=y;smithi178=z;count:3 2024-09-21T02:44:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:18.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:18.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: [21/Sep/2024:02:44:17] ENGINE Bus STARTING 2024-09-21T02:44:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: [21/Sep/2024:02:44:17] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T02:44:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: [21/Sep/2024:02:44:17] ENGINE Error in HTTPServer.serve 2024-09-21T02:44:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-21T02:44:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:44:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-21T02:44:19.085 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: [21/Sep/2024:02:44:17] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: [21/Sep/2024:02:44:17] ENGINE Bus STARTED 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: mgrmap e18: x(active, since 2s) 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:19.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:18 smithi031 ceph-mon[31209]: from='client.? ' entity='client.admin' 2024-09-21T02:44:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: [21/Sep/2024:02:44:17] ENGINE Bus STARTING 2024-09-21T02:44:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: [21/Sep/2024:02:44:17] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T02:44:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: [21/Sep/2024:02:44:17] ENGINE Error in HTTPServer.serve 2024-09-21T02:44:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout: Traceback (most recent call last): 2024-09-21T02:44:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: self._run(expiration_interval) 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: s = self.context.wrap_socket( 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: return self.sslsocket_class._create( 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:44:19.192 INFO:journalctl@ceph.mon.b.smithi165.stdout: self.do_handshake() 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout: self._sslobj.do_handshake() 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: [21/Sep/2024:02:44:17] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: [21/Sep/2024:02:44:17] ENGINE Bus STARTED 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: mgrmap e18: x(active, since 2s) 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:19.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:18 smithi165 ceph-mon[34818]: from='client.? ' entity='client.admin' 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: [21/Sep/2024:02:44:17] ENGINE Bus STARTING 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: [21/Sep/2024:02:44:17] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: [21/Sep/2024:02:44:17] ENGINE Error in HTTPServer.serve 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout: Traceback (most recent call last): 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:44:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: self._run(expiration_interval) 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: s = self.context.wrap_socket( 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: return self.sslsocket_class._create( 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: self.do_handshake() 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:44:19.229 INFO:journalctl@ceph.mon.c.smithi178.stdout: self._sslobj.do_handshake() 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: [21/Sep/2024:02:44:17] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: [21/Sep/2024:02:44:17] ENGINE Bus STARTED 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: mgrmap e18: x(active, since 2s) 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:19.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:18 smithi178 ceph-mon[34759]: from='client.? ' entity='client.admin' 2024-09-21T02:44:19.278 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi031') 2024-09-21T02:44:20.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:19 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1089902438' entity='client.admin' 2024-09-21T02:44:20.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:20.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:20.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:20.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:19 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1089902438' entity='client.admin' 2024-09-21T02:44:20.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:20.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:20.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:20.228 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi031') 2024-09-21T02:44:20.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:19 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1089902438' entity='client.admin' 2024-09-21T02:44:20.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T02:44:20.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:20.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.678 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.696 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.716 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.717 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.717 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.717 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.799 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T02:44:20.799 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-21T02:44:20.799 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-21T02:44:20.799 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.800 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:20.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:44:20.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:20.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:20.802 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:20.895 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:20.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:44:20.897 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:20.897 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.930+0000 7f9f32f4d640 1 -- 172.21.15.31:0/409333752 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f1c03fb70 msgr2=0x7f9f1c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.930+0000 7f9f32f4d640 1 --2- 172.21.15.31:0/409333752 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f1c03fb70 0x7f9f1c042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000ba60 tx=0x7f9f20005db0 comp rx=0 tx=0).stop 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.930+0000 7f9f32f4d640 1 -- 172.21.15.31:0/409333752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f2c1036d0 msgr2=0x7f9f2c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.930+0000 7f9f32f4d640 1 --2- 172.21.15.31:0/409333752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f2c1036d0 0x7f9f2c19a3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005ab0 tx=0x7f9f180051d0 comp rx=0 tx=0).stop 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 -- 172.21.15.31:0/409333752 shutdown_connections 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 --2- 172.21.15.31:0/409333752 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f1c03fb70 0x7f9f1c042080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 --2- 172.21.15.31:0/409333752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f2c104070 0x7f9f2c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 --2- 172.21.15.31:0/409333752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f2c1036d0 0x7f9f2c19a3f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:20.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 --2- 172.21.15.31:0/409333752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f2c102640 0x7f9f2c199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:20.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 -- 172.21.15.31:0/409333752 >> 172.21.15.31:0/409333752 conn(0x7f9f2c0fdfb0 msgr2=0x7f9f2c0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:20.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 -- 172.21.15.31:0/409333752 shutdown_connections 2024-09-21T02:44:20.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:20.931+0000 7f9f32f4d640 1 -- 172.21.15.31:0/409333752 wait complete. 2024-09-21T02:44:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T02:44:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-21T02:44:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-21T02:44:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: Updating smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:21.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:21.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/604358466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc108cf0 msgr2=0x7f0dbc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/604358466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc10f520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009ca0 tx=0x7f0db402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/604358466 shutdown_connections 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/604358466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dbc1083a0 0x7f0dbc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/604358466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc10f520 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/604358466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc102a60 0x7f0dbc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.686+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/604358466 >> 172.21.15.31:0/604358466 conn(0x7f0dbc079730 msgr2=0x7f0dbc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.687+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/604358466 shutdown_connections 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.687+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/604358466 wait complete. 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.687+0000 7f0dc2c3c640 1 Processor -- start 2024-09-21T02:44:21.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.687+0000 7f0dc2c3c640 1 -- start start 2024-09-21T02:44:21.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc2c3c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 0x7f0dbc19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc2c3c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dbc1083a0 0x7f0dbc19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc2c3c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc2c3c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc1126d0 con 0x7f0dbc102a60 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc2c3c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc112550 con 0x7f0dbc108cf0 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc2c3c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc112850 con 0x7f0dbc1083a0 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 0x7f0dbc19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 0x7f0dbc19b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37646/0 (socket says 172.21.15.31:37646) 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 -- 172.21.15.31:0/4106021788 learned_addr learned my addr 172.21.15.31:0/4106021788 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dbbfff640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dbc1083a0 0x7f0dbc19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc11d9640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 -- 172.21.15.31:0/4106021788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dbc1083a0 msgr2=0x7f0dbc19b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:21.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dbc1083a0 0x7f0dbc19b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 -- 172.21.15.31:0/4106021788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc108cf0 msgr2=0x7f0dbc19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc09d8640 1 -- 172.21.15.31:0/4106021788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc1a63a0 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.688+0000 7f0dc11d9640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.689+0000 7f0dc09d8640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 0x7f0dbc19b0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00e9f0 tx=0x7f0dac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.689+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00cd80 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.689+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f0dac0108a0 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.689+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc1a6690 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.689+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac018760 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.689+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a6bd0 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.690+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f0dac018900 con 0x7f0dbc102a60 2024-09-21T02:44:21.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.690+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc102e70 con 0x7f0dbc102a60 2024-09-21T02:44:21.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.690+0000 7f0db9ffb640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0da803fb70 0x7f0da8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:21.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.690+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0dac054ab0 con 0x7f0dbc102a60 2024-09-21T02:44:21.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.691+0000 7f0dbbfff640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0da803fb70 0x7f0da8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:21.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.691+0000 7f0dbbfff640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0da803fb70 0x7f0da8042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc079b80 tx=0x7f0da4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.695+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0dac0071c0 con 0x7f0dbc102a60 2024-09-21T02:44:21.728 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:21 smithi178 systemd[1]: Starting Ceph mgr.z for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:44:21.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.890+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f0dbc10aae0 con 0x7f0dbc102a60 2024-09-21T02:44:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.893+0000 7f0db9ffb640 1 -- 172.21.15.31:0/4106021788 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v13)=0 v13) ==== 116+0+0 (secure 0 0 0) 0x7f0dac010a50 con 0x7f0dbc102a60 2024-09-21T02:44:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0da803fb70 msgr2=0x7f0da8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0da803fb70 0x7f0da8042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc079b80 tx=0x7f0da4005f90 comp rx=0 tx=0).stop 2024-09-21T02:44:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 msgr2=0x7f0dbc19b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 0x7f0dbc19b0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00e9f0 tx=0x7f0dac00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 shutdown_connections 2024-09-21T02:44:21.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0da803fb70 0x7f0da8042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dbc1083a0 0x7f0dbc19b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dbc108cf0 0x7f0dbc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 --2- 172.21.15.31:0/4106021788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dbc102a60 0x7f0dbc19b0e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:21.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 >> 172.21.15.31:0/4106021788 conn(0x7f0dbc079730 msgr2=0x7f0dbc104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:21.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 shutdown_connections 2024-09-21T02:44:21.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:21.896+0000 7f0dc2c3c640 1 -- 172.21.15.31:0/4106021788 wait complete. 2024-09-21T02:44:21.961 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:21 smithi031 ceph-mon[31209]: Deploying daemon mgr.z on smithi178 2024-09-21T02:44:21.961 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:21 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/409333752' entity='client.admin' 2024-09-21T02:44:22.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:21 smithi178 ceph-mon[34759]: Deploying daemon mgr.z on smithi178 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:21 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/409333752' entity='client.admin' 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:21 smithi178 podman[36706]: 2024-09-21 02:44:21.81431624 +0000 UTC m=+0.090559348 container create 9fc30afc7849b264bd92e2c082d09a999f1689d65abc0c248d86972686c729b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-z, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:21 smithi178 podman[36706]: 2024-09-21 02:44:21.740100212 +0000 UTC m=+0.016343291 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:22 smithi178 podman[36706]: 2024-09-21 02:44:22.0000278 +0000 UTC m=+0.276270878 container init 9fc30afc7849b264bd92e2c082d09a999f1689d65abc0c248d86972686c729b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:22 smithi178 podman[36706]: 2024-09-21 02:44:22.002804377 +0000 UTC m=+0.279047452 container start 9fc30afc7849b264bd92e2c082d09a999f1689d65abc0c248d86972686c729b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mgr[36725]: -- 172.21.15.178:0/325126501 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abe17e10e0 con 0x55abe17ffa80 2024-09-21T02:44:22.059 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:22 smithi178 bash[36706]: 9fc30afc7849b264bd92e2c082d09a999f1689d65abc0c248d86972686c729b7 2024-09-21T02:44:22.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:21 smithi165 ceph-mon[34818]: Deploying daemon mgr.z on smithi178 2024-09-21T02:44:22.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:21 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/409333752' entity='client.admin' 2024-09-21T02:44:22.478 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 02:44:22 smithi178 systemd[1]: Started Ceph mgr.z for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:44:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.585+0000 7f7a92042640 1 -- 172.21.15.31:0/1466585688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 msgr2=0x7f7a8c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.585+0000 7f7a92042640 1 --2- 172.21.15.31:0/1466585688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c108b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009cd0 tx=0x7f7a7c02a8e0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 -- 172.21.15.31:0/1466585688 shutdown_connections 2024-09-21T02:44:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 --2- 172.21.15.31:0/1466585688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c101e20 0x7f7a8c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 --2- 172.21.15.31:0/1466585688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a8c102770 0x7f7a8c1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 --2- 172.21.15.31:0/1466585688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c108b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 -- 172.21.15.31:0/1466585688 >> 172.21.15.31:0/1466585688 conn(0x7f7a8c077cc0 msgr2=0x7f7a8c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 -- 172.21.15.31:0/1466585688 shutdown_connections 2024-09-21T02:44:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.586+0000 7f7a92042640 1 -- 172.21.15.31:0/1466585688 wait complete. 2024-09-21T02:44:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 Processor -- start 2024-09-21T02:44:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 -- start start 2024-09-21T02:44:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a8c101e20 0x7f7a8c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c102770 0x7f7a8c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c113870 con 0x7f7a8c108730 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1136f0 con 0x7f7a8c101e20 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a92042640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1139f0 con 0x7f7a8c102770 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a8bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a8c101e20 0x7f7a8c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a8bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c19fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37658/0 (socket says 172.21.15.31:37658) 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c102770 0x7f7a8c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.587+0000 7f7a8bfff640 1 -- 172.21.15.31:0/3761986031 learned_addr learned my addr 172.21.15.31:0/3761986031 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a8affd640 1 -- 172.21.15.31:0/3761986031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a8c101e20 msgr2=0x7f7a8c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a8affd640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a8c101e20 0x7f7a8c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a8affd640 1 -- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 msgr2=0x7f7a8c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a8affd640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a8affd640 1 -- 172.21.15.31:0/3761986031 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a02f0 con 0x7f7a8c102770 2024-09-21T02:44:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a8affd640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c102770 0x7f7a8c19a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000e9c0 tx=0x7f7a8000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000ccc0 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f7a80004490 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a05e0 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.588+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800047a0 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.589+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c102f30 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.590+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c102230 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.590+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f7a800049a0 con 0x7f7a8c102770 2024-09-21T02:44:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.590+0000 7f7a88ff9640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a7403fb70 0x7f7a74042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.590+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7a800545b0 con 0x7f7a8c102770 2024-09-21T02:44:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.590+0000 7f7a8b7fe640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a7403fb70 0x7f7a74042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.591+0000 7f7a8b7fe640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a7403fb70 0x7f7a74042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006010 tx=0x7f7a7c009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.595+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a8000fda0 con 0x7f7a8c102770 2024-09-21T02:44:22.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.783+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f7a8c10bc60 con 0x7f7a8c102770 2024-09-21T02:44:22.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.788+0000 7f7a88ff9640 1 -- 172.21.15.31:0/3761986031 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v14)=0 v14) ==== 115+0+0 (secure 0 0 0) 0x7f7a80004de0 con 0x7f7a8c102770 2024-09-21T02:44:22.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a7403fb70 msgr2=0x7f7a74042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a7403fb70 0x7f7a74042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006010 tx=0x7f7a7c009360 comp rx=0 tx=0).stop 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c102770 msgr2=0x7f7a8c19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c102770 0x7f7a8c19a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000e9c0 tx=0x7f7a8000ee80 comp rx=0 tx=0).stop 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 shutdown_connections 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a7403fb70 0x7f7a74042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a8c102770 0x7f7a8c19a920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c108730 0x7f7a8c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 --2- 172.21.15.31:0/3761986031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a8c101e20 0x7f7a8c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 >> 172.21.15.31:0/3761986031 conn(0x7f7a8c077cc0 msgr2=0x7f7a8c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 shutdown_connections 2024-09-21T02:44:22.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:22.791+0000 7f7a92042640 1 -- 172.21.15.31:0/3761986031 wait complete. 2024-09-21T02:44:23.153 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/4106021788' entity='client.admin' 2024-09-21T02:44:23.153 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: Deploying daemon mgr.y on smithi165 2024-09-21T02:44:23.154 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:22 smithi178 ceph-mon[34759]: from='client.? ' entity='client.admin' 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/4106021788' entity='client.admin' 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: Deploying daemon mgr.y on smithi165 2024-09-21T02:44:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:22 smithi165 ceph-mon[34818]: from='client.? ' entity='client.admin' 2024-09-21T02:44:23.295 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4106021788' entity='client.admin' 2024-09-21T02:44:23.295 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.295 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: Deploying daemon mgr.y on smithi165 2024-09-21T02:44:23.296 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:22 smithi031 ceph-mon[31209]: from='client.? ' entity='client.admin' 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 -- 172.21.15.31:0/2232161865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 msgr2=0x7f72bc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/2232161865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc10f520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009ca0 tx=0x7f72b402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 -- 172.21.15.31:0/2232161865 shutdown_connections 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/2232161865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc1083a0 0x7f72bc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/2232161865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc10f520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/2232161865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc102a60 0x7f72bc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 -- 172.21.15.31:0/2232161865 >> 172.21.15.31:0/2232161865 conn(0x7f72bc079730 msgr2=0x7f72bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 -- 172.21.15.31:0/2232161865 shutdown_connections 2024-09-21T02:44:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.820+0000 7f72c2d1f640 1 -- 172.21.15.31:0/2232161865 wait complete. 2024-09-21T02:44:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 Processor -- start 2024-09-21T02:44:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 -- start start 2024-09-21T02:44:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc102a60 0x7f72bc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc1083a0 0x7f72bc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc112770 con 0x7f72bc108cf0 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1125f0 con 0x7f72bc102a60 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.821+0000 7f72c2d1f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1128f0 con 0x7f72bc1083a0 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc19fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37682/0 (socket says 172.21.15.31:37682) 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 -- 172.21.15.31:0/4092146074 learned_addr learned my addr 172.21.15.31:0/4092146074 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72bbfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc1083a0 0x7f72bc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c0abb640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc102a60 0x7f72bc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 -- 172.21.15.31:0/4092146074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc1083a0 msgr2=0x7f72bc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc1083a0 0x7f72bc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 -- 172.21.15.31:0/4092146074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc102a60 msgr2=0x7f72bc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc102a60 0x7f72bc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 -- 172.21.15.31:0/4092146074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc1a01e0 con 0x7f72bc108cf0 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c12bc640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc19fab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f72b4033010 tx=0x7f72b4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4003cb0 con 0x7f72bc108cf0 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f72b4003e50 con 0x7f72bc108cf0 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc1a0470 con 0x7f72bc108cf0 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.822+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40309d0 con 0x7f72bc108cf0 2024-09-21T02:44:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.823+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc104130 con 0x7f72bc108cf0 2024-09-21T02:44:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.824+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f72b4030bb0 con 0x7f72bc108cf0 2024-09-21T02:44:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.824+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72bc103ce0 con 0x7f72bc108cf0 2024-09-21T02:44:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.824+0000 7f72b9ffb640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72a403fb70 0x7f72a4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.824+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f72b4075920 con 0x7f72bc108cf0 2024-09-21T02:44:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.824+0000 7f72c0abb640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72a403fb70 0x7f72a4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.825+0000 7f72c0abb640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72a403fb70 0x7f72a4042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f72ac010d80 tx=0x7f72ac012040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:23.829+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72b403b410 con 0x7f72bc108cf0 2024-09-21T02:44:24.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.018+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f72bc19b3a0 con 0x7f72bc108cf0 2024-09-21T02:44:24.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.022+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7f72b404d580 con 0x7f72bc108cf0 2024-09-21T02:44:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.023+0000 7f72b9ffb640 1 -- 172.21.15.31:0/4092146074 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f72b4036020 con 0x7f72bc108cf0 2024-09-21T02:44:24.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72a403fb70 msgr2=0x7f72a4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72a403fb70 0x7f72a4042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f72ac010d80 tx=0x7f72ac012040 comp rx=0 tx=0).stop 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 msgr2=0x7f72bc19fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc19fab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f72b4033010 tx=0x7f72b4009880 comp rx=0 tx=0).stop 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 shutdown_connections 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72a403fb70 0x7f72a4042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72bc1083a0 0x7f72bc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72bc108cf0 0x7f72bc19fab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 --2- 172.21.15.31:0/4092146074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc102a60 0x7f72bc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 >> 172.21.15.31:0/4092146074 conn(0x7f72bc079730 msgr2=0x7f72bc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 shutdown_connections 2024-09-21T02:44:24.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.024+0000 7f72c2d1f640 1 -- 172.21.15.31:0/4092146074 wait complete. 2024-09-21T02:44:24.191 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:23 smithi165 systemd[1]: Starting Ceph mgr.y for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:44:24.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 -- 172.21.15.31:0/776436553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 msgr2=0x7efcfc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 --2- 172.21.15.31:0/776436553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc110640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009ca0 tx=0x7efcf402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 -- 172.21.15.31:0/776436553 shutdown_connections 2024-09-21T02:44:24.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 --2- 172.21.15.31:0/776436553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc1094c0 0x7efcfc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 --2- 172.21.15.31:0/776436553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc110640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 --2- 172.21.15.31:0/776436553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcfc103b80 0x7efcfc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 -- 172.21.15.31:0/776436553 >> 172.21.15.31:0/776436553 conn(0x7efcfc0ff610 msgr2=0x7efcfc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 -- 172.21.15.31:0/776436553 shutdown_connections 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.390+0000 7efd0439f640 1 -- 172.21.15.31:0/776436553 wait complete. 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.391+0000 7efd0439f640 1 Processor -- start 2024-09-21T02:44:24.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.391+0000 7efd0439f640 1 -- start start 2024-09-21T02:44:24.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.391+0000 7efd0439f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcfc103b80 0x7efcfc19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:24.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.391+0000 7efd0439f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc1094c0 0x7efcfc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:24.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.391+0000 7efd0439f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:24.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.391+0000 7efd0439f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1139b0 con 0x7efcfc109e10 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0439f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113830 con 0x7efcfc103b80 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0439f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc113b30 con 0x7efcfc1094c0 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0213b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcfc103b80 0x7efcfc19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37698/0 (socket says 172.21.15.31:37698) 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0193a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc1094c0 0x7efcfc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 -- 172.21.15.31:0/1131187069 learned_addr learned my addr 172.21.15.31:0/1131187069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 -- 172.21.15.31:0/1131187069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc1094c0 msgr2=0x7efcfc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc1094c0 0x7efcfc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 -- 172.21.15.31:0/1131187069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcfc103b80 msgr2=0x7efcfc19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcfc103b80 0x7efcfc19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 -- 172.21.15.31:0/1131187069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc1a02b0 con 0x7efcfc109e10 2024-09-21T02:44:24.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0293c640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc19fb80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efcf4005e30 tx=0x7efcf40305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4003c30 con 0x7efcfc109e10 2024-09-21T02:44:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efcf4003dd0 con 0x7efcfc109e10 2024-09-21T02:44:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc1a0540 con 0x7efcfc109e10 2024-09-21T02:44:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4004710 con 0x7efcfc109e10 2024-09-21T02:44:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.392+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc1a6d00 con 0x7efcfc109e10 2024-09-21T02:44:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.393+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcfc104e00 con 0x7efcfc109e10 2024-09-21T02:44:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.394+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7efcf404d440 con 0x7efcfc109e10 2024-09-21T02:44:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.394+0000 7efcf37fe640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efce803fb70 0x7efce8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.394+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efcf4075700 con 0x7efcfc109e10 2024-09-21T02:44:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.394+0000 7efd0213b640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efce803fb70 0x7efce8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:24.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.394+0000 7efd0213b640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efce803fb70 0x7efce8042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efcf80058c0 tx=0x7efcf800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:24.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.399+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efcf403f1e0 con 0x7efcfc109e10 2024-09-21T02:44:24.467 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 podman[36321]: 2024-09-21 02:44:24.209090973 +0000 UTC m=+0.106561181 container create 41c6c7592df232b8c598c0b574ae4bf747c4ae9d8b800030214c4ad7f6a81769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:44:24.467 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 podman[36321]: 2024-09-21 02:44:24.117812906 +0000 UTC m=+0.015283115 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:44:24.467 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 podman[36321]: 2024-09-21 02:44:24.428325566 +0000 UTC m=+0.325795785 container init 41c6c7592df232b8c598c0b574ae4bf747c4ae9d8b800030214c4ad7f6a81769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T02:44:24.467 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 podman[36321]: 2024-09-21 02:44:24.431083129 +0000 UTC m=+0.328553339 container start 41c6c7592df232b8c598c0b574ae4bf747c4ae9d8b800030214c4ad7f6a81769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-21T02:44:24.467 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 ceph-mgr[36340]: -- 172.21.15.165:0/571903699 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594d4e530e0 con 0x5594d9bf6480 2024-09-21T02:44:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.600+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7efcfc10b670 con 0x7efcfc109e10 2024-09-21T02:44:24.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.603+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7efcf403b400 con 0x7efcfc109e10 2024-09-21T02:44:24.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.605+0000 7efcf37fe640 1 -- 172.21.15.31:0/1131187069 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf4036030 con 0x7efcfc109e10 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efce803fb70 msgr2=0x7efce8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efce803fb70 0x7efce8042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efcf80058c0 tx=0x7efcf800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 msgr2=0x7efcfc19fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc19fb80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efcf4005e30 tx=0x7efcf40305c0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 shutdown_connections 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efce803fb70 0x7efce8042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc1094c0 0x7efcfc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcfc109e10 0x7efcfc19fb80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 --2- 172.21.15.31:0/1131187069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcfc103b80 0x7efcfc19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 >> 172.21.15.31:0/1131187069 conn(0x7efcfc0ff610 msgr2=0x7efcfc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:24.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 shutdown_connections 2024-09-21T02:44:24.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:24.606+0000 7efd0439f640 1 -- 172.21.15.31:0/1131187069 wait complete. 2024-09-21T02:44:24.941 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 bash[36321]: 41c6c7592df232b8c598c0b574ae4bf747c4ae9d8b800030214c4ad7f6a81769 2024-09-21T02:44:24.941 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 02:44:24 smithi165 systemd[1]: Started Ceph mgr.y for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 -- 172.21.15.31:0/443714941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8103b60 msgr2=0x7f72e8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/443714941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8103b60 0x7f72e8103f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009900 tx=0x7f72e002a410 comp rx=0 tx=0).stop 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 -- 172.21.15.31:0/443714941 shutdown_connections 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/443714941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72e8109df0 0x7f72e8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/443714941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e81094a0 0x7f72e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/443714941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8103b60 0x7f72e8103f70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 -- 172.21.15.31:0/443714941 >> 172.21.15.31:0/443714941 conn(0x7f72e807b850 msgr2=0x7f72e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 -- 172.21.15.31:0/443714941 shutdown_connections 2024-09-21T02:44:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.011+0000 7f72ed6e3640 1 -- 172.21.15.31:0/443714941 wait complete. 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.012+0000 7f72ed6e3640 1 Processor -- start 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.012+0000 7f72ed6e3640 1 -- start start 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72ed6e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e8103b60 0x7f72e819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72ed6e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72e81094a0 0x7f72e819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72ed6e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 0x7f72e819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72ed6e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113ab0 con 0x7f72e8109df0 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72ed6e3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113930 con 0x7f72e8103b60 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72ed6e3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113c30 con 0x7f72e81094a0 2024-09-21T02:44:25.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 0x7f72e819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e8103b60 0x7f72e819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 0x7f72e819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37714/0 (socket says 172.21.15.31:37714) 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 -- 172.21.15.31:0/432610236 learned_addr learned my addr 172.21.15.31:0/432610236 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e67fc640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72e81094a0 0x7f72e819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 -- 172.21.15.31:0/432610236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72e81094a0 msgr2=0x7f72e819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72e81094a0 0x7f72e819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 -- 172.21.15.31:0/432610236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e8103b60 msgr2=0x7f72e819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e8103b60 0x7f72e819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 -- 172.21.15.31:0/432610236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e81a03f0 con 0x7f72e8109df0 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e6ffd640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e8103b60 0x7f72e819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72e77fe640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 0x7f72e819fcc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72d800eb10 tx=0x7f72d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:25.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.013+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004b20 con 0x7f72e8109df0 2024-09-21T02:44:25.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.014+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d8004cc0 con 0x7f72e8109df0 2024-09-21T02:44:25.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.014+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8010670 con 0x7f72e8109df0 2024-09-21T02:44:25.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.014+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e81a06e0 con 0x7f72e8109df0 2024-09-21T02:44:25.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.014+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a6e40 con 0x7f72e8109df0 2024-09-21T02:44:25.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.015+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f72d8010810 con 0x7f72e8109df0 2024-09-21T02:44:25.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.015+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e8103f70 con 0x7f72e8109df0 2024-09-21T02:44:25.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.015+0000 7f72dffff640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72d403fb70 0x7f72d4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.015+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f72d8054b60 con 0x7f72e8109df0 2024-09-21T02:44:25.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.015+0000 7f72e6ffd640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72d403fb70 0x7f72d4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.016+0000 7f72e6ffd640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72d403fb70 0x7f72d4042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f72e0031440 tx=0x7f72e0032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:25.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.020+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72d8016330 con 0x7f72e8109df0 2024-09-21T02:44:25.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.212+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f72e802e640 con 0x7f72e8109df0 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4092146074' entity='client.admin' 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1131187069' entity='client.admin' 2024-09-21T02:44:25.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:25 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.216+0000 7f72dffff640 1 -- 172.21.15.31:0/432610236 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v17)=0 v17) ==== 115+0+0 (secure 0 0 0) 0x7f72d8019da0 con 0x7f72e8109df0 2024-09-21T02:44:25.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72d403fb70 msgr2=0x7f72d4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72d403fb70 0x7f72d4042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f72e0031440 tx=0x7f72e0032ed0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 msgr2=0x7f72e819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 0x7f72e819fcc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72d800eb10 tx=0x7f72d800c2d0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 shutdown_connections 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72d403fb70 0x7f72d4042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72e81094a0 0x7f72e819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72e8109df0 0x7f72e819fcc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 --2- 172.21.15.31:0/432610236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e8103b60 0x7f72e819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 >> 172.21.15.31:0/432610236 conn(0x7f72e807b850 msgr2=0x7f72e8107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 shutdown_connections 2024-09-21T02:44:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.219+0000 7f72ed6e3640 1 -- 172.21.15.31:0/432610236 wait complete. 2024-09-21T02:44:25.274 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/4092146074' entity='client.admin' 2024-09-21T02:44:25.274 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.274 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.275 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.275 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.275 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1131187069' entity='client.admin' 2024-09-21T02:44:25.275 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:25 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:25.314 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/4092146074' entity='client.admin' 2024-09-21T02:44:25.314 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.314 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:25.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1131187069' entity='client.admin' 2024-09-21T02:44:25.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:25 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 -- 172.21.15.31:0/3859533792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61041094a0 msgr2=0x7f61041098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 --2- 172.21.15.31:0/3859533792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61041094a0 0x7f61041098b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009900 tx=0x7f60f402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 -- 172.21.15.31:0/3859533792 shutdown_connections 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 --2- 172.21.15.31:0/3859533792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6104109df0 0x7f6104110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 --2- 172.21.15.31:0/3859533792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61041094a0 0x7f61041098b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 --2- 172.21.15.31:0/3859533792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6104103b60 0x7f6104103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 -- 172.21.15.31:0/3859533792 >> 172.21.15.31:0/3859533792 conn(0x7f610407b850 msgr2=0x7f610407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 -- 172.21.15.31:0/3859533792 shutdown_connections 2024-09-21T02:44:25.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.414+0000 7f610ba1d640 1 -- 172.21.15.31:0/3859533792 wait complete. 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.415+0000 7f610ba1d640 1 Processor -- start 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.415+0000 7f610ba1d640 1 -- start start 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 0x7f610419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61041094a0 0x7f610419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6104109df0 0x7f610419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6104113ab0 con 0x7f6104103b60 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6104113930 con 0x7f61041094a0 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6104113c30 con 0x7f6104109df0 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 0x7f610419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 0x7f610419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37742/0 (socket says 172.21.15.31:37742) 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 -- 172.21.15.31:0/959806931 learned_addr learned my addr 172.21.15.31:0/959806931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:25.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f6109fba640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6104109df0 0x7f610419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f6108fb8640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61041094a0 0x7f610419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 -- 172.21.15.31:0/959806931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6104109df0 msgr2=0x7f610419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6104109df0 0x7f610419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 -- 172.21.15.31:0/959806931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61041094a0 msgr2=0x7f610419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61041094a0 0x7f610419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 -- 172.21.15.31:0/959806931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61041a0470 con 0x7f6104103b60 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f61097b9640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 0x7f610419a500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f610000e9f0 tx=0x7f610000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000cd80 con 0x7f6104103b60 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61000108e0 con 0x7f6104103b60 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61041a0760 con 0x7f6104103b60 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.416+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100018980 con 0x7f6104103b60 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.417+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61041a6ec0 con 0x7f6104103b60 2024-09-21T02:44:25.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.417+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f6100010430 con 0x7f6104103b60 2024-09-21T02:44:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.418+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6104103f70 con 0x7f6104103b60 2024-09-21T02:44:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.418+0000 7f60fa7fc640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60f003fb70 0x7f60f0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.418+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f61000549d0 con 0x7f6104103b60 2024-09-21T02:44:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.418+0000 7f6108fb8640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60f003fb70 0x7f60f0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.418+0000 7f6108fb8640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60f003fb70 0x7f60f0042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009230 tx=0x7f60f4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.423+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6100014230 con 0x7f6104103b60 2024-09-21T02:44:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.614+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f610402e640 con 0x7f6104103b60 2024-09-21T02:44:25.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.617+0000 7f60fa7fc640 1 -- 172.21.15.31:0/959806931 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v18)=0 v18) ==== 119+0+0 (secure 0 0 0) 0x7f6100010a90 con 0x7f6104103b60 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60f003fb70 msgr2=0x7f60f0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60f003fb70 0x7f60f0042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009230 tx=0x7f60f4031400 comp rx=0 tx=0).stop 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 msgr2=0x7f610419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 0x7f610419a500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f610000e9f0 tx=0x7f610000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 shutdown_connections 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60f003fb70 0x7f60f0042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6104109df0 0x7f610419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61041094a0 0x7f610419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 --2- 172.21.15.31:0/959806931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6104103b60 0x7f610419a500 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.619+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 >> 172.21.15.31:0/959806931 conn(0x7f610407b850 msgr2=0x7f61041075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.620+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 shutdown_connections 2024-09-21T02:44:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.620+0000 7f610ba1d640 1 -- 172.21.15.31:0/959806931 wait complete. 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/432610236' entity='client.admin' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/959806931' entity='client.admin' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:26.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: Reconfiguring daemon mgr.x on smithi031 2024-09-21T02:44:26.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:26 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1377598202' entity='client.admin' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/432610236' entity='client.admin' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/959806931' entity='client.admin' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:26.523 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: Reconfiguring daemon mgr.x on smithi031 2024-09-21T02:44:26.524 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:26 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1377598202' entity='client.admin' 2024-09-21T02:44:26.592 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/432610236' entity='client.admin' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/959806931' entity='client.admin' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:44:26.593 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:44:26.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:26.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: Reconfiguring daemon mgr.x on smithi031 2024-09-21T02:44:26.594 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:26 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1377598202' entity='client.admin' 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 -- 172.21.15.31:0/1736468857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 msgr2=0x7f8148103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 --2- 172.21.15.31:0/1736468857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f8148103fa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ca0 tx=0x7f813802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 -- 172.21.15.31:0/1736468857 shutdown_connections 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 --2- 172.21.15.31:0/1736468857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148109e20 0x7f8148110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 --2- 172.21.15.31:0/1736468857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81481094d0 0x7f81481098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 --2- 172.21.15.31:0/1736468857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f8148103fa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 -- 172.21.15.31:0/1736468857 >> 172.21.15.31:0/1736468857 conn(0x7f81480ff600 msgr2=0x7f8148101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 -- 172.21.15.31:0/1736468857 shutdown_connections 2024-09-21T02:44:26.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.935+0000 7f815045e640 1 -- 172.21.15.31:0/1736468857 wait complete. 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 Processor -- start 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 -- start start 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f814819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81481094d0 0x7f814819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148109e20 0x7f814819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113b30 con 0x7f8148103b90 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f81481139b0 con 0x7f81481094d0 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.936+0000 7f815045e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8148113cb0 con 0x7f8148109e20 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f814819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f814819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37762/0 (socket says 172.21.15.31:37762) 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 -- 172.21.15.31:0/1377598202 learned_addr learned my addr 172.21.15.31:0/1377598202 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814d9f9640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81481094d0 0x7f814819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e9fb640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148109e20 0x7f814819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 -- 172.21.15.31:0/1377598202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148109e20 msgr2=0x7f814819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148109e20 0x7f814819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 -- 172.21.15.31:0/1377598202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81481094d0 msgr2=0x7f814819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81481094d0 0x7f814819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 -- 172.21.15.31:0/1377598202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81481a0410 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f814e1fa640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f814819a4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8138005e30 tx=0x7f813802f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138003bf0 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8138003d90 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481a06a0 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.937+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813802fb10 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.938+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a6e60 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.938+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f813802fcd0 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.938+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8148104e10 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.939+0000 7f81477fe640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f813003fb70 0x7f8130042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.939+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8138075940 con 0x7f8148103b90 2024-09-21T02:44:26.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.939+0000 7f814d9f9640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f813003fb70 0x7f8130042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.939+0000 7f814d9f9640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f813003fb70 0x7f8130042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8148108fe0 tx=0x7f813c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:25.944+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f813803cd10 con 0x7f8148103b90 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.138+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f8148103fa0 con 0x7f8148103b90 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.142+0000 7f81477fe640 1 -- 172.21.15.31:0/1377598202 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v19)=0 v19) ==== 116+0+0 (secure 0 0 0) 0x7f813802edb0 con 0x7f8148103b90 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f813003fb70 msgr2=0x7f8130042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f813003fb70 0x7f8130042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8148108fe0 tx=0x7f813c00b410 comp rx=0 tx=0).stop 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 msgr2=0x7f814819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f814819a4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8138005e30 tx=0x7f813802f3e0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 shutdown_connections 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f813003fb70 0x7f8130042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8148109e20 0x7f814819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81481094d0 0x7f814819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 --2- 172.21.15.31:0/1377598202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8148103b90 0x7f814819a4a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 >> 172.21.15.31:0/1377598202 conn(0x7f81480ff600 msgr2=0x7f8148101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 shutdown_connections 2024-09-21T02:44:26.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.145+0000 7f815045e640 1 -- 172.21.15.31:0/1377598202 wait complete. 2024-09-21T02:44:27.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:27 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3221744770' entity='client.admin' 2024-09-21T02:44:27.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:27 smithi031 ceph-mon[31209]: Standby manager daemon z started 2024-09-21T02:44:27.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:27 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T02:44:27.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:27 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:44:27.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:27 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T02:44:27.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:27 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:44:27.608 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:27 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3221744770' entity='client.admin' 2024-09-21T02:44:27.608 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:27 smithi165 ceph-mon[34818]: Standby manager daemon z started 2024-09-21T02:44:27.608 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:27 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T02:44:27.608 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:27 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:44:27.608 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:27 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T02:44:27.609 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:27 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:44:27.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:27 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3221744770' entity='client.admin' 2024-09-21T02:44:27.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:27 smithi178 ceph-mon[34759]: Standby manager daemon z started 2024-09-21T02:44:27.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:27 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T02:44:27.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:27 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:44:27.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:27 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T02:44:27.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:27 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.178:0/901046565' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 -- 172.21.15.31:0/2216365351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 msgr2=0x7f6438103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 --2- 172.21.15.31:0/2216365351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f6438103f70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6434009900 tx=0x7f643402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 -- 172.21.15.31:0/2216365351 shutdown_connections 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 --2- 172.21.15.31:0/2216365351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64381094a0 0x7f64381098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 --2- 172.21.15.31:0/2216365351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6438109df0 0x7f6438110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 --2- 172.21.15.31:0/2216365351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f6438103f70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 -- 172.21.15.31:0/2216365351 >> 172.21.15.31:0/2216365351 conn(0x7f643807b850 msgr2=0x7f643807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 -- 172.21.15.31:0/2216365351 shutdown_connections 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 -- 172.21.15.31:0/2216365351 wait complete. 2024-09-21T02:44:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.445+0000 7f643f442640 1 Processor -- start 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 -- start start 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f643819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64381094a0 0x7f643819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6438109df0 0x7f643819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6438113ab0 con 0x7f6438103b60 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6438113930 con 0x7f64381094a0 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643f442640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6438113c30 con 0x7f6438109df0 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f643819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f643819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37786/0 (socket says 172.21.15.31:37786) 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 -- 172.21.15.31:0/336755954 learned_addr learned my addr 172.21.15.31:0/336755954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643c9dd640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64381094a0 0x7f643819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d9df640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6438109df0 0x7f643819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 -- 172.21.15.31:0/336755954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6438109df0 msgr2=0x7f643819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6438109df0 0x7f643819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 -- 172.21.15.31:0/336755954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64381094a0 msgr2=0x7f643819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64381094a0 0x7f643819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d1de640 1 -- 172.21.15.31:0/336755954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381a0410 con 0x7f6438103b60 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.446+0000 7f643d9df640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6438109df0 0x7f643819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.447+0000 7f643d1de640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f643819a4a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6434005940 tx=0x7f6434004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.447+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643403f030 con 0x7f6438103b60 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.447+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6434003aa0 con 0x7f6438103b60 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.447+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434035070 con 0x7f6438103b60 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.447+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64381a06a0 con 0x7f6438103b60 2024-09-21T02:44:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.447+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381a6e60 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.448+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438104de0 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.449+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f6434042030 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.449+0000 7f642e7fc640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f642407b020 0x7f642407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.450+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f64340bd4e0 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.450+0000 7f643c9dd640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f642407b020 0x7f642407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.450+0000 7f643c9dd640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f642407b020 0x7f642407d530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6438106530 tx=0x7f642800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.454+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6434089ef0 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.645+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f643810bbc0 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.649+0000 7f642e7fc640 1 -- 172.21.15.31:0/336755954 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v21)=0 v21) ==== 108+0+0 (secure 0 0 0) 0x7f643408eda0 con 0x7f6438103b60 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f642407b020 msgr2=0x7f642407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f642407b020 0x7f642407d530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6438106530 tx=0x7f642800b410 comp rx=0 tx=0).stop 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 msgr2=0x7f643819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f643819a4a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6434005940 tx=0x7f6434004270 comp rx=0 tx=0).stop 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 shutdown_connections 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f642407b020 0x7f642407d530 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6438109df0 0x7f643819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64381094a0 0x7f643819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 --2- 172.21.15.31:0/336755954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6438103b60 0x7f643819a4a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 >> 172.21.15.31:0/336755954 conn(0x7f643807b850 msgr2=0x7f6438107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 shutdown_connections 2024-09-21T02:44:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:27.651+0000 7f643f442640 1 -- 172.21.15.31:0/336755954 wait complete. 2024-09-21T02:44:28.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:28 smithi031 ceph-mon[31209]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T02:44:28.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T02:44:28.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:28 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/336755954' entity='client.admin' 2024-09-21T02:44:28.476 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T02:44:28.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/3386413505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa410de70 msgr2=0x7f4aa4110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/3386413505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa410de70 0x7f4aa4110380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ca0 tx=0x7f4a9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/3386413505 shutdown_connections 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/3386413505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa410de70 0x7f4aa4110380 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/3386413505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4aa41052f0 0x7f4aa410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/3386413505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4aa41048c0 0x7f4aa4104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/3386413505 >> 172.21.15.31:0/3386413505 conn(0x7f4aa4079770 msgr2=0x7f4aa4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/3386413505 shutdown_connections 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/3386413505 wait complete. 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.142+0000 7f4aa9b6a640 1 Processor -- start 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 -- start start 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 0x7f4aa419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:28.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4aa41052f0 0x7f4aa419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4aa410de70 0x7f4aa419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4113b40 con 0x7f4aa41048c0 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa41139c0 con 0x7f4aa41052f0 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa9b6a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4113cc0 con 0x7f4aa410de70 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 0x7f4aa419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 0x7f4aa419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37796/0 (socket says 172.21.15.31:37796) 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 -- 172.21.15.31:0/519475247 learned_addr learned my addr 172.21.15.31:0/519475247 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa3fff640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4aa410de70 0x7f4aa419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa2ffd640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4aa41052f0 0x7f4aa419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 -- 172.21.15.31:0/519475247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4aa410de70 msgr2=0x7f4aa419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4aa410de70 0x7f4aa419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 -- 172.21.15.31:0/519475247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4aa41052f0 msgr2=0x7f4aa419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:28.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4aa41052f0 0x7f4aa419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 -- 172.21.15.31:0/519475247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa41a0460 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.143+0000 7f4aa37fe640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 0x7f4aa419a4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400d950 tx=0x7f4a9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.144+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94017070 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.144+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a94009c30 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.144+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa41a0750 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.144+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400f620 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.144+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa41a6eb0 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.145+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa406c200 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.146+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f4a94010460 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.146+0000 7f4aa0ff9640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4a8c07b020 0x7f4a8c07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.147+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4a9409c9f0 con 0x7f4aa41048c0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.147+0000 7f4aa2ffd640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4a8c07b020 0x7f4a8c07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:28.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.147+0000 7f4aa2ffd640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4a8c07b020 0x7f4a8c07d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4100170 tx=0x7f4a9800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.151+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a94009de0 con 0x7f4aa41048c0 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.342+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f4aa4104cd0 con 0x7f4aa41048c0 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.346+0000 7f4aa0ff9640 1 -- 172.21.15.31:0/519475247 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v22)=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f4a94069400 con 0x7f4aa41048c0 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4a8c07b020 msgr2=0x7f4a8c07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4a8c07b020 0x7f4a8c07d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4100170 tx=0x7f4a9800b410 comp rx=0 tx=0).stop 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 msgr2=0x7f4aa419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 0x7f4aa419a4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400d950 tx=0x7f4a9400de10 comp rx=0 tx=0).stop 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 shutdown_connections 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4a8c07b020 0x7f4a8c07d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4aa410de70 0x7f4aa419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4aa41052f0 0x7f4aa419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 --2- 172.21.15.31:0/519475247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aa41048c0 0x7f4aa419a4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:28.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 >> 172.21.15.31:0/519475247 conn(0x7f4aa4079770 msgr2=0x7f4aa4107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 shutdown_connections 2024-09-21T02:44:28.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:28.348+0000 7f4aa9b6a640 1 -- 172.21.15.31:0/519475247 wait complete. 2024-09-21T02:44:28.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:28 smithi165 ceph-mon[34818]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T02:44:28.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T02:44:28.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:28 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/336755954' entity='client.admin' 2024-09-21T02:44:28.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:28 smithi178 ceph-mon[34759]: mgrmap e19: x(active, since 12s), standbys: z 2024-09-21T02:44:28.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T02:44:28.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:28 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/336755954' entity='client.admin' 2024-09-21T02:44:29.190 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:29.436 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:29 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/519475247' entity='client.admin' 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.459+0000 7f5f2d729640 1 -- 172.21.15.31:0/240508579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28109df0 msgr2=0x7f5f28110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.459+0000 7f5f2d729640 1 --2- 172.21.15.31:0/240508579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28109df0 0x7f5f28110620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009ca0 tx=0x7f5f1802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 -- 172.21.15.31:0/240508579 shutdown_connections 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 --2- 172.21.15.31:0/240508579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f281094a0 0x7f5f281098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 --2- 172.21.15.31:0/240508579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28109df0 0x7f5f28110620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 --2- 172.21.15.31:0/240508579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f28103b60 0x7f5f28103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 -- 172.21.15.31:0/240508579 >> 172.21.15.31:0/240508579 conn(0x7f5f2807b850 msgr2=0x7f5f2807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 -- 172.21.15.31:0/240508579 shutdown_connections 2024-09-21T02:44:29.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 -- 172.21.15.31:0/240508579 wait complete. 2024-09-21T02:44:29.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 Processor -- start 2024-09-21T02:44:29.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.460+0000 7f5f2d729640 1 -- start start 2024-09-21T02:44:29.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f2d729640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 0x7f5f2819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f2d729640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f281094a0 0x7f5f2819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f2d729640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28109df0 0x7f5f2819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f2d729640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28113ab0 con 0x7f5f28103b60 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f2d729640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28113930 con 0x7f5f281094a0 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f2d729640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28113c30 con 0x7f5f28109df0 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 0x7f5f2819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 0x7f5f2819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59284/0 (socket says 172.21.15.31:59284) 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 -- 172.21.15.31:0/23950956 learned_addr learned my addr 172.21.15.31:0/23950956 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f267fc640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f281094a0 0x7f5f2819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f277fe640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28109df0 0x7f5f2819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 -- 172.21.15.31:0/23950956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28109df0 msgr2=0x7f5f2819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28109df0 0x7f5f2819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 -- 172.21.15.31:0/23950956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f281094a0 msgr2=0x7f5f2819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f281094a0 0x7f5f2819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.461+0000 7f5f26ffd640 1 -- 172.21.15.31:0/23950956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f281a03d0 con 0x7f5f28103b60 2024-09-21T02:44:29.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.462+0000 7f5f26ffd640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 0x7f5f2819a460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d950 tx=0x7f5f2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:29.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.462+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20017030 con 0x7f5f28103b60 2024-09-21T02:44:29.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.462+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f281a06c0 con 0x7f5f28103b60 2024-09-21T02:44:29.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.462+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f20004020 con 0x7f5f28103b60 2024-09-21T02:44:29.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.462+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200047a0 con 0x7f5f28103b60 2024-09-21T02:44:29.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.462+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281a6e20 con 0x7f5f28103b60 2024-09-21T02:44:29.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.463+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f28103f70 con 0x7f5f28103b60 2024-09-21T02:44:29.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.464+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f5f20009a30 con 0x7f5f28103b60 2024-09-21T02:44:29.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.464+0000 7f5f1ffff640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f1407b020 0x7f5f1407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.464+0000 7f5f267fc640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f1407b020 0x7f5f1407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.464+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5f2009c900 con 0x7f5f28103b60 2024-09-21T02:44:29.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.465+0000 7f5f267fc640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f1407b020 0x7f5f1407d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5f28109040 tx=0x7f5f1000a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:29.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.469+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f200691f0 con 0x7f5f28103b60 2024-09-21T02:44:29.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 -- 172.21.15.31:0/1624398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 msgr2=0x7f824410d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 --2- 172.21.15.31:0/1624398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824410d970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8238009900 tx=0x7f823802a410 comp rx=0 tx=0).stop 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 -- 172.21.15.31:0/1624398267 shutdown_connections 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 --2- 172.21.15.31:0/1624398267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824410deb0 0x7f8244110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 --2- 172.21.15.31:0/1624398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824410d970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 --2- 172.21.15.31:0/1624398267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8244104900 0x7f8244104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 -- 172.21.15.31:0/1624398267 >> 172.21.15.31:0/1624398267 conn(0x7f82440797b0 msgr2=0x7f8244079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 -- 172.21.15.31:0/1624398267 shutdown_connections 2024-09-21T02:44:29.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.628+0000 7f824aa06640 1 -- 172.21.15.31:0/1624398267 wait complete. 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 Processor -- start 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 -- start start 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8244104900 0x7f824419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824410deb0 0x7f824419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8244113af0 con 0x7f8244105250 2024-09-21T02:44:29.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8244113970 con 0x7f824410deb0 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.629+0000 7f824aa06640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8244113c70 con 0x7f8244104900 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f82437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824410deb0 0x7f824419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f82437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59300/0 (socket says 172.21.15.31:59300) 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f82437fe640 1 -- 172.21.15.31:0/3368120944 learned_addr learned my addr 172.21.15.31:0/3368120944 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8243fff640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8244104900 0x7f824419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 -- 172.21.15.31:0/3368120944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8244104900 msgr2=0x7f824419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8244104900 0x7f824419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 -- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 msgr2=0x7f824419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 -- 172.21.15.31:0/3368120944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441a03f0 con 0x7f824410deb0 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8243fff640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8244104900 0x7f824419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:44:29.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f8248fa3640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824410deb0 0x7f824419fcc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e940 tx=0x7f823c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:29.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c004d20 con 0x7f824410deb0 2024-09-21T02:44:29.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823c005020 con 0x7f824410deb0 2024-09-21T02:44:29.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0116d0 con 0x7f824410deb0 2024-09-21T02:44:29.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.630+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441a06e0 con 0x7f824410deb0 2024-09-21T02:44:29.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.631+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a6e40 con 0x7f824410deb0 2024-09-21T02:44:29.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.631+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244104d10 con 0x7f824410deb0 2024-09-21T02:44:29.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.632+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f823c011870 con 0x7f824410deb0 2024-09-21T02:44:29.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.633+0000 7f82417fa640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f822c07b020 0x7f822c07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:29.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.633+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f823c09d240 con 0x7f824410deb0 2024-09-21T02:44:29.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.633+0000 7f8243fff640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f822c07b020 0x7f822c07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:29.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.633+0000 7f8243fff640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f822c07b020 0x7f822c07d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8234005d00 tx=0x7f8234005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:29.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.638+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f823c016030 con 0x7f824410deb0 2024-09-21T02:44:29.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.664+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f5f2802e640 con 0x7f5f28103b60 2024-09-21T02:44:29.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.669+0000 7f5f1ffff640 1 -- 172.21.15.31:0/23950956 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f5f2006e0a0 con 0x7f5f28103b60 2024-09-21T02:44:29.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f1407b020 msgr2=0x7f5f1407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f1407b020 0x7f5f1407d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5f28109040 tx=0x7f5f1000a250 comp rx=0 tx=0).stop 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 msgr2=0x7f5f2819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 0x7f5f2819a460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000d950 tx=0x7f5f2000de10 comp rx=0 tx=0).stop 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 shutdown_connections 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f1407b020 0x7f5f1407d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f28109df0 0x7f5f2819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f281094a0 0x7f5f2819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 --2- 172.21.15.31:0/23950956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f28103b60 0x7f5f2819a460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 >> 172.21.15.31:0/23950956 conn(0x7f5f2807b850 msgr2=0x7f5f281074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 shutdown_connections 2024-09-21T02:44:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.671+0000 7f5f2d729640 1 -- 172.21.15.31:0/23950956 wait complete. 2024-09-21T02:44:29.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:29 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/519475247' entity='client.admin' 2024-09-21T02:44:29.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:29 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/519475247' entity='client.admin' 2024-09-21T02:44:29.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.834+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f82441a70a0 con 0x7f824410deb0 2024-09-21T02:44:29.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.839+0000 7f82417fa640 1 -- 172.21.15.31:0/3368120944 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v24)=0 v24) ==== 119+0+0 (secure 0 0 0) 0x7f823c06ce20 con 0x7f824410deb0 2024-09-21T02:44:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f822c07b020 msgr2=0x7f822c07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f822c07b020 0x7f822c07d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8234005d00 tx=0x7f8234005c70 comp rx=0 tx=0).stop 2024-09-21T02:44:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824410deb0 msgr2=0x7f824419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824410deb0 0x7f824419fcc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e940 tx=0x7f823c00ee00 comp rx=0 tx=0).stop 2024-09-21T02:44:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 shutdown_connections 2024-09-21T02:44:29.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f822c07b020 0x7f822c07d530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824410deb0 0x7f824419fcc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8244105250 0x7f824419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 --2- 172.21.15.31:0/3368120944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8244104900 0x7f824419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 >> 172.21.15.31:0/3368120944 conn(0x7f82440797b0 msgr2=0x7f82441079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 shutdown_connections 2024-09-21T02:44:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:29.842+0000 7f824aa06640 1 -- 172.21.15.31:0/3368120944 wait complete. 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: Standby manager daemon y started 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/23950956' entity='client.admin' 2024-09-21T02:44:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:30 smithi031 ceph-mon[31209]: from='client.? ' entity='client.admin' 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3892583292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4102ab0 msgr2=0x7fc5b4102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3892583292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4102ab0 0x7fc5b4102ec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009900 tx=0x7fc5a402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3892583292 shutdown_connections 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3892583292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b4108cf0 0x7fc5b410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3892583292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b41083a0 0x7fc5b41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3892583292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4102ab0 0x7fc5b4102ec0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3892583292 >> 172.21.15.31:0/3892583292 conn(0x7fc5b4079680 msgr2=0x7fc5b4079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3892583292 shutdown_connections 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.053+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3892583292 wait complete. 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 Processor -- start 2024-09-21T02:44:30.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 -- start start 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b4102ab0 0x7fc5b419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b41083a0 0x7fc5b419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 0x7fc5b419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41129b0 con 0x7fc5b4108cf0 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4112830 con 0x7fc5b4102ab0 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5ba93b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4112b30 con 0x7fc5b41083a0 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5b8ed8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 0x7fc5b419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5b8ed8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 0x7fc5b419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37776/0 (socket says 172.21.15.31:37776) 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5b8ed8640 1 -- 172.21.15.31:0/3221744770 learned_addr learned my addr 172.21.15.31:0/3221744770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5b3fff640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b4102ab0 0x7fc5b419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.054+0000 7fc5b37fe640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b41083a0 0x7fc5b419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b8ed8640 1 -- 172.21.15.31:0/3221744770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b41083a0 msgr2=0x7fc5b419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:30.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b8ed8640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b41083a0 0x7fc5b419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b8ed8640 1 -- 172.21.15.31:0/3221744770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b4102ab0 msgr2=0x7fc5b419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b8ed8640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b4102ab0 0x7fc5b419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b8ed8640 1 -- 172.21.15.31:0/3221744770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b41a0400 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b37fe640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b41083a0 0x7fc5b419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b8ed8640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 0x7fc5b419fcd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00eb10 tx=0x7fc5ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac004b20 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5ac004cc0 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b41a06f0 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac010670 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.055+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b41a6e50 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.056+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fc5ac010810 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.056+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b4103ce0 con 0x7fc5b4108cf0 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.056+0000 7fc5b17fa640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc59c03fb70 0x7fc59c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:30.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.056+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc5ac055af0 con 0x7fc5b4108cf0 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.057+0000 7fc5b3fff640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc59c03fb70 0x7fc59c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.057+0000 7fc5b3fff640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc59c03fb70 0x7fc59c042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4031440 tx=0x7fc5a4009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.062+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc5ac020e10 con 0x7fc5b4108cf0 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.275+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fc5b41a7130 con 0x7fc5b4108cf0 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.278+0000 7fc5b17fa640 1 -- 172.21.15.31:0/3221744770 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v20)=0 v20) ==== 127+0+0 (secure 0 0 0) 0x7fc5ac00f3c0 con 0x7fc5b4108cf0 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.280+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc59c03fb70 msgr2=0x7fc59c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.280+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc59c03fb70 0x7fc59c042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4031440 tx=0x7fc5a4009560 comp rx=0 tx=0).stop 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.280+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 msgr2=0x7fc5b419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.280+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 0x7fc5b419fcd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00eb10 tx=0x7fc5ac00c2d0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 shutdown_connections 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc59c03fb70 0x7fc59c042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5b41083a0 0x7fc5b419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5b4108cf0 0x7fc5b419fcd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 --2- 172.21.15.31:0/3221744770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b4102ab0 0x7fc5b419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 >> 172.21.15.31:0/3221744770 conn(0x7fc5b4079680 msgr2=0x7fc5b41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 shutdown_connections 2024-09-21T02:44:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:26.281+0000 7fc5ba93b640 1 -- 172.21.15.31:0/3221744770 wait complete. 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: Standby manager daemon y started 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/23950956' entity='client.admin' 2024-09-21T02:44:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:30 smithi165 ceph-mon[34818]: from='client.? ' entity='client.admin' 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: Standby manager daemon y started 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: from='mgr.? 172.21.15.165:0/459119924' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/23950956' entity='client.admin' 2024-09-21T02:44:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:30 smithi178 ceph-mon[34759]: from='client.? ' entity='client.admin' 2024-09-21T02:44:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:31.288 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:31.309 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:31.328 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T02:44:31.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:31 smithi031 ceph-mon[31209]: mgrmap e20: x(active, since 15s), standbys: z, y 2024-09-21T02:44:31.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T02:44:31.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:31 smithi165 ceph-mon[34818]: mgrmap e20: x(active, since 15s), standbys: z, y 2024-09-21T02:44:31.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T02:44:31.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:31 smithi178 ceph-mon[34759]: mgrmap e20: x(active, since 15s), standbys: z, y 2024-09-21T02:44:31.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.761+0000 7f794d32a640 1 -- 172.21.15.31:0/2664512037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 msgr2=0x7f794810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.761+0000 7f794d32a640 1 --2- 172.21.15.31:0/2664512037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794810f530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7940009ca0 tx=0x7f794002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.761+0000 7f794d32a640 1 -- 172.21.15.31:0/2664512037 shutdown_connections 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.761+0000 7f794d32a640 1 --2- 172.21.15.31:0/2664512037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79481083b0 0x7f79481087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.762+0000 7f794d32a640 1 --2- 172.21.15.31:0/2664512037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794810f530 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.762+0000 7f794d32a640 1 --2- 172.21.15.31:0/2664512037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7948102a70 0x7f7948102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.762+0000 7f794d32a640 1 -- 172.21.15.31:0/2664512037 >> 172.21.15.31:0/2664512037 conn(0x7f7948079730 msgr2=0x7f7948079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.762+0000 7f794d32a640 1 -- 172.21.15.31:0/2664512037 shutdown_connections 2024-09-21T02:44:31.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.762+0000 7f794d32a640 1 -- 172.21.15.31:0/2664512037 wait complete. 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.762+0000 7f794d32a640 1 Processor -- start 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 -- start start 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7948102a70 0x7f794819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79481083b0 0x7f794819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7948113a90 con 0x7f7948108d00 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7948113910 con 0x7f79481083b0 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f794d32a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7948113c10 con 0x7f7948102a70 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f79477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f79477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59308/0 (socket says 172.21.15.31:59308) 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f79477fe640 1 -- 172.21.15.31:0/2550154036 learned_addr learned my addr 172.21.15.31:0/2550154036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:31.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f7946ffd640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7948102a70 0x7f794819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.763+0000 7f79467fc640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79481083b0 0x7f794819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f79477fe640 1 -- 172.21.15.31:0/2550154036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7948102a70 msgr2=0x7f794819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f79477fe640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7948102a70 0x7f794819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f79477fe640 1 -- 172.21.15.31:0/2550154036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79481083b0 msgr2=0x7f794819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f79477fe640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79481083b0 0x7f794819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f79477fe640 1 -- 172.21.15.31:0/2550154036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79481a0410 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f79477fe640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794819fce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7940005cc0 tx=0x7f794002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940003bf0 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7940003d90 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794002fbc0 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79481a06a0 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.764+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79481a6e60 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.765+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7948103cf0 con 0x7f7948108d00 2024-09-21T02:44:31.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.767+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f794002fd60 con 0x7f7948108d00 2024-09-21T02:44:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.768+0000 7f792ffff640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f79300b6550 0x7f79300b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.768+0000 7f7946ffd640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f79300b6550 0x7f79300b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.768+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7940106d10 con 0x7f7948108d00 2024-09-21T02:44:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.768+0000 7f7946ffd640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f79300b6550 0x7f79300b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f793800bbe0 tx=0x7f793800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.771+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79400cf100 con 0x7f7948108d00 2024-09-21T02:44:31.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.959+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f7948102e80 con 0x7f7948108d00 2024-09-21T02:44:31.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.963+0000 7f792ffff640 1 -- 172.21.15.31:0/2550154036 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v25)=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f7948102e80 con 0x7f7948108d00 2024-09-21T02:44:31.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f79300b6550 msgr2=0x7f79300b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:31.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f79300b6550 0x7f79300b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f793800bbe0 tx=0x7f793800b390 comp rx=0 tx=0).stop 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 msgr2=0x7f794819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794819fce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7940005cc0 tx=0x7f794002f380 comp rx=0 tx=0).stop 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 shutdown_connections 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f79300b6550 0x7f79300b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7948108d00 0x7f794819fce0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79481083b0 0x7f794819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 --2- 172.21.15.31:0/2550154036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7948102a70 0x7f794819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 >> 172.21.15.31:0/2550154036 conn(0x7f7948079730 msgr2=0x7f7948104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 shutdown_connections 2024-09-21T02:44:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:31.966+0000 7f794d32a640 1 -- 172.21.15.31:0/2550154036 wait complete. 2024-09-21T02:44:32.720 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi031') 2024-09-21T02:44:33.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:32 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2550154036' entity='client.admin' 2024-09-21T02:44:33.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:32 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2717998382' entity='client.admin' 2024-09-21T02:44:33.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:32 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2550154036' entity='client.admin' 2024-09-21T02:44:33.272 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:32 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2717998382' entity='client.admin' 2024-09-21T02:44:33.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:32 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2550154036' entity='client.admin' 2024-09-21T02:44:33.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:32 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2717998382' entity='client.admin' 2024-09-21T02:44:35.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.012+0000 7fcaf7e30640 1 -- 172.21.15.31:0/757670861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0077380 msgr2=0x7fcaf00757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:35.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.012+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/757670861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0077380 0x7fcaf00757b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009900 tx=0x7fcaec02a410 comp rx=0 tx=0).stop 2024-09-21T02:44:35.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 -- 172.21.15.31:0/757670861 shutdown_connections 2024-09-21T02:44:35.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/757670861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 0x7fcaf0116950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/757670861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf0075dd0 0x7fcaf00761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/757670861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0077380 0x7fcaf00757b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 -- 172.21.15.31:0/757670861 >> 172.21.15.31:0/757670861 conn(0x7fcaf00ff600 msgr2=0x7fcaf0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 -- 172.21.15.31:0/757670861 shutdown_connections 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 -- 172.21.15.31:0/757670861 wait complete. 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 Processor -- start 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.013+0000 7fcaf7e30640 1 -- start start 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf7e30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 0x7fcaf01a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf7e30640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf0077380 0x7fcaf01a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf7e30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 0x7fcaf01a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf7e30640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf011a120 con 0x7fcaf0075dd0 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf7e30640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0119fa0 con 0x7fcaf0077380 2024-09-21T02:44:35.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf7e30640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf011a2a0 con 0x7fcaf0114480 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 0x7fcaf01a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 0x7fcaf01a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59330/0 (socket says 172.21.15.31:59330) 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 -- 172.21.15.31:0/2717998382 learned_addr learned my addr 172.21.15.31:0/2717998382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf53cb640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf0077380 0x7fcaf01a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf63cd640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 0x7fcaf01a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 -- 172.21.15.31:0/2717998382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 msgr2=0x7fcaf01a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 0x7fcaf01a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 -- 172.21.15.31:0/2717998382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf0077380 msgr2=0x7fcaf01a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf0077380 0x7fcaf01a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 -- 172.21.15.31:0/2717998382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf01a6a60 con 0x7fcaf0075dd0 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf63cd640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 0x7fcaf01a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.014+0000 7fcaf5bcc640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 0x7fcaf01a0af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005ab0 tx=0x7fcaec003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:35.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.015+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec03f070 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.015+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec031d60 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.015+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf01a6cf0 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.015+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec035030 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.015+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf01ad4b0 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.016+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00761e0 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.017+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fcaec042070 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.018+0000 7fcae6ffd640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6550 0x7fcadc0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.018+0000 7fcaf53cb640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6550 0x7fcadc0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.018+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcaec106050 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.019+0000 7fcaf53cb640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6550 0x7fcadc0b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcaf01164a0 tx=0x7fcae000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.022+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcaec0ce4e0 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.215+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fcaf01a1c20 con 0x7fcaf0075dd0 2024-09-21T02:44:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.221+0000 7fcae6ffd640 1 -- 172.21.15.31:0/2717998382 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fcaec0d28c0 con 0x7fcaf0075dd0 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.223+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6550 msgr2=0x7fcadc0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.223+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6550 0x7fcadc0b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcaf01164a0 tx=0x7fcae000b410 comp rx=0 tx=0).stop 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 msgr2=0x7fcaf01a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 0x7fcaf01a0af0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005ab0 tx=0x7fcaec003cd0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 shutdown_connections 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6550 0x7fcadc0b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf0114480 0x7fcaf01a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf0077380 0x7fcaf01a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 --2- 172.21.15.31:0/2717998382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf0075dd0 0x7fcaf01a0af0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 >> 172.21.15.31:0/2717998382 conn(0x7fcaf00ff600 msgr2=0x7fcaf0110380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 shutdown_connections 2024-09-21T02:44:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:32.224+0000 7fcaf7e30640 1 -- 172.21.15.31:0/2717998382 wait complete. 2024-09-21T02:44:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:36.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 -- 172.21.15.31:0/3020695476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8103b60 msgr2=0x7f72a8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 --2- 172.21.15.31:0/3020695476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8103b60 0x7f72a8103f70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009900 tx=0x7f72a402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:36.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 -- 172.21.15.31:0/3020695476 shutdown_connections 2024-09-21T02:44:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 --2- 172.21.15.31:0/3020695476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a8109df0 0x7f72a8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 --2- 172.21.15.31:0/3020695476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a81094a0 0x7f72a81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 --2- 172.21.15.31:0/3020695476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8103b60 0x7f72a8103f70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 -- 172.21.15.31:0/3020695476 >> 172.21.15.31:0/3020695476 conn(0x7f72a807b850 msgr2=0x7f72a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 -- 172.21.15.31:0/3020695476 shutdown_connections 2024-09-21T02:44:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.232+0000 7f72b0358640 1 -- 172.21.15.31:0/3020695476 wait complete. 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 Processor -- start 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 -- start start 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a8103b60 0x7f72a819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a81094a0 0x7f72a819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 0x7f72a819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8113b10 con 0x7f72a8109df0 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8113990 con 0x7f72a8103b60 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72b0358640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8113c90 con 0x7f72a81094a0 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 0x7f72a819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 0x7f72a819fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59348/0 (socket says 172.21.15.31:59348) 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 -- 172.21.15.31:0/3999532230 learned_addr learned my addr 172.21.15.31:0/3999532230 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ad8f3640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a81094a0 0x7f72a819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae0f4640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a8103b60 0x7f72a819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 -- 172.21.15.31:0/3999532230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a81094a0 msgr2=0x7f72a819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a81094a0 0x7f72a819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 -- 172.21.15.31:0/3999532230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a8103b60 msgr2=0x7f72a819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a8103b60 0x7f72a819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.233+0000 7f72ae8f5640 1 -- 172.21.15.31:0/3999532230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a81a0550 con 0x7f72a8109df0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.234+0000 7f72ae8f5640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 0x7f72a819fe20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72a000eaa0 tx=0x7f72a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.234+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0004d20 con 0x7f72a8109df0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.234+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a81a0840 con 0x7f72a8109df0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.234+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a0005020 con 0x7f72a8109df0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.234+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00116d0 con 0x7f72a8109df0 2024-09-21T02:44:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.234+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81a6fa0 con 0x7f72a8109df0 2024-09-21T02:44:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.235+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a8103f70 con 0x7f72a8109df0 2024-09-21T02:44:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.237+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f72a000caf0 con 0x7f72a8109df0 2024-09-21T02:44:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.238+0000 7f729f7fe640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72940b64d0 0x7f72940b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.238+0000 7f72ae0f4640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72940b64d0 0x7f72940b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.238+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f72a00e5c40 con 0x7f72a8109df0 2024-09-21T02:44:36.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.239+0000 7f72ae0f4640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72940b64d0 0x7f72940b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f72a4031440 tx=0x7f72a4005d10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:36.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.241+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72a0016030 con 0x7f72a8109df0 2024-09-21T02:44:36.329 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:36 smithi031 ceph-mon[31209]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:36.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.434+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f72a819b710 con 0x7f72a8109df0 2024-09-21T02:44:36.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.438+0000 7f729f7fe640 1 -- 172.21.15.31:0/3999532230 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v27)=0 v27) ==== 106+0+0 (secure 0 0 0) 0x7f72a819b710 con 0x7f72a8109df0 2024-09-21T02:44:36.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:36 smithi165 ceph-mon[34818]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72940b64d0 msgr2=0x7f72940b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72940b64d0 0x7f72940b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f72a4031440 tx=0x7f72a4005d10 comp rx=0 tx=0).stop 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 msgr2=0x7f72a819fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 0x7f72a819fe20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72a000eaa0 tx=0x7f72a000eea0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 shutdown_connections 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72940b64d0 0x7f72940b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a81094a0 0x7f72a819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72a8109df0 0x7f72a819fe20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 --2- 172.21.15.31:0/3999532230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72a8103b60 0x7f72a819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.440+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 >> 172.21.15.31:0/3999532230 conn(0x7f72a807b850 msgr2=0x7f72a8107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.441+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 shutdown_connections 2024-09-21T02:44:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.441+0000 7f72b0358640 1 -- 172.21.15.31:0/3999532230 wait complete. 2024-09-21T02:44:36.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.424+0000 7fb179348640 1 -- 172.21.15.31:0/3997987154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 msgr2=0x7fb174103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.424+0000 7fb179348640 1 --2- 172.21.15.31:0/3997987154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb174103f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb164009900 tx=0x7fb16402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 -- 172.21.15.31:0/3997987154 shutdown_connections 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 --2- 172.21.15.31:0/3997987154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 0x7fb174110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 --2- 172.21.15.31:0/3997987154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1741094a0 0x7fb1741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 --2- 172.21.15.31:0/3997987154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb174103f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 -- 172.21.15.31:0/3997987154 >> 172.21.15.31:0/3997987154 conn(0x7fb17407b850 msgr2=0x7fb17407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 -- 172.21.15.31:0/3997987154 shutdown_connections 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.425+0000 7fb179348640 1 -- 172.21.15.31:0/3997987154 wait complete. 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 Processor -- start 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 -- start start 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb17419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1741094a0 0x7fb17419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 0x7fb17419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113ab0 con 0x7fb174103b60 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113930 con 0x7fb1741094a0 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb179348640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113c30 con 0x7fb174109df0 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb172ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb17419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb172ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb17419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59380/0 (socket says 172.21.15.31:59380) 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb172ffd640 1 -- 172.21.15.31:0/2878780579 learned_addr learned my addr 172.21.15.31:0/2878780579 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.426+0000 7fb1727fc640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1741094a0 0x7fb17419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb1737fe640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 0x7fb17419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb172ffd640 1 -- 172.21.15.31:0/2878780579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 msgr2=0x7fb17419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb172ffd640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 0x7fb17419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb172ffd640 1 -- 172.21.15.31:0/2878780579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1741094a0 msgr2=0x7fb17419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb172ffd640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1741094a0 0x7fb17419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb172ffd640 1 -- 172.21.15.31:0/2878780579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1741a0410 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb1737fe640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 0x7fb17419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb172ffd640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb17419a4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb164005940 tx=0x7fb1640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16403f030 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741a06a0 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb164003cb0 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164035070 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.427+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741a6e60 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.428+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb174104de0 con 0x7fb174103b60 2024-09-21T02:44:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.430+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fb164042030 con 0x7fb174103b60 2024-09-21T02:44:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.431+0000 7fb15bfff640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb15c0b64d0 0x7fb15c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.431+0000 7fb1727fc640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb15c0b64d0 0x7fb15c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.431+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb164106240 con 0x7fb174103b60 2024-09-21T02:44:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.431+0000 7fb1727fc640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb15c0b64d0 0x7fb15c0b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb174106530 tx=0x7fb16800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.434+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1640ce6d0 con 0x7fb174103b60 2024-09-21T02:44:36.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:36 smithi178 ceph-mon[34759]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.626+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb174103f70 con 0x7fb174103b60 2024-09-21T02:44:36.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.631+0000 7fb15bfff640 1 -- 172.21.15.31:0/2878780579 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fb1640d2ab0 con 0x7fb174103b60 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb15c0b64d0 msgr2=0x7fb15c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb15c0b64d0 0x7fb15c0b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb174106530 tx=0x7fb16800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 msgr2=0x7fb17419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb17419a4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb164005940 tx=0x7fb1640042f0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 shutdown_connections 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb15c0b64d0 0x7fb15c0b89e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb174109df0 0x7fb17419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1741094a0 0x7fb17419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 --2- 172.21.15.31:0/2878780579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb174103b60 0x7fb17419a4a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.633+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 >> 172.21.15.31:0/2878780579 conn(0x7fb17407b850 msgr2=0x7fb174107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:36.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.634+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 shutdown_connections 2024-09-21T02:44:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:36.634+0000 7fb179348640 1 -- 172.21.15.31:0/2878780579 wait complete. 2024-09-21T02:44:37.533 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:37 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3999532230' entity='client.admin' 2024-09-21T02:44:37.533 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:37 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2878780579' entity='client.admin' 2024-09-21T02:44:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 -- 172.21.15.31:0/2468759286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 msgr2=0x7f87c8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 --2- 172.21.15.31:0/2468759286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c8110300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009ca0 tx=0x7f87c002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 -- 172.21.15.31:0/2468759286 shutdown_connections 2024-09-21T02:44:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 --2- 172.21.15.31:0/2468759286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c8110300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 --2- 172.21.15.31:0/2468759286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c8105300 0x7f87c810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 --2- 172.21.15.31:0/2468759286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87c81049b0 0x7f87c8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 -- 172.21.15.31:0/2468759286 >> 172.21.15.31:0/2468759286 conn(0x7f87c8079730 msgr2=0x7f87c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.633+0000 7f87cec0a640 1 -- 172.21.15.31:0/2468759286 shutdown_connections 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.634+0000 7f87cec0a640 1 -- 172.21.15.31:0/2468759286 wait complete. 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.634+0000 7f87cec0a640 1 Processor -- start 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 -- start start 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87c81049b0 0x7f87c819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c8105300 0x7f87c819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8113ae0 con 0x7f87c810de10 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8113960 con 0x7f87c8105300 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cec0a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8113c60 con 0x7f87c81049b0 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:37.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c819fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59418/0 (socket says 172.21.15.31:59418) 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 -- 172.21.15.31:0/973227183 learned_addr learned my addr 172.21.15.31:0/973227183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cc9a6640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87c81049b0 0x7f87c819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87c7fff640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c8105300 0x7f87c819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 -- 172.21.15.31:0/973227183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87c81049b0 msgr2=0x7f87c819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87c81049b0 0x7f87c819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 -- 172.21.15.31:0/973227183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c8105300 msgr2=0x7f87c819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c8105300 0x7f87c819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.635+0000 7f87cd1a7640 1 -- 172.21.15.31:0/973227183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c81a0420 con 0x7f87c810de10 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.636+0000 7f87cd1a7640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c819fcf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87c002f540 tx=0x7f87c00316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.636+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0003b70 con 0x7f87c810de10 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.636+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c81a06b0 con 0x7f87c810de10 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.636+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c0003d10 con 0x7f87c810de10 2024-09-21T02:44:37.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.636+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c002f990 con 0x7f87c810de10 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.636+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c81a6e70 con 0x7f87c810de10 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.637+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c8104dc0 con 0x7f87c810de10 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.640+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f87c0004b40 con 0x7f87c810de10 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.640+0000 7f87c5ffb640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87b80b64d0 0x7f87b80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.641+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f87c0106c50 con 0x7f87c810de10 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.641+0000 7f87cc9a6640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87b80b64d0 0x7f87b80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.641+0000 7f87cc9a6640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87b80b64d0 0x7f87b80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87c806bc60 tx=0x7f87bc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:37.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.643+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87c00cf0e0 con 0x7f87c810de10 2024-09-21T02:44:37.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:37 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3999532230' entity='client.admin' 2024-09-21T02:44:37.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:37 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2878780579' entity='client.admin' 2024-09-21T02:44:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:37 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3999532230' entity='client.admin' 2024-09-21T02:44:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:37 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2878780579' entity='client.admin' 2024-09-21T02:44:37.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.834+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f87c819b5e0 con 0x7f87c810de10 2024-09-21T02:44:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.838+0000 7f87c5ffb640 1 -- 172.21.15.31:0/973227183 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v30)=0 v30) ==== 111+0+0 (secure 0 0 0) 0x7f87c00d34c0 con 0x7f87c810de10 2024-09-21T02:44:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87b80b64d0 msgr2=0x7f87b80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87b80b64d0 0x7f87b80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87c806bc60 tx=0x7f87bc005f90 comp rx=0 tx=0).stop 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 msgr2=0x7f87c819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c819fcf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87c002f540 tx=0x7f87c00316f0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 shutdown_connections 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87b80b64d0 0x7f87b80b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c810de10 0x7f87c819fcf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87c8105300 0x7f87c819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 --2- 172.21.15.31:0/973227183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87c81049b0 0x7f87c819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 >> 172.21.15.31:0/973227183 conn(0x7f87c8079730 msgr2=0x7f87c80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 shutdown_connections 2024-09-21T02:44:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.841+0000 7f87cec0a640 1 -- 172.21.15.31:0/973227183 wait complete. 2024-09-21T02:44:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:38.531 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi031') 2024-09-21T02:44:38.532 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi031') 2024-09-21T02:44:38.749 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:38 smithi031 ceph-mon[31209]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:38.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1930323821' entity='client.admin' 2024-09-21T02:44:38.750 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/973227183' entity='client.admin' 2024-09-21T02:44:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:38 smithi165 ceph-mon[34818]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:38 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1930323821' entity='client.admin' 2024-09-21T02:44:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:38 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/973227183' entity='client.admin' 2024-09-21T02:44:39.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:38 smithi178 ceph-mon[34759]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:39.231 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:38 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1930323821' entity='client.admin' 2024-09-21T02:44:39.231 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:38 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/973227183' entity='client.admin' 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 -- 172.21.15.31:0/981866093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 msgr2=0x7f44b0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 --2- 172.21.15.31:0/981866093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b0110640 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009ca0 tx=0x7f44a802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 -- 172.21.15.31:0/981866093 shutdown_connections 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 --2- 172.21.15.31:0/981866093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b0110640 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 --2- 172.21.15.31:0/981866093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44b01094c0 0x7f44b01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 --2- 172.21.15.31:0/981866093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b0103b80 0x7f44b0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.284+0000 7f44b50b3640 1 -- 172.21.15.31:0/981866093 >> 172.21.15.31:0/981866093 conn(0x7f44b00ff5b0 msgr2=0x7f44b0101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.285+0000 7f44b50b3640 1 -- 172.21.15.31:0/981866093 shutdown_connections 2024-09-21T02:44:39.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.285+0000 7f44b50b3640 1 -- 172.21.15.31:0/981866093 wait complete. 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.285+0000 7f44b50b3640 1 Processor -- start 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 -- start start 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44b0103b80 0x7f44b019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b01094c0 0x7f44b019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0113ad0 con 0x7f44b0109e10 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0113950 con 0x7f44b0103b80 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44b50b3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0113c50 con 0x7f44b01094c0 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:39.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58570/0 (socket says 172.21.15.31:58570) 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 -- 172.21.15.31:0/315506936 learned_addr learned my addr 172.21.15.31:0/315506936 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44aed9d640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44b0103b80 0x7f44b019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44ae59c640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b01094c0 0x7f44b019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 -- 172.21.15.31:0/315506936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b01094c0 msgr2=0x7f44b019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b01094c0 0x7f44b019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 -- 172.21.15.31:0/315506936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44b0103b80 msgr2=0x7f44b019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44b0103b80 0x7f44b019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.286+0000 7f44af59e640 1 -- 172.21.15.31:0/315506936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b01a0430 con 0x7f44b0109e10 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.287+0000 7f44af59e640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b019fd00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005cc0 tx=0x7f44a802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.287+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003bf0 con 0x7f44b0109e10 2024-09-21T02:44:39.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.287+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b01a06c0 con 0x7f44b0109e10 2024-09-21T02:44:39.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.287+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a8003d90 con 0x7f44b0109e10 2024-09-21T02:44:39.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.287+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a802fbc0 con 0x7f44b0109e10 2024-09-21T02:44:39.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.287+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b01a6e80 con 0x7f44b0109e10 2024-09-21T02:44:39.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.288+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b0104e00 con 0x7f44b0109e10 2024-09-21T02:44:39.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.289+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f44a802fd60 con 0x7f44b0109e10 2024-09-21T02:44:39.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.290+0000 7f4497fff640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44980b64d0 0x7f44980b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:39.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.290+0000 7f44aed9d640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44980b64d0 0x7f44980b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:39.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.290+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f44a8106c70 con 0x7f44b0109e10 2024-09-21T02:44:39.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.291+0000 7f44aed9d640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44980b64d0 0x7f44980b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44a00058c0 tx=0x7f44a000a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:39.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.294+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44a80cf100 con 0x7f44b0109e10 2024-09-21T02:44:39.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.483+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f44b010bc30 con 0x7f44b0109e10 2024-09-21T02:44:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.487+0000 7f4497fff640 1 -- 172.21.15.31:0/315506936 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v31)=0 v31) ==== 120+0+0 (secure 0 0 0) 0x7f44a80098b0 con 0x7f44b0109e10 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44980b64d0 msgr2=0x7f44980b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44980b64d0 0x7f44980b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44a00058c0 tx=0x7f44a000a250 comp rx=0 tx=0).stop 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 msgr2=0x7f44b019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b019fd00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005cc0 tx=0x7f44a802f380 comp rx=0 tx=0).stop 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 shutdown_connections 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44980b64d0 0x7f44980b89e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b01094c0 0x7f44b019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b0109e10 0x7f44b019fd00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 --2- 172.21.15.31:0/315506936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44b0103b80 0x7f44b019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.489+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 >> 172.21.15.31:0/315506936 conn(0x7f44b00ff5b0 msgr2=0x7f44b0108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.490+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 shutdown_connections 2024-09-21T02:44:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:39.490+0000 7f44b50b3640 1 -- 172.21.15.31:0/315506936 wait complete. 2024-09-21T02:44:39.737 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.737 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi031') 2024-09-21T02:44:39.738 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-21T02:44:40.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 -- 172.21.15.31:0/56199389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 msgr2=0x7fb2781098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:40.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 --2- 172.21.15.31:0/56199389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb2781098b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb268009900 tx=0x7fb26802a410 comp rx=0 tx=0).stop 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 -- 172.21.15.31:0/56199389 shutdown_connections 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 --2- 172.21.15.31:0/56199389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb278109df0 0x7fb278110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 --2- 172.21.15.31:0/56199389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb2781098b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 --2- 172.21.15.31:0/56199389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb278103b60 0x7fb278103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 -- 172.21.15.31:0/56199389 >> 172.21.15.31:0/56199389 conn(0x7fb27807b850 msgr2=0x7fb27807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 -- 172.21.15.31:0/56199389 shutdown_connections 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.531+0000 7fb27ff70640 1 -- 172.21.15.31:0/56199389 wait complete. 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 Processor -- start 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 -- start start 2024-09-21T02:44:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb278103b60 0x7fb27819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb27819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb278109df0 0x7fb27819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb278113af0 con 0x7fb2781094a0 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb278113970 con 0x7fb278109df0 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.532+0000 7fb27ff70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb278113c70 con 0x7fb278103b60 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb27819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb27819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59394/0 (socket says 172.21.15.31:59394) 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 -- 172.21.15.31:0/1930323821 learned_addr learned my addr 172.21.15.31:0/1930323821 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27dd0c640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb278103b60 0x7fb27819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27e50d640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb278109df0 0x7fb27819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:40.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 -- 172.21.15.31:0/1930323821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb278103b60 msgr2=0x7fb27819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb278103b60 0x7fb27819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 -- 172.21.15.31:0/1930323821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb278109df0 msgr2=0x7fb27819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb278109df0 0x7fb27819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 -- 172.21.15.31:0/1930323821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2781a0450 con 0x7fb2781094a0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27d50b640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb27819aa80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb26802a930 tx=0x7fb268004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26803f030 con 0x7fb2781094a0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb268003aa0 con 0x7fb2781094a0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781a06e0 con 0x7fb2781094a0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268035070 con 0x7fb2781094a0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.533+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a6ea0 con 0x7fb2781094a0 2024-09-21T02:44:40.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.535+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278103f70 con 0x7fb2781094a0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.537+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fb268042030 con 0x7fb2781094a0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.538+0000 7fb26effd640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb2640b64d0 0x7fb2640b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.538+0000 7fb27dd0c640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb2640b64d0 0x7fb2640b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.538+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb268105e10 con 0x7fb2781094a0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.538+0000 7fb27dd0c640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb2640b64d0 0x7fb2640b89e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb274005fd0 tx=0x7fb27400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.542+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb2680ce2a0 con 0x7fb2781094a0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.736+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fb27810bbe0 con 0x7fb2781094a0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.740+0000 7fb26effd640 1 -- 172.21.15.31:0/1930323821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7fb2680d2680 con 0x7fb2781094a0 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb2640b64d0 msgr2=0x7fb2640b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:40.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb2640b64d0 0x7fb2640b89e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb274005fd0 tx=0x7fb27400b390 comp rx=0 tx=0).stop 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 msgr2=0x7fb27819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb27819aa80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb26802a930 tx=0x7fb268004270 comp rx=0 tx=0).stop 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 shutdown_connections 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb2640b64d0 0x7fb2640b89e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb278109df0 0x7fb27819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2781094a0 0x7fb27819aa80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 --2- 172.21.15.31:0/1930323821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb278103b60 0x7fb27819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 >> 172.21.15.31:0/1930323821 conn(0x7fb27807b850 msgr2=0x7fb2781054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 shutdown_connections 2024-09-21T02:44:40.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:37.743+0000 7fb27ff70640 1 -- 172.21.15.31:0/1930323821 wait complete. 2024-09-21T02:44:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:40 smithi031 ceph-mon[31209]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:40 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/315506936' entity='client.admin' 2024-09-21T02:44:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:40 smithi165 ceph-mon[34818]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:40 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/315506936' entity='client.admin' 2024-09-21T02:44:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:40 smithi178 ceph-mon[34759]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:40 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/315506936' entity='client.admin' 2024-09-21T02:44:41.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 -- 172.21.15.31:0/1099292866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc1083d0 msgr2=0x7f26fc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:41.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 --2- 172.21.15.31:0/1099292866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc1083d0 0x7f26fc1087e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009900 tx=0x7f26e402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:41.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 -- 172.21.15.31:0/1099292866 shutdown_connections 2024-09-21T02:44:41.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 --2- 172.21.15.31:0/1099292866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26fc108d20 0x7f26fc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 --2- 172.21.15.31:0/1099292866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc1083d0 0x7f26fc1087e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 --2- 172.21.15.31:0/1099292866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26fc102ae0 0x7f26fc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 -- 172.21.15.31:0/1099292866 >> 172.21.15.31:0/1099292866 conn(0x7f26fc0797b0 msgr2=0x7f26fc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 -- 172.21.15.31:0/1099292866 shutdown_connections 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.516+0000 7f2702665640 1 -- 172.21.15.31:0/1099292866 wait complete. 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.517+0000 7f2702665640 1 Processor -- start 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.517+0000 7f2702665640 1 -- start start 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 0x7f26fc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26fc1083d0 0x7f26fc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26fc108d20 0x7f26fc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1129e0 con 0x7f26fc102ae0 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc112860 con 0x7f26fc1083d0 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc112b60 con 0x7f26fc108d20 2024-09-21T02:44:41.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 0x7f26fc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 0x7f26fc19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58598/0 (socket says 172.21.15.31:58598) 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 -- 172.21.15.31:0/3055020712 learned_addr learned my addr 172.21.15.31:0/3055020712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26f3fff640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26fc1083d0 0x7f26fc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 -- 172.21.15.31:0/3055020712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26fc108d20 msgr2=0x7f26fc19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26fc108d20 0x7f26fc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 -- 172.21.15.31:0/3055020712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26fc1083d0 msgr2=0x7f26fc19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26fc1083d0 0x7f26fc19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 -- 172.21.15.31:0/3055020712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc1a04f0 con 0x7f26fc102ae0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26fbfff640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 0x7f26fc19a580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00d950 tx=0x7f26ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec017070 con 0x7f26fc102ae0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ec009c30 con 0x7f26fc102ae0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc1a07e0 con 0x7f26fc102ae0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00f620 con 0x7f26fc102ae0 2024-09-21T02:44:41.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.518+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1a6f40 con 0x7f26fc102ae0 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.520+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc102ef0 con 0x7f26fc102ae0 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.521+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f26ec005030 con 0x7f26fc102ae0 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.522+0000 7f26f9ffb640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f26e80b6550 0x7f26e80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.523+0000 7f26f3fff640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f26e80b6550 0x7f26e80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.523+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f26ec0e5350 con 0x7f26fc102ae0 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.523+0000 7f26f3fff640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f26e80b6550 0x7f26e80b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009230 tx=0x7f26e4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:41.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.526+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26ec0ad860 con 0x7f26fc102ae0 2024-09-21T02:44:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.714+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f26fc02e640 con 0x7f26fc102ae0 2024-09-21T02:44:41.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.718+0000 7f26f9ffb640 1 -- 172.21.15.31:0/3055020712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f26ec009de0 con 0x7f26fc102ae0 2024-09-21T02:44:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f26e80b6550 msgr2=0x7f26e80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f26e80b6550 0x7f26e80b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009230 tx=0x7f26e4031400 comp rx=0 tx=0).stop 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 msgr2=0x7f26fc19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 0x7f26fc19a580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00d950 tx=0x7f26ec00de10 comp rx=0 tx=0).stop 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 shutdown_connections 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f26e80b6550 0x7f26e80b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26fc108d20 0x7f26fc19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26fc1083d0 0x7f26fc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 --2- 172.21.15.31:0/3055020712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26fc102ae0 0x7f26fc19a580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 >> 172.21.15.31:0/3055020712 conn(0x7f26fc0797b0 msgr2=0x7f26fc101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 shutdown_connections 2024-09-21T02:44:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:41.721+0000 7f2702665640 1 -- 172.21.15.31:0/3055020712 wait complete. 2024-09-21T02:44:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 -- 172.21.15.31:0/4153842400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 msgr2=0x7f30f8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/4153842400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f8102ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009900 tx=0x7f30f002a410 comp rx=0 tx=0).stop 2024-09-21T02:44:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 -- 172.21.15.31:0/4153842400 shutdown_connections 2024-09-21T02:44:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/4153842400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f81083d0 0x7f30f81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/4153842400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f8108d20 0x7f30f810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/4153842400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f8102ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 -- 172.21.15.31:0/4153842400 >> 172.21.15.31:0/4153842400 conn(0x7f30f8079790 msgr2=0x7f30f8079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 -- 172.21.15.31:0/4153842400 shutdown_connections 2024-09-21T02:44:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.518+0000 7f30fcb3b640 1 -- 172.21.15.31:0/4153842400 wait complete. 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 Processor -- start 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 -- start start 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f81083d0 0x7f30f819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f8108d20 0x7f30f819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30f81129e0 con 0x7f30f8102ae0 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8112860 con 0x7f30f81083d0 2024-09-21T02:44:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30fcb3b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8112b60 con 0x7f30f8108d20 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58622/0 (socket says 172.21.15.31:58622) 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 -- 172.21.15.31:0/2066879655 learned_addr learned my addr 172.21.15.31:0/2066879655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f6d9d640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f8108d20 0x7f30f819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 -- 172.21.15.31:0/2066879655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f8108d20 msgr2=0x7f30f819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f8108d20 0x7f30f819fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f5d9b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f81083d0 0x7f30f819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 -- 172.21.15.31:0/2066879655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f81083d0 msgr2=0x7f30f819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f81083d0 0x7f30f819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f659c640 1 -- 172.21.15.31:0/2066879655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f81a02b0 con 0x7f30f8102ae0 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.519+0000 7f30f5d9b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f81083d0 0x7f30f819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.520+0000 7f30f659c640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f819a400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30f0006010 tx=0x7f30f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:42.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.520+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f003f030 con 0x7f30f8102ae0 2024-09-21T02:44:42.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.520+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f0003800 con 0x7f30f8102ae0 2024-09-21T02:44:42.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.520+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0035070 con 0x7f30f8102ae0 2024-09-21T02:44:42.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.520+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f81a0540 con 0x7f30f8102ae0 2024-09-21T02:44:42.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.520+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81a0a70 con 0x7f30f8102ae0 2024-09-21T02:44:42.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.521+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f8102ef0 con 0x7f30f8102ae0 2024-09-21T02:44:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.522+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f30f0003cb0 con 0x7f30f8102ae0 2024-09-21T02:44:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.523+0000 7f30ef7fe640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30e40b6580 0x7f30e40b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.524+0000 7f30f5d9b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30e40b6580 0x7f30e40b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.524+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f30f01063f0 con 0x7f30f8102ae0 2024-09-21T02:44:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.524+0000 7f30f5d9b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30e40b6580 0x7f30e40b8a90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30f8079d90 tx=0x7f30e0005d90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.527+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30f81a0a70 con 0x7f30f8102ae0 2024-09-21T02:44:42.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.715+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f30f819b530 con 0x7f30f8102ae0 2024-09-21T02:44:42.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.719+0000 7f30ef7fe640 1 -- 172.21.15.31:0/2066879655 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v33)=0 v33) ==== 120+0+0 (secure 0 0 0) 0x7f30f00ce7e0 con 0x7f30f8102ae0 2024-09-21T02:44:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30e40b6580 msgr2=0x7f30e40b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30e40b6580 0x7f30e40b8a90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f30f8079d90 tx=0x7f30e0005d90 comp rx=0 tx=0).stop 2024-09-21T02:44:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 msgr2=0x7f30f819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f819a400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30f0006010 tx=0x7f30f00042f0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 shutdown_connections 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30e40b6580 0x7f30e40b8a90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30f8108d20 0x7f30f819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30f81083d0 0x7f30f819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 --2- 172.21.15.31:0/2066879655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30f8102ae0 0x7f30f819a400 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 >> 172.21.15.31:0/2066879655 conn(0x7f30f8079790 msgr2=0x7f30f8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 shutdown_connections 2024-09-21T02:44:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:42.722+0000 7f30fcb3b640 1 -- 172.21.15.31:0/2066879655 wait complete. 2024-09-21T02:44:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:42 smithi178 ceph-mon[34759]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:42 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3055020712' entity='client.admin' 2024-09-21T02:44:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:42 smithi031 ceph-mon[31209]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:42 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3055020712' entity='client.admin' 2024-09-21T02:44:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:42 smithi165 ceph-mon[34818]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:42 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3055020712' entity='client.admin' 2024-09-21T02:44:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 -- 172.21.15.31:0/3022549721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf0104840 msgr2=0x7fdaf0104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 --2- 172.21.15.31:0/3022549721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf0104840 0x7fdaf0104c50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009ca0 tx=0x7fdae002a8e0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 -- 172.21.15.31:0/3022549721 shutdown_connections 2024-09-21T02:44:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 --2- 172.21.15.31:0/3022549721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0105270 0x7fdaf010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 --2- 172.21.15.31:0/3022549721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdaf010ddf0 0x7fdaf0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 --2- 172.21.15.31:0/3022549721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf0104840 0x7fdaf0104c50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 -- 172.21.15.31:0/3022549721 >> 172.21.15.31:0/3022549721 conn(0x7fdaf0079730 msgr2=0x7fdaf0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 -- 172.21.15.31:0/3022549721 shutdown_connections 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 -- 172.21.15.31:0/3022549721 wait complete. 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.268+0000 7fdaf5511640 1 Processor -- start 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 -- start start 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0104840 0x7fdaf019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdaf0105270 0x7fdaf019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 0x7fdaf019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0113ad0 con 0x7fdaf010ddf0 2024-09-21T02:44:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0113950 con 0x7fdaf0105270 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaf5511640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0113c50 con 0x7fdaf0104840 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 0x7fdaf019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 0x7fdaf019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58634/0 (socket says 172.21.15.31:58634) 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaef7fe640 1 -- 172.21.15.31:0/972021871 learned_addr learned my addr 172.21.15.31:0/972021871 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaee7fc640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdaf0105270 0x7fdaf019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.269+0000 7fdaeeffd640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0104840 0x7fdaf019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaef7fe640 1 -- 172.21.15.31:0/972021871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0104840 msgr2=0x7fdaf019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaef7fe640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0104840 0x7fdaf019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaef7fe640 1 -- 172.21.15.31:0/972021871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdaf0105270 msgr2=0x7fdaf019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:43.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaef7fe640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdaf0105270 0x7fdaf019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaef7fe640 1 -- 172.21.15.31:0/972021871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf01a0470 con 0x7fdaf010ddf0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaeeffd640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0104840 0x7fdaf019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaef7fe640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 0x7fdaf019fd40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d750 tx=0x7fdae800db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8017070 con 0x7fdaf010ddf0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae80097c0 con 0x7fdaf010ddf0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8004ba0 con 0x7fdaf010ddf0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf01a0760 con 0x7fdaf010ddf0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.270+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01a6ec0 con 0x7fdaf010ddf0 2024-09-21T02:44:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.271+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf0104c50 con 0x7fdaf010ddf0 2024-09-21T02:44:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.273+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fdae8009c90 con 0x7fdaf010ddf0 2024-09-21T02:44:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.273+0000 7fdae7fff640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdadc0b64d0 0x7fdadc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.274+0000 7fdaeeffd640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdadc0b64d0 0x7fdadc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.274+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdae80e5570 con 0x7fdaf010ddf0 2024-09-21T02:44:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.274+0000 7fdaeeffd640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdadc0b64d0 0x7fdadc0b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdae00094e0 tx=0x7fdae00093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:43.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.277+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdae80ada00 con 0x7fdaf010ddf0 2024-09-21T02:44:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.470+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fdaf01a71a0 con 0x7fdaf010ddf0 2024-09-21T02:44:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.474+0000 7fdae7fff640 1 -- 172.21.15.31:0/972021871 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7fdae80b1de0 con 0x7fdaf010ddf0 2024-09-21T02:44:43.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdadc0b64d0 msgr2=0x7fdadc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:43.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdadc0b64d0 0x7fdadc0b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdae00094e0 tx=0x7fdae00093f0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 msgr2=0x7fdaf019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 0x7fdaf019fd40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d750 tx=0x7fdae800db50 comp rx=0 tx=0).stop 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 shutdown_connections 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdadc0b64d0 0x7fdadc0b89e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf010ddf0 0x7fdaf019fd40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdaf0105270 0x7fdaf019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 --2- 172.21.15.31:0/972021871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf0104840 0x7fdaf019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 >> 172.21.15.31:0/972021871 conn(0x7fdaf0079730 msgr2=0x7fdaf0107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 shutdown_connections 2024-09-21T02:44:43.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:43.476+0000 7fdaf5511640 1 -- 172.21.15.31:0/972021871 wait complete. 2024-09-21T02:44:44.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:43 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2066879655' entity='client.admin' 2024-09-21T02:44:44.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:43 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/972021871' entity='client.admin' 2024-09-21T02:44:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:43 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2066879655' entity='client.admin' 2024-09-21T02:44:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:43 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/972021871' entity='client.admin' 2024-09-21T02:44:44.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:43 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2066879655' entity='client.admin' 2024-09-21T02:44:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:43 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/972021871' entity='client.admin' 2024-09-21T02:44:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 -- 172.21.15.31:0/1801343544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b81083a0 msgr2=0x7f83b81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 --2- 172.21.15.31:0/1801343544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b81083a0 0x7f83b81087b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009900 tx=0x7f83ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:44:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 -- 172.21.15.31:0/1801343544 shutdown_connections 2024-09-21T02:44:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 --2- 172.21.15.31:0/1801343544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b8108cf0 0x7f83b810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 --2- 172.21.15.31:0/1801343544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b81083a0 0x7f83b81087b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 --2- 172.21.15.31:0/1801343544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83b8102a60 0x7f83b8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 -- 172.21.15.31:0/1801343544 >> 172.21.15.31:0/1801343544 conn(0x7f83b8079730 msgr2=0x7f83b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 -- 172.21.15.31:0/1801343544 shutdown_connections 2024-09-21T02:44:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.275+0000 7f83bf261640 1 -- 172.21.15.31:0/1801343544 wait complete. 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 Processor -- start 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 -- start start 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83b8102a60 0x7f83b819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b81083a0 0x7f83b819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 0x7f83b819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8113ac0 con 0x7f83b8108cf0 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8113940 con 0x7f83b8102a60 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bf261640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8113c40 con 0x7f83b81083a0 2024-09-21T02:44:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bd7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 0x7f83b819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b81083a0 0x7f83b819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bd7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 0x7f83b819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58654/0 (socket says 172.21.15.31:58654) 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bd7fe640 1 -- 172.21.15.31:0/3610450553 learned_addr learned my addr 172.21.15.31:0/3610450553 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.276+0000 7f83bcffd640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83b8102a60 0x7f83b819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b7fff640 1 -- 172.21.15.31:0/3610450553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83b8102a60 msgr2=0x7f83b819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b7fff640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83b8102a60 0x7f83b819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b7fff640 1 -- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 msgr2=0x7f83b819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b7fff640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 0x7f83b819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b7fff640 1 -- 172.21.15.31:0/3610450553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b81a0500 con 0x7f83b81083a0 2024-09-21T02:44:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83bd7fe640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 0x7f83b819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b7fff640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b81083a0 0x7f83b819ab30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83ac02a930 tx=0x7f83ac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac03f070 con 0x7f83b81083a0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ac003ca0 con 0x7f83b81083a0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b81a0790 con 0x7f83b81083a0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac035070 con 0x7f83b81083a0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.277+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b81a6f50 con 0x7f83b81083a0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.278+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8103ce0 con 0x7f83b81083a0 2024-09-21T02:44:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.280+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f83ac03b460 con 0x7f83b81083a0 2024-09-21T02:44:44.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.281+0000 7f83b5ffb640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f83a00b64d0 0x7f83a00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:44.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.281+0000 7f83bcffd640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f83a00b64d0 0x7f83a00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:44.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.281+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f83ac105fb0 con 0x7f83b81083a0 2024-09-21T02:44:44.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.281+0000 7f83bcffd640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f83a00b64d0 0x7f83a00b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009990 tx=0x7f83a8009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:44.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.284+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83ac137710 con 0x7f83b81083a0 2024-09-21T02:44:44.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.491+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f83b8102e70 con 0x7f83b81083a0 2024-09-21T02:44:44.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.499+0000 7f83b5ffb640 1 -- 172.21.15.31:0/3610450553 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v35)=0 v35) ==== 129+0+0 (secure 0 0 0) 0x7f83b8102e70 con 0x7f83b81083a0 2024-09-21T02:44:44.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f83a00b64d0 msgr2=0x7f83a00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f83a00b64d0 0x7f83a00b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009990 tx=0x7f83a8009110 comp rx=0 tx=0).stop 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b81083a0 msgr2=0x7f83b819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b81083a0 0x7f83b819ab30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83ac02a930 tx=0x7f83ac004270 comp rx=0 tx=0).stop 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 shutdown_connections 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f83a00b64d0 0x7f83a00b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b81083a0 0x7f83b819ab30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83b8108cf0 0x7f83b819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 --2- 172.21.15.31:0/3610450553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83b8102a60 0x7f83b819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 >> 172.21.15.31:0/3610450553 conn(0x7f83b8079730 msgr2=0x7f83b8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 shutdown_connections 2024-09-21T02:44:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:44.502+0000 7f83bf261640 1 -- 172.21.15.31:0/3610450553 wait complete. 2024-09-21T02:44:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:44.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:44 smithi178 ceph-mon[34759]: from='client.? ' entity='client.admin' 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:44 smithi031 ceph-mon[31209]: from='client.? ' entity='client.admin' 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:44 smithi165 ceph-mon[34818]: from='client.? ' entity='client.admin' 2024-09-21T02:44:45.818 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T02:44:45.818 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T02:44:45.818 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi031') 2024-09-21T02:44:45.818 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi031') 2024-09-21T02:44:45.818 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi031') 2024-09-21T02:44:45.819 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi031') 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: Reconfiguring mon.c (monmap changed)... 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: Reconfiguring daemon mon.c on smithi178 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:45.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: Reconfiguring mon.c (monmap changed)... 2024-09-21T02:44:46.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: Reconfiguring daemon mon.c on smithi178 2024-09-21T02:44:46.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:46.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:46.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:46.014 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: Reconfiguring mon.c (monmap changed)... 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: Reconfiguring daemon mon.c on smithi178 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:46.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 -- 172.21.15.31:0/748138795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c01083c0 msgr2=0x7f49c01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 --2- 172.21.15.31:0/748138795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c01083c0 0x7f49c01087d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009900 tx=0x7f49b002a410 comp rx=0 tx=0).stop 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 -- 172.21.15.31:0/748138795 shutdown_connections 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 --2- 172.21.15.31:0/748138795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c0108d10 0x7f49c010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 --2- 172.21.15.31:0/748138795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c01083c0 0x7f49c01087d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 --2- 172.21.15.31:0/748138795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49c0102a80 0x7f49c0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 -- 172.21.15.31:0/748138795 >> 172.21.15.31:0/748138795 conn(0x7f49c0079740 msgr2=0x7f49c0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 -- 172.21.15.31:0/748138795 shutdown_connections 2024-09-21T02:44:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.236+0000 7f49c73e0640 1 -- 172.21.15.31:0/748138795 wait complete. 2024-09-21T02:44:46.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 Processor -- start 2024-09-21T02:44:46.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 -- start start 2024-09-21T02:44:46.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 0x7f49c019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:46.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c01083c0 0x7f49c019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:46.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49c0108d10 0x7f49c019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0112a40 con 0x7f49c0102a80 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49c01128c0 con 0x7f49c0108d10 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c73e0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0112bc0 con 0x7f49c01083c0 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c517c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 0x7f49c019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c517c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 0x7f49c019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58668/0 (socket says 172.21.15.31:58668) 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.237+0000 7f49c517c640 1 -- 172.21.15.31:0/3900723180 learned_addr learned my addr 172.21.15.31:0/3900723180 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c517c640 1 -- 172.21.15.31:0/3900723180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c01083c0 msgr2=0x7f49c019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c497b640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c01083c0 0x7f49c019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c517c640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c01083c0 0x7f49c019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c517c640 1 -- 172.21.15.31:0/3900723180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49c0108d10 msgr2=0x7f49c019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c597d640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49c0108d10 0x7f49c019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c517c640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49c0108d10 0x7f49c019fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c517c640 1 -- 172.21.15.31:0/3900723180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c01a0610 con 0x7f49c0102a80 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c517c640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 0x7f49c019a6a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b4a0 tx=0x7f49bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc014070 con 0x7f49c0102a80 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c01a0900 con 0x7f49c0102a80 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49bc010ac0 con 0x7f49c0102a80 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc017c30 con 0x7f49c0102a80 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.238+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a7060 con 0x7f49c0102a80 2024-09-21T02:44:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.239+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c0103d00 con 0x7f49c0102a80 2024-09-21T02:44:46.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.241+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f49bc017070 con 0x7f49c0102a80 2024-09-21T02:44:46.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.242+0000 7f49b67fc640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49ac0b6550 0x7f49ac0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:46.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.242+0000 7f49c497b640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49ac0b6550 0x7f49ac0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:46.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.242+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f49bc0e4de0 con 0x7f49c0102a80 2024-09-21T02:44:46.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.242+0000 7f49c497b640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49ac0b6550 0x7f49ac0b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49b002a930 tx=0x7f49b0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.245+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49bc0ad1d0 con 0x7f49c0102a80 2024-09-21T02:44:46.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.444+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f49c019b7d0 con 0x7f49c0102a80 2024-09-21T02:44:46.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.444+0000 7f49b67fc640 1 -- 172.21.15.31:0/3900723180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v35)=0 v35) ==== 108+0+0 (secure 0 0 0) 0x7f49c019b7d0 con 0x7f49c0102a80 2024-09-21T02:44:46.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49ac0b6550 msgr2=0x7f49ac0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:46.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49ac0b6550 0x7f49ac0b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49b002a930 tx=0x7f49b0000bc0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 msgr2=0x7f49c019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 0x7f49c019a6a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b4a0 tx=0x7f49bc00b960 comp rx=0 tx=0).stop 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 shutdown_connections 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49ac0b6550 0x7f49ac0b8a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c01083c0 0x7f49c019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49c0108d10 0x7f49c019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 --2- 172.21.15.31:0/3900723180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49c0102a80 0x7f49c019a6a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 >> 172.21.15.31:0/3900723180 conn(0x7f49c0079740 msgr2=0x7f49c0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 shutdown_connections 2024-09-21T02:44:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:46.446+0000 7f49c73e0640 1 -- 172.21.15.31:0/3900723180 wait complete. 2024-09-21T02:44:47.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:46 smithi031 ceph-mon[31209]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 -- 172.21.15.31:0/1721087570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 msgr2=0x7f6224104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 --2- 172.21.15.31:0/1721087570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f6224104c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6218009900 tx=0x7f621802a410 comp rx=0 tx=0).stop 2024-09-21T02:44:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 -- 172.21.15.31:0/1721087570 shutdown_connections 2024-09-21T02:44:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 --2- 172.21.15.31:0/1721087570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62241052b0 0x7f622410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 --2- 172.21.15.31:0/1721087570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f622410de30 0x7f6224110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 --2- 172.21.15.31:0/1721087570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f6224104c90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 -- 172.21.15.31:0/1721087570 >> 172.21.15.31:0/1721087570 conn(0x7f6224079730 msgr2=0x7f6224079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 -- 172.21.15.31:0/1721087570 shutdown_connections 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.150+0000 7f622ae2f640 1 -- 172.21.15.31:0/1721087570 wait complete. 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 Processor -- start 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 -- start start 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f622419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62241052b0 0x7f622419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f622410de30 0x7f622419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113a70 con 0x7f6224104880 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62241138f0 con 0x7f622410de30 2024-09-21T02:44:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f622ae2f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113bf0 con 0x7f62241052b0 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f6228bcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f622419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f6228bcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f622419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58678/0 (socket says 172.21.15.31:58678) 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f6228bcb640 1 -- 172.21.15.31:0/3412138567 learned_addr learned my addr 172.21.15.31:0/3412138567 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.151+0000 7f62293cc640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f622410de30 0x7f622419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6228bcb640 1 -- 172.21.15.31:0/3412138567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62241052b0 msgr2=0x7f622419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6228bcb640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62241052b0 0x7f622419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6228bcb640 1 -- 172.21.15.31:0/3412138567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f622410de30 msgr2=0x7f622419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6228bcb640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f622410de30 0x7f622419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6228bcb640 1 -- 172.21.15.31:0/3412138567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62241a0450 con 0x7f6224104880 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6228bcb640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f622419a4e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6218032fd0 tx=0x7f6218004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621803f030 con 0x7f6224104880 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241a06e0 con 0x7f6224104880 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6218004490 con 0x7f6224104880 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218035070 con 0x7f6224104880 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.152+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a6ea0 con 0x7f6224104880 2024-09-21T02:44:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.153+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224104c90 con 0x7f6224104880 2024-09-21T02:44:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.155+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f6218042030 con 0x7f6224104880 2024-09-21T02:44:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.156+0000 7f6221ffb640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62140b64d0 0x7f62140b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.156+0000 7f6223fff640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62140b64d0 0x7f62140b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.156+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f62181060d0 con 0x7f6224104880 2024-09-21T02:44:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.156+0000 7f6223fff640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62140b64d0 0x7f62140b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6210005fd0 tx=0x7f6210005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:47.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.159+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62180ce560 con 0x7f6224104880 2024-09-21T02:44:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:46 smithi165 ceph-mon[34818]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:46 smithi178 ceph-mon[34759]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.355+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f622419b610 con 0x7f6224104880 2024-09-21T02:44:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.356+0000 7f6221ffb640 1 -- 172.21.15.31:0/3412138567 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v35)=0 v35) ==== 106+0+0 (secure 0 0 0) 0x7f62180d2940 con 0x7f6224104880 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62140b64d0 msgr2=0x7f62140b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62140b64d0 0x7f62140b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6210005fd0 tx=0x7f6210005850 comp rx=0 tx=0).stop 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 msgr2=0x7f622419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f622419a4e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6218032fd0 tx=0x7f6218004270 comp rx=0 tx=0).stop 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 shutdown_connections 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62140b64d0 0x7f62140b89e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62241052b0 0x7f622419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f622410de30 0x7f622419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 --2- 172.21.15.31:0/3412138567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6224104880 0x7f622419a4e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 >> 172.21.15.31:0/3412138567 conn(0x7f6224079730 msgr2=0x7f62241008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.358+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 shutdown_connections 2024-09-21T02:44:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:47.359+0000 7f622ae2f640 1 -- 172.21.15.31:0/3412138567 wait complete. 2024-09-21T02:44:47.720 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi031') 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 -- 172.21.15.31:0/4179450255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 msgr2=0x7f0f4c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 --2- 172.21.15.31:0/4179450255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c1087d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02a410 comp rx=0 tx=0).stop 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 -- 172.21.15.31:0/4179450255 shutdown_connections 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 --2- 172.21.15.31:0/4179450255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c108d10 0x7f0f4c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 --2- 172.21.15.31:0/4179450255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c1087d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 --2- 172.21.15.31:0/4179450255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f4c102a80 0x7f0f4c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 -- 172.21.15.31:0/4179450255 >> 172.21.15.31:0/4179450255 conn(0x7f0f4c079740 msgr2=0x7f0f4c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:48.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 -- 172.21.15.31:0/4179450255 shutdown_connections 2024-09-21T02:44:48.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.058+0000 7f0f544b1640 1 -- 172.21.15.31:0/4179450255 wait complete. 2024-09-21T02:44:48.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.059+0000 7f0f544b1640 1 Processor -- start 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.059+0000 7f0f544b1640 1 -- start start 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f544b1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c102a80 0x7f0f4c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f544b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f544b1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f4c108d10 0x7f0f4c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f544b1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c113aa0 con 0x7f0f4c1083c0 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f544b1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c113920 con 0x7f0f4c102a80 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f544b1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c113c20 con 0x7f0f4c108d10 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58694/0 (socket says 172.21.15.31:58694) 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 -- 172.21.15.31:0/939961386 learned_addr learned my addr 172.21.15.31:0/939961386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:48.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f5224d640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c102a80 0x7f0f4c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f52a4e640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f4c108d10 0x7f0f4c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 -- 172.21.15.31:0/939961386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f4c108d10 msgr2=0x7f0f4c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f4c108d10 0x7f0f4c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 -- 172.21.15.31:0/939961386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c102a80 msgr2=0x7f0f4c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c102a80 0x7f0f4c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 -- 172.21.15.31:0/939961386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1a04c0 con 0x7f0f4c1083c0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.060+0000 7f0f51a4c640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c19aaf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c032fd0 tx=0x7f0f3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.061+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c03f030 con 0x7f0f4c1083c0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.061+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c004490 con 0x7f0f4c1083c0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.061+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c035070 con 0x7f0f4c1083c0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.061+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c1a0750 con 0x7f0f4c1083c0 2024-09-21T02:44:48.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.061+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1a6f10 con 0x7f0f4c1083c0 2024-09-21T02:44:48.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.062+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4c103d00 con 0x7f0f4c1083c0 2024-09-21T02:44:48.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.064+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f0f3c042030 con 0x7f0f4c1083c0 2024-09-21T02:44:48.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.065+0000 7f0f437fe640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f380b6550 0x7f0f380b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.065+0000 7f0f5224d640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f380b6550 0x7f0f380b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.065+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0f3c105f60 con 0x7f0f4c1083c0 2024-09-21T02:44:48.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.065+0000 7f0f5224d640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f380b6550 0x7f0f380b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bbe0 tx=0x7f0f4800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:48.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.068+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f3c0d27d0 con 0x7f0f4c1083c0 2024-09-21T02:44:48.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.260+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f0f4c102e90 con 0x7f0f4c1083c0 2024-09-21T02:44:48.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.264+0000 7f0f437fe640 1 -- 172.21.15.31:0/939961386 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v36)=0 v36) ==== 125+0+0 (secure 0 0 0) 0x7f0f3c0d7710 con 0x7f0f4c1083c0 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.266+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f380b6550 msgr2=0x7f0f380b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.266+0000 7f0f544b1640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f380b6550 0x7f0f380b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bbe0 tx=0x7f0f4800b390 comp rx=0 tx=0).stop 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.266+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 msgr2=0x7f0f4c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.266+0000 7f0f544b1640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c19aaf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c032fd0 tx=0x7f0f3c004270 comp rx=0 tx=0).stop 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 shutdown_connections 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f380b6550 0x7f0f380b8a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f4c108d10 0x7f0f4c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4c1083c0 0x7f0f4c19aaf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 --2- 172.21.15.31:0/939961386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c102a80 0x7f0f4c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 >> 172.21.15.31:0/939961386 conn(0x7f0f4c079740 msgr2=0x7f0f4c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:48.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 shutdown_connections 2024-09-21T02:44:48.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.267+0000 7f0f544b1640 1 -- 172.21.15.31:0/939961386 wait complete. 2024-09-21T02:44:48.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:48 smithi031 ceph-mon[31209]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:48.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/939961386' entity='client.admin' 2024-09-21T02:44:48.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:48.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:48.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:48.823 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 -- 172.21.15.31:0/874308692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48110200 msgr2=0x7fcb48112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/874308692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48110200 0x7fcb48112620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009ca0 tx=0x7fcb4002a950 comp rx=0 tx=0).stop 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 -- 172.21.15.31:0/874308692 shutdown_connections 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/874308692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48110200 0x7fcb48112620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/874308692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb48102fa0 0x7fcb4810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/874308692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102650 0x7fcb48102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 -- 172.21.15.31:0/874308692 >> 172.21.15.31:0/874308692 conn(0x7fcb48079760 msgr2=0x7fcb48079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 -- 172.21.15.31:0/874308692 shutdown_connections 2024-09-21T02:44:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.920+0000 7fcb4fa93640 1 -- 172.21.15.31:0/874308692 wait complete. 2024-09-21T02:44:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 Processor -- start 2024-09-21T02:44:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 -- start start 2024-09-21T02:44:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 0x7fcb4819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102fa0 0x7fcb4819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb48110200 0x7fcb481a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48115ca0 con 0x7fcb48102650 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48115b20 con 0x7fcb48110200 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4fa93640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48115e20 con 0x7fcb48102fa0 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d82f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 0x7fcb4819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d82f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 0x7fcb4819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58712/0 (socket says 172.21.15.31:58712) 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d82f640 1 -- 172.21.15.31:0/822370571 learned_addr learned my addr 172.21.15.31:0/822370571 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4e030640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb48110200 0x7fcb481a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d02e640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102fa0 0x7fcb4819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d82f640 1 -- 172.21.15.31:0/822370571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102fa0 msgr2=0x7fcb4819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d82f640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102fa0 0x7fcb4819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.921+0000 7fcb4d82f640 1 -- 172.21.15.31:0/822370571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb48110200 msgr2=0x7fcb481a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb4d82f640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb48110200 0x7fcb481a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb4d82f640 1 -- 172.21.15.31:0/822370571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb481a2560 con 0x7fcb48102650 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb4d02e640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102fa0 0x7fcb4819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb4d82f640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 0x7fcb4819c5f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d950 tx=0x7fcb4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb48102650 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb44009c30 con 0x7fcb48102650 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb481a2850 con 0x7fcb48102650 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400f620 con 0x7fcb48102650 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.922+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481a8fb0 con 0x7fcb48102650 2024-09-21T02:44:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.923+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4806c1f0 con 0x7fcb48102650 2024-09-21T02:44:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.925+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fcb44005030 con 0x7fcb48102650 2024-09-21T02:44:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.925+0000 7fcb3effd640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb340b6550 0x7fcb340b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.926+0000 7fcb4d02e640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb340b6550 0x7fcb340b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.926+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcb440e52b0 con 0x7fcb48102650 2024-09-21T02:44:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.926+0000 7fcb4d02e640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb340b6550 0x7fcb340b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb380058c0 tx=0x7fcb38005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:48.929+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb440ad7c0 con 0x7fcb48102650 2024-09-21T02:44:49.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.121+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fcb4819d720 con 0x7fcb48102650 2024-09-21T02:44:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.126+0000 7fcb3effd640 1 -- 172.21.15.31:0/822370571 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v37)=0 v37) ==== 121+0+0 (secure 0 0 0) 0x7fcb44009de0 con 0x7fcb48102650 2024-09-21T02:44:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb340b6550 msgr2=0x7fcb340b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb340b6550 0x7fcb340b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb380058c0 tx=0x7fcb38005850 comp rx=0 tx=0).stop 2024-09-21T02:44:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 msgr2=0x7fcb4819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 0x7fcb4819c5f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d950 tx=0x7fcb4400de10 comp rx=0 tx=0).stop 2024-09-21T02:44:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 shutdown_connections 2024-09-21T02:44:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb340b6550 0x7fcb340b8a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb48102fa0 0x7fcb4819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb48110200 0x7fcb481a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 --2- 172.21.15.31:0/822370571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb48102650 0x7fcb4819c5f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 >> 172.21.15.31:0/822370571 conn(0x7fcb48079760 msgr2=0x7fcb48109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 shutdown_connections 2024-09-21T02:44:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:49.128+0000 7fcb4fa93640 1 -- 172.21.15.31:0/822370571 wait complete. 2024-09-21T02:44:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:48 smithi165 ceph-mon[34818]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/939961386' entity='client.admin' 2024-09-21T02:44:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:49.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:48 smithi178 ceph-mon[34759]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/939961386' entity='client.admin' 2024-09-21T02:44:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:44:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:44:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:44:49.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi031') 2024-09-21T02:44:49.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi031') 2024-09-21T02:44:49.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi031') 2024-09-21T02:44:49.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi031') 2024-09-21T02:44:49.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi031') 2024-09-21T02:44:50.315 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:50 smithi031 ceph-mon[31209]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:50.315 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:50 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/822370571' entity='client.admin' 2024-09-21T02:44:50.350 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fdd37b14-77c2-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-21T02:44:50.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T02:44:50.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:50 smithi165 ceph-mon[34818]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:50.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:50 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/822370571' entity='client.admin' 2024-09-21T02:44:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:50 smithi178 ceph-mon[34759]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/822370571' entity='client.admin' 2024-09-21T02:44:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:51.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 -- 172.21.15.31:0/2857369633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c102a60 msgr2=0x7faa0c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:51.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 --2- 172.21.15.31:0/2857369633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c102a60 0x7faa0c102e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009900 tx=0x7fa9fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:44:51.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 -- 172.21.15.31:0/2857369633 shutdown_connections 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 --2- 172.21.15.31:0/2857369633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa0c1083a0 0x7faa0c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 --2- 172.21.15.31:0/2857369633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa0c108cf0 0x7faa0c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 --2- 172.21.15.31:0/2857369633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c102a60 0x7faa0c102e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 -- 172.21.15.31:0/2857369633 >> 172.21.15.31:0/2857369633 conn(0x7faa0c079730 msgr2=0x7faa0c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 -- 172.21.15.31:0/2857369633 shutdown_connections 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.032+0000 7faa11d59640 1 -- 172.21.15.31:0/2857369633 wait complete. 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.033+0000 7faa11d59640 1 Processor -- start 2024-09-21T02:44:51.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.033+0000 7faa11d59640 1 -- start start 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa11d59640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa0c102a60 0x7faa0c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa11d59640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 0x7faa0c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa11d59640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa0c108cf0 0x7faa0c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa11d59640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c112960 con 0x7faa0c1083a0 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa11d59640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1127e0 con 0x7faa0c102a60 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa11d59640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c112ae0 con 0x7faa0c108cf0 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 0x7faa0c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 0x7faa0c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47888/0 (socket says 172.21.15.31:47888) 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 -- 172.21.15.31:0/3521093475 learned_addr learned my addr 172.21.15.31:0/3521093475 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0b7fe640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa0c102a60 0x7faa0c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:51.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0bfff640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa0c108cf0 0x7faa0c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 -- 172.21.15.31:0/3521093475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa0c108cf0 msgr2=0x7faa0c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa0c108cf0 0x7faa0c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 -- 172.21.15.31:0/3521093475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa0c102a60 msgr2=0x7faa0c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa0c102a60 0x7faa0c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 -- 172.21.15.31:0/3521093475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c1a02b0 con 0x7faa0c1083a0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.034+0000 7faa0affd640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 0x7faa0c19a9a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c970 tx=0x7faa0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.035+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00014070 con 0x7faa0c1083a0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.035+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa00004440 con 0x7faa0c1083a0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.035+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c1a05a0 con 0x7faa0c1083a0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.035+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00003e50 con 0x7faa0c1083a0 2024-09-21T02:44:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.035+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c1a6dc0 con 0x7faa0c1083a0 2024-09-21T02:44:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.036+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c102e70 con 0x7faa0c1083a0 2024-09-21T02:44:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.037+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7faa00017070 con 0x7faa0c1083a0 2024-09-21T02:44:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.038+0000 7faa08ff9640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9f40b66a0 0x7fa9f40b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.038+0000 7faa0b7fe640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9f40b66a0 0x7fa9f40b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.038+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7faa000e5270 con 0x7faa0c1083a0 2024-09-21T02:44:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.038+0000 7faa0b7fe640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9f40b66a0 0x7fa9f40b8bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc031440 tx=0x7fa9fc005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.042+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa0001e030 con 0x7faa0c1083a0 2024-09-21T02:44:51.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.235+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7faa0c1a70a0 con 0x7faa0c1083a0 2024-09-21T02:44:51.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.236+0000 7faa08ff9640 1 -- 172.21.15.31:0/3521093475 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22330 (secure 0 0 0) 0x7faa000ad700 con 0x7faa0c1083a0 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global dev auth_debug true 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global basic log_to_file true 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global basic log_to_journald false 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global basic log_to_stderr false 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_allow_pool_delete true 2024-09-21T02:44:51.238 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global dev ms_die_on_bug true 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global dev ms_die_on_old_message true 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global dev ms_die_on_skipped_message false 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:44:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced debug_mon 20/20 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced debug_ms 1/1 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced debug_paxos 20/20 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T02:44:51.240 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_client 20/20 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:44:51.241 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev bdev_debug_aio true 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T02:44:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_bluefs 20/20 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_bluestore 20/20 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_ms 1/1 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_osd 20/20 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_op_order true 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_shutdown true 2024-09-21T02:44:51.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T02:44:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced debug_mds 20/20 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced debug_ms 2/2 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_split_size 100 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds dev mds_debug_frag true 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:44:51.245 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:mds dev mds_verify_scatter true 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced client_mount_timeout 600 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced debug_client 20/20 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced debug_ms 1/1 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T02:44:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T02:44:51.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9f40b66a0 msgr2=0x7fa9f40b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:51.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9f40b66a0 0x7fa9f40b8bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc031440 tx=0x7fa9fc005e80 comp rx=0 tx=0).stop 2024-09-21T02:44:51.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 msgr2=0x7faa0c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:51.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 0x7faa0c19a9a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c970 tx=0x7faa0000ce30 comp rx=0 tx=0).stop 2024-09-21T02:44:51.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 shutdown_connections 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9f40b66a0 0x7fa9f40b8bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa0c108cf0 0x7faa0c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa0c1083a0 0x7faa0c19a9a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 --2- 172.21.15.31:0/3521093475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa0c102a60 0x7faa0c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 >> 172.21.15.31:0/3521093475 conn(0x7faa0c079730 msgr2=0x7faa0c106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 shutdown_connections 2024-09-21T02:44:51.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:51.240+0000 7faa11d59640 1 -- 172.21.15.31:0/3521093475 wait complete. 2024-09-21T02:44:51.987 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T02:44:51.988 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:44:51.988 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:44:52.006 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:44:52.006 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 674 Links: 1 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 02:43:28.708509546 +0000 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 02:39:02.688800180 +0000 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 02:39:02.688800180 +0000 2024-09-21T02:44:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 02:39:02.688800180 +0000 2024-09-21T02:44:52.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:44:52.130 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T02:44:52.130 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T02:44:52.130 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00019448 s, 2.6 MB/s 2024-09-21T02:44:52.131 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:44:52.189 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-21T02:44:52.245 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:44:52.245 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:52.245 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 692 Links: 1 2024-09-21T02:44:52.246 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:52.246 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:52.246 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 02:43:28.741508981 +0000 2024-09-21T02:44:52.246 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 02:39:03.035794142 +0000 2024-09-21T02:44:52.246 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 02:39:03.035794142 +0000 2024-09-21T02:44:52.246 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 02:39:03.035794142 +0000 2024-09-21T02:44:52.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:44:52.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:52 smithi031 ceph-mon[31209]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:52.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:52 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3521093475' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T02:44:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T02:44:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T02:44:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00020436 s, 2.5 MB/s 2024-09-21T02:44:52.312 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:44:52.370 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-21T02:44:52.426 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:44:52.426 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:52.426 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 711 Links: 1 2024-09-21T02:44:52.427 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:52.427 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:52.427 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 02:43:28.773508433 +0000 2024-09-21T02:44:52.427 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 02:39:03.365788401 +0000 2024-09-21T02:44:52.427 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 02:39:03.365788401 +0000 2024-09-21T02:44:52.427 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 02:39:03.365788401 +0000 2024-09-21T02:44:52.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:44:52.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:52 smithi165 ceph-mon[34818]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:52.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:52 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3521093475' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T02:44:52.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:52 smithi178 ceph-mon[34759]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:52.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:52 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3521093475' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T02:44:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T02:44:52.491 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T02:44:52.492 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000225687 s, 2.3 MB/s 2024-09-21T02:44:52.493 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:44:52.549 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-21T02:44:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 731 Links: 1 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 02:43:28.805507886 +0000 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 02:39:03.699782589 +0000 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 02:39:03.699782589 +0000 2024-09-21T02:44:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 02:39:03.699782589 +0000 2024-09-21T02:44:52.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:44:52.670 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T02:44:52.670 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T02:44:52.670 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000196212 s, 2.6 MB/s 2024-09-21T02:44:52.671 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:44:52.728 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-21T02:44:52.729 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:44:52.749 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:44:52.749 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-21 02:44:23.936901623 +0000 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-21 02:39:11.780266872 +0000 2024-09-21T02:44:52.808 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-21 02:39:11.780266872 +0000 2024-09-21T02:44:52.809 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-21 02:39:11.780266872 +0000 2024-09-21T02:44:52.809 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:44:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-21T02:44:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-21T02:44:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000265448 s, 1.9 MB/s 2024-09-21T02:44:52.880 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:44:52.939 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-09-21T02:44:52.996 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:44:52.996 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:52.996 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T02:44:52.996 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:52.997 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:52.997 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-21 02:44:23.937901608 +0000 2024-09-21T02:44:52.997 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-21 02:39:12.107261888 +0000 2024-09-21T02:44:52.997 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-21 02:39:12.107261888 +0000 2024-09-21T02:44:52.997 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-21 02:39:12.107261888 +0000 2024-09-21T02:44:52.997 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:44:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-21T02:44:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-21T02:44:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000258366 s, 2.0 MB/s 2024-09-21T02:44:53.064 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:44:53.123 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-21 02:44:23.937901608 +0000 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-21 02:39:12.443256767 +0000 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-21 02:39:12.443256767 +0000 2024-09-21T02:44:53.180 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-21 02:39:12.443256767 +0000 2024-09-21T02:44:53.181 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:44:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-21T02:44:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-21T02:44:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.00023219 s, 2.2 MB/s 2024-09-21T02:44:53.246 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:44:53.303 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-21 02:44:23.937901608 +0000 2024-09-21T02:44:53.359 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-21 02:39:12.760251936 +0000 2024-09-21T02:44:53.360 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-21 02:39:12.760251936 +0000 2024-09-21T02:44:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-21 02:39:12.760251936 +0000 2024-09-21T02:44:53.361 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:44:53.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-21T02:44:53.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-21T02:44:53.423 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000227743 s, 2.2 MB/s 2024-09-21T02:44:53.424 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:44:53.480 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-21T02:44:53.480 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:44:53.500 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:44:53.500 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-21 02:44:21.568410112 +0000 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-21 02:39:10.708263780 +0000 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-21 02:39:10.708263780 +0000 2024-09-21T02:44:53.560 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-21 02:39:10.708263780 +0000 2024-09-21T02:44:53.561 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:44:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-21T02:44:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-21T02:44:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000159878 s, 3.2 MB/s 2024-09-21T02:44:53.631 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:44:53.690 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-21 02:44:21.569410093 +0000 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-21 02:39:11.028257728 +0000 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-21 02:39:11.028257728 +0000 2024-09-21T02:44:53.748 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-21 02:39:11.028257728 +0000 2024-09-21T02:44:53.749 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:44:53.813 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-21T02:44:53.813 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-21T02:44:53.813 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000119825 s, 4.3 MB/s 2024-09-21T02:44:53.814 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:44:53.872 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-09-21T02:44:53.929 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:44:53.929 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:53.929 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T02:44:53.929 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:53.929 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:53.929 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-21 02:44:21.569410093 +0000 2024-09-21T02:44:53.930 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-21 02:39:11.339251846 +0000 2024-09-21T02:44:53.930 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-21 02:39:11.339251846 +0000 2024-09-21T02:44:53.930 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-21 02:39:11.339251846 +0000 2024-09-21T02:44:53.930 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:44:53.993 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-21T02:44:53.994 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-21T02:44:53.994 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000119829 s, 4.3 MB/s 2024-09-21T02:44:53.995 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:44:54.052 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-21 02:44:21.569410093 +0000 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-21 02:39:11.652245926 +0000 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-21 02:39:11.652245926 +0000 2024-09-21T02:44:54.108 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-21 02:39:11.652245926 +0000 2024-09-21T02:44:54.109 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:44:54.172 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-21T02:44:54.172 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-21T02:44:54.172 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000111889 s, 4.6 MB/s 2024-09-21T02:44:54.173 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:44:54.230 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-09-21T02:44:54.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T02:44:54.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:54 smithi165 ceph-mon[34818]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:54.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:54 smithi178 ceph-mon[34759]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:54.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:54 smithi031 ceph-mon[31209]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:56.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:56 smithi165 ceph-mon[34818]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:56.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:56 smithi178 ceph-mon[34759]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:56 smithi031 ceph-mon[31209]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:56.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:44:56.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-09-21T02:44:57.119 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 -- 172.21.15.31:0/2224292561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c01083a0 msgr2=0x7f27c01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 --2- 172.21.15.31:0/2224292561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c01083a0 0x7f27c01087b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009900 tx=0x7f27b402a410 comp rx=0 tx=0).stop 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 -- 172.21.15.31:0/2224292561 shutdown_connections 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 --2- 172.21.15.31:0/2224292561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0108cf0 0x7f27c010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 --2- 172.21.15.31:0/2224292561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c01083a0 0x7f27c01087b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 --2- 172.21.15.31:0/2224292561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c0102a60 0x7f27c0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 -- 172.21.15.31:0/2224292561 >> 172.21.15.31:0/2224292561 conn(0x7f27c0079730 msgr2=0x7f27c0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:44:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 -- 172.21.15.31:0/2224292561 shutdown_connections 2024-09-21T02:44:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.599+0000 7f27c572b640 1 -- 172.21.15.31:0/2224292561 wait complete. 2024-09-21T02:44:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 Processor -- start 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 -- start start 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 0x7f27c019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c01083a0 0x7f27c019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 0x7f27c019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f27c01129b0 con 0x7f27c0108cf0 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0112830 con 0x7f27c0102a60 2024-09-21T02:44:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.600+0000 7f27c572b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0112b30 con 0x7f27c01083a0 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 0x7f27c019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27beffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 0x7f27c019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 0x7f27c019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47904/0 (socket says 172.21.15.31:47904) 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27beffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 0x7f27c019b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:56894/0 (socket says 172.21.15.31:56894) 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 -- 172.21.15.31:0/1978226179 learned_addr learned my addr 172.21.15.31:0/1978226179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27be7fc640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c01083a0 0x7f27c019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 -- 172.21.15.31:0/1978226179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c01083a0 msgr2=0x7f27c019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c01083a0 0x7f27c019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 -- 172.21.15.31:0/1978226179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 msgr2=0x7f27c019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 0x7f27c019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 -- 172.21.15.31:0/1978226179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c01a6530 con 0x7f27c0108cf0 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27beffd640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 0x7f27c019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:44:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27bf7fe640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 0x7f27c019bca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e970 tx=0x7f27b800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b800ccc0 con 0x7f27c0108cf0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b800ce60 con 0x7f27c0108cf0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c01a6820 con 0x7f27c0108cf0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8010430 con 0x7f27c0108cf0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.601+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c01a6d30 con 0x7f27c0108cf0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.602+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c0102e70 con 0x7f27c0108cf0 2024-09-21T02:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.604+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f27b8004fb0 con 0x7f27c0108cf0 2024-09-21T02:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.605+0000 7f27a7fff640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f27a80b6550 0x7f27a80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.605+0000 7f27beffd640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f27a80b6550 0x7f27a80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.605+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f27b80e5c20 con 0x7f27c0108cf0 2024-09-21T02:44:57.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.606+0000 7f27beffd640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f27a80b6550 0x7f27a80b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27c006ba20 tx=0x7f27b0009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:44:57.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.609+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27b8056710 con 0x7f27c0108cf0 2024-09-21T02:44:57.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:44:57.808+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f27c0063800 con 0x7f27a80b6550 2024-09-21T02:44:58.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:58 smithi165 ceph-mon[34818]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:58.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:58 smithi165 ceph-mon[34818]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:44:58.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:44:58.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:44:58.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:44:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:58.474 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:58 smithi031 ceph-mon[31209]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:58.474 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:58 smithi031 ceph-mon[31209]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:44:58.474 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:44:58.475 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:44:58.475 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:44:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:44:58.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:58 smithi178 ceph-mon[34759]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:44:58.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:58 smithi178 ceph-mon[34759]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:44:58.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:44:58.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:44:58.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:44:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:00.429 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:00 smithi031 ceph-mon[31209]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:00.429 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:00 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1588414207' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "23d2c70a-c096-4c33-acfd-b18efa64387a"} : dispatch 2024-09-21T02:45:00.429 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:00 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1588414207' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23d2c70a-c096-4c33-acfd-b18efa64387a"}]': finished 2024-09-21T02:45:00.429 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:00 smithi031 ceph-mon[31209]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T02:45:00.429 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:00.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:00 smithi165 ceph-mon[34818]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:00.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:00 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1588414207' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "23d2c70a-c096-4c33-acfd-b18efa64387a"} : dispatch 2024-09-21T02:45:00.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:00 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1588414207' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23d2c70a-c096-4c33-acfd-b18efa64387a"}]': finished 2024-09-21T02:45:00.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:00 smithi165 ceph-mon[34818]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T02:45:00.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:00.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:00 smithi178 ceph-mon[34759]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:00.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:00 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1588414207' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "23d2c70a-c096-4c33-acfd-b18efa64387a"} : dispatch 2024-09-21T02:45:00.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:00 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1588414207' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23d2c70a-c096-4c33-acfd-b18efa64387a"}]': finished 2024-09-21T02:45:00.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:00 smithi178 ceph-mon[34759]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T02:45:00.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:01.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:01 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2009739171' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:01.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:01 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2009739171' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:01.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:01 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2009739171' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:02.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:02 smithi165 ceph-mon[34818]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:02.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:02 smithi178 ceph-mon[34759]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:02 smithi031 ceph-mon[31209]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:04.402 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:04 smithi031 ceph-mon[31209]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:04.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:04 smithi165 ceph-mon[34818]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:04.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:04 smithi178 ceph-mon[34759]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:06.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:06 smithi031 ceph-mon[31209]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:06 smithi165 ceph-mon[34818]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:06 smithi178 ceph-mon[34759]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:08.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:08 smithi165 ceph-mon[34818]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:08.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:08 smithi178 ceph-mon[34759]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:08.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:08 smithi031 ceph-mon[31209]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:09.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T02:45:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:09.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:09 smithi031 ceph-mon[31209]: Deploying daemon osd.0 on smithi031 2024-09-21T02:45:09.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T02:45:09.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:09.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:09 smithi165 ceph-mon[34818]: Deploying daemon osd.0 on smithi031 2024-09-21T02:45:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T02:45:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:09 smithi178 ceph-mon[34759]: Deploying daemon osd.0 on smithi031 2024-09-21T02:45:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:10 smithi165 ceph-mon[34818]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:10.449 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:10 smithi031 ceph-mon[31209]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:10 smithi178 ceph-mon[34759]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:12.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:12 smithi031 ceph-mon[31209]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:12.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:12 smithi165 ceph-mon[34818]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:12 smithi178 ceph-mon[34759]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:14 smithi165 ceph-mon[34818]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:14.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:14 smithi178 ceph-mon[34759]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:14.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:14 smithi031 ceph-mon[31209]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:15.172 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:15.173 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:15.447 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:16.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:16 smithi031 ceph-mon[31209]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:16.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:16 smithi165 ceph-mon[34818]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:16.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:16 smithi178 ceph-mon[34759]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:18 smithi165 ceph-mon[34818]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:18 smithi178 ceph-mon[34759]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:18 smithi031 ceph-mon[31209]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:18.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:18.823 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:18.823 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:18.823 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.120+0000 7f27a7fff640 1 -- 172.21.15.31:0/1978226179 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f27c0063800 con 0x7f27a80b6550 2024-09-21T02:45:19.122 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.123+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f27a80b6550 msgr2=0x7f27a80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.123+0000 7f27c572b640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f27a80b6550 0x7f27a80b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27c006ba20 tx=0x7f27b0009760 comp rx=0 tx=0).stop 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.123+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 msgr2=0x7f27c019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.123+0000 7f27c572b640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 0x7f27c019bca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e970 tx=0x7f27b800ed70 comp rx=0 tx=0).stop 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 shutdown_connections 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f27a80b6550 0x7f27a80b8a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c01083a0 0x7f27c019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27c0108cf0 0x7f27c019bca0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 --2- 172.21.15.31:0/1978226179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27c0102a60 0x7f27c019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 >> 172.21.15.31:0/1978226179 conn(0x7f27c0079730 msgr2=0x7f27c0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:45:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 shutdown_connections 2024-09-21T02:45:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:45:19.124+0000 7f27c572b640 1 -- 172.21.15.31:0/1978226179 wait complete. 2024-09-21T02:45:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:19.728 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.728 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:19.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:19.761 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T02:45:19.763 INFO:tasks.cephadm:Deploying osd.1 on smithi165 with /dev/vg_nvme/lv_4... 2024-09-21T02:45:19.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T02:45:20.003 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:45:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:20 smithi165 ceph-mon[34818]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:20.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:20 smithi178 ceph-mon[34759]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:20 smithi031 ceph-mon[31209]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:20.834 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:45:20 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0[47986]: 2024-09-21T02:45:20.552+0000 7f687c82d640 -1 osd.0 0 log_to_monitors true 2024-09-21T02:45:21.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:21 smithi031 ceph-mon[31209]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T02:45:21.656 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:21 smithi165 ceph-mon[34818]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T02:45:21.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:21 smithi178 ceph-mon[34759]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T02:45:22.343 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:45:22.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-09-21T02:45:22.549 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0[47986]: 2024-09-21T02:45:22.421+0000 7f68787af640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T02:45:22.549 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0[47986]: 2024-09-21T02:45:22.424+0000 7f68745ad640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:45:22.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-mon[31209]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:22.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-mon[31209]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T02:45:22.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-mon[31209]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T02:45:22.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-mon[31209]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:45:22.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:22.611 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:45:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:22 smithi165 ceph-mon[34818]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:22 smithi165 ceph-mon[34818]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T02:45:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:22 smithi165 ceph-mon[34818]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T02:45:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:22 smithi165 ceph-mon[34818]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:45:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:22.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:22 smithi178 ceph-mon[34759]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:22.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:22 smithi178 ceph-mon[34759]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T02:45:22.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:22 smithi178 ceph-mon[34759]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T02:45:22.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:22 smithi178 ceph-mon[34759]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:45:22.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 -- 172.21.15.165:0/4287367301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0103190 msgr2=0x7fafb01035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 --2- 172.21.15.165:0/4287367301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0103190 0x7fafb01035a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009900 tx=0x7fafa002a410 comp rx=0 tx=0).stop 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 -- 172.21.15.165:0/4287367301 shutdown_connections 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 --2- 172.21.15.165:0/4287367301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb0108d50 0x7fafb010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 --2- 172.21.15.165:0/4287367301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108400 0x7fafb0108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 --2- 172.21.15.165:0/4287367301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0103190 0x7fafb01035a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 -- 172.21.15.165:0/4287367301 >> 172.21.15.165:0/4287367301 conn(0x7fafb0079f90 msgr2=0x7fafb007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 -- 172.21.15.165:0/4287367301 shutdown_connections 2024-09-21T02:45:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.035+0000 7fafb4824640 1 -- 172.21.15.165:0/4287367301 wait complete. 2024-09-21T02:45:23.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 Processor -- start 2024-09-21T02:45:23.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 -- start start 2024-09-21T02:45:23.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb0103190 0x7fafb0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 0x7fafb01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108d50 0x7fafb019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fafb0112a10 con 0x7fafb0108d50 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fafb0112890 con 0x7fafb0108400 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafb4824640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fafb0112b90 con 0x7fafb0103190 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafadd9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 0x7fafb01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafadd9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 0x7fafb01999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56172/0 (socket says 172.21.15.165:56172) 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafadd9b640 1 -- 172.21.15.165:0/3722726502 learned_addr learned my addr 172.21.15.165:0/3722726502 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.036+0000 7fafae59c640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb0103190 0x7fafb0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafaed9d640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108d50 0x7fafb019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafadd9b640 1 -- 172.21.15.165:0/3722726502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb0103190 msgr2=0x7fafb0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafadd9b640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb0103190 0x7fafb0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafadd9b640 1 -- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108d50 msgr2=0x7fafb019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafadd9b640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108d50 0x7fafb019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafadd9b640 1 -- 172.21.15.165:0/3722726502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb019f390 con 0x7fafb0108400 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafaed9d640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108d50 0x7fafb019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafadd9b640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 0x7fafb01999c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafa400e9f0 tx=0x7fafa400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400ccc0 con 0x7fafb0108400 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa40108e0 con 0x7fafb0108400 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb019f680 con 0x7fafb0108400 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4018860 con 0x7fafb0108400 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.037+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01a5de0 con 0x7fafb0108400 2024-09-21T02:45:23.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.038+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb01035a0 con 0x7fafb0108400 2024-09-21T02:45:23.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.040+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fafa4010430 con 0x7fafb0108400 2024-09-21T02:45:23.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.041+0000 7faf9f7fe640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7faf980b6550 0x7faf980b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:23.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.041+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7fafa40e60b0 con 0x7fafb0108400 2024-09-21T02:45:23.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.041+0000 7fafae59c640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7faf980b6550 0x7faf980b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:23.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.041+0000 7fafae59c640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7faf980b6550 0x7faf980b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fafa0031440 tx=0x7fafa0005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:45:23.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.044+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fafa40ae3d0 con 0x7fafb0108400 2024-09-21T02:45:23.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:23.230+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fafb010a5f0 con 0x7faf980b6550 2024-09-21T02:45:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:45:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T02:45:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi031 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:45:23.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi031 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-21T02:45:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:23.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:23.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:45:23.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:45:23.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:45:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T02:45:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi031 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:45:23.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:24.637 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:45:24.638 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:45:24.638 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:24.638 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:45:24.638 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] boot 2024-09-21T02:45:24.638 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T02:45:24.638 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] boot 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T02:45:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: osd.0 [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] boot 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T02:45:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:45:25.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:25 smithi165 ceph-mon[34818]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T02:45:25.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:25 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/1281205381' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"} : dispatch 2024-09-21T02:45:25.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:25 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"} : dispatch 2024-09-21T02:45:25.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:25 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"}]': finished 2024-09-21T02:45:25.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:25 smithi165 ceph-mon[34818]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T02:45:25.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:25 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:25.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:25 smithi178 ceph-mon[34759]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T02:45:25.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:25 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/1281205381' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"} : dispatch 2024-09-21T02:45:25.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:25 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"} : dispatch 2024-09-21T02:45:25.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:25 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"}]': finished 2024-09-21T02:45:25.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:25 smithi178 ceph-mon[34759]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T02:45:25.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:25 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:25.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:25 smithi031 ceph-mon[31209]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T02:45:25.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:25 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/1281205381' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"} : dispatch 2024-09-21T02:45:25.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:25 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"} : dispatch 2024-09-21T02:45:25.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:25 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8"}]': finished 2024-09-21T02:45:25.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:25 smithi031 ceph-mon[31209]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T02:45:25.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:25 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:26.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:26 smithi165 ceph-mon[34818]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:26.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:26 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/320504786' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:26.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:26 smithi178 ceph-mon[34759]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:26.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:26 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/320504786' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:26 smithi031 ceph-mon[31209]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:26 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/320504786' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:28.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:28 smithi178 ceph-mon[34759]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:28 smithi031 ceph-mon[31209]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:28 smithi165 ceph-mon[34818]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:30.689 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:30 smithi165 ceph-mon[34818]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:30.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:30 smithi178 ceph-mon[34759]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:30.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:30 smithi031 ceph-mon[31209]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:32 smithi165 ceph-mon[34818]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:32 smithi178 ceph-mon[34759]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:32.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:32 smithi031 ceph-mon[31209]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:33.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:33 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T02:45:33.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:33 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:33.737 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:33 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T02:45:33.737 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:33 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:33.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:33 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T02:45:33.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:33 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:34.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:34 smithi178 ceph-mon[34759]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:34.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:34 smithi178 ceph-mon[34759]: Deploying daemon osd.1 on smithi165 2024-09-21T02:45:34.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:34 smithi031 ceph-mon[31209]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:34.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:34 smithi031 ceph-mon[31209]: Deploying daemon osd.1 on smithi165 2024-09-21T02:45:34.771 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:34 smithi165 ceph-mon[34818]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:34.771 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:34 smithi165 ceph-mon[34818]: Deploying daemon osd.1 on smithi165 2024-09-21T02:45:36.550 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:36 smithi165 ceph-mon[34818]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:36 smithi178 ceph-mon[34759]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:36 smithi031 ceph-mon[31209]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:38.569 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:38 smithi165 ceph-mon[34818]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:38 smithi178 ceph-mon[34759]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:38 smithi031 ceph-mon[31209]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:39.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:39 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:39.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:39 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:39.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:39 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:39.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:39 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:39.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:39 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:39.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:39 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:39.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:39 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:39.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:39 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:39.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:39 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:40 smithi178 ceph-mon[34759]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:40.771 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:40 smithi165 ceph-mon[34818]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:40 smithi031 ceph-mon[31209]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:42 smithi031 ceph-mon[31209]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:42 smithi165 ceph-mon[34818]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.095 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:43.096 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:42 smithi178 ceph-mon[34759]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:43.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.395+0000 7faf9f7fe640 1 -- 172.21.15.165:0/3722726502 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fafb010a5f0 con 0x7faf980b6550 2024-09-21T02:45:43.397 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 1 on host 'smithi165' 2024-09-21T02:45:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7faf980b6550 msgr2=0x7faf980b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7faf980b6550 0x7faf980b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fafa0031440 tx=0x7fafa0005d50 comp rx=0 tx=0).stop 2024-09-21T02:45:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 msgr2=0x7fafb01999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 0x7fafb01999c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafa400e9f0 tx=0x7fafa400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:45:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 shutdown_connections 2024-09-21T02:45:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7faf980b6550 0x7faf980b8a60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0108d50 0x7fafb019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafb0108400 0x7fafb01999c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 --2- 172.21.15.165:0/3722726502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb0103190 0x7fafb0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 >> 172.21.15.165:0/3722726502 conn(0x7fafb0079f90 msgr2=0x7fafb01064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:45:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 shutdown_connections 2024-09-21T02:45:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:45:43.398+0000 7fafb4824640 1 -- 172.21.15.165:0/3722726502 wait complete. 2024-09-21T02:45:44.049 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:43 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:44.049 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:43 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:44.049 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:43 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:44.051 DEBUG:teuthology.orchestra.run.smithi165:osd.1> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T02:45:44.053 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_4... 2024-09-21T02:45:44.053 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T02:45:44.079 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:43 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:44.079 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:43 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:44.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:43 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:44.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:43 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:45:44.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:43 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:44.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:43 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:44.303 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:45:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:44 smithi031 ceph-mon[31209]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:45.086 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:44 smithi178 ceph-mon[34759]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:45.147 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:44 smithi165 ceph-mon[34818]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:45.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:45 smithi178 ceph-mon[34759]: from='osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:45:45.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:45 smithi178 ceph-mon[34759]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:45:46.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:45 smithi031 ceph-mon[31209]: from='osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:45:46.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:45 smithi031 ceph-mon[31209]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:45:46.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:45 smithi165 ceph-mon[34818]: from='osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:45:46.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:45 smithi165 ceph-mon[34818]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:45:46.107 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:45:45 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1[39455]: 2024-09-21T02:45:45.696+0000 7f78d911d640 -1 osd.1 0 log_to_monitors true 2024-09-21T02:45:46.690 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:45:46.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:45:47.008 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:45:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T02:45:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:45:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T02:45:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:47.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:45:47.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:47.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:47.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:45:47.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 -- 172.21.15.178:0/3826046666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d41094b0 msgr2=0x7f34d41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:47.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 --2- 172.21.15.178:0/3826046666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d41094b0 0x7f34d41098c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009900 tx=0x7f34c802a410 comp rx=0 tx=0).stop 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 -- 172.21.15.178:0/3826046666 shutdown_connections 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 --2- 172.21.15.178:0/3826046666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d41094b0 0x7f34d41098c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 --2- 172.21.15.178:0/3826046666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d4109e00 0x7f34d4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 --2- 172.21.15.178:0/3826046666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d4103b70 0x7f34d4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 -- 172.21.15.178:0/3826046666 >> 172.21.15.178:0/3826046666 conn(0x7f34d40ff5e0 msgr2=0x7f34d4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 -- 172.21.15.178:0/3826046666 shutdown_connections 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 -- 172.21.15.178:0/3826046666 wait complete. 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.481+0000 7f34da33d640 1 Processor -- start 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 -- start start 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 0x7f34d419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d41094b0 0x7f34d419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:47.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d4109e00 0x7f34d419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34d4113ac0 con 0x7f34d41094b0 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34d4113940 con 0x7f34d4109e00 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34da33d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34d4113c40 con 0x7f34d4103b70 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 0x7f34d419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 0x7f34d419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:50762/0 (socket says 172.21.15.178:50762) 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 -- 172.21.15.178:0/3760931289 learned_addr learned my addr 172.21.15.178:0/3760931289 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d88da640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d4109e00 0x7f34d419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d37fe640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d41094b0 0x7f34d419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 -- 172.21.15.178:0/3760931289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d4109e00 msgr2=0x7f34d419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d4109e00 0x7f34d419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 -- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d41094b0 msgr2=0x7f34d419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:45:47.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d41094b0 0x7f34d419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d3fff640 1 -- 172.21.15.178:0/3760931289 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d41a03a0 con 0x7f34d4103b70 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.482+0000 7f34d37fe640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d41094b0 0x7f34d419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.483+0000 7f34d3fff640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 0x7f34d419a4f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00eab0 tx=0x7f34cc00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.483+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00ce40 con 0x7f34d4103b70 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.483+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34cc011920 con 0x7f34d4103b70 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.483+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d41a0690 con 0x7f34d4103b70 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.483+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc010a30 con 0x7f34d4103b70 2024-09-21T02:45:47.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.483+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d41a6eb0 con 0x7f34d4103b70 2024-09-21T02:45:47.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.484+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d4103f80 con 0x7f34d4103b70 2024-09-21T02:45:47.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.487+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f34cc010bd0 con 0x7f34d4103b70 2024-09-21T02:45:47.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.487+0000 7f34d17fa640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34bc0b6580 0x7f34bc0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:45:47.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.487+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7f34cc0e6020 con 0x7f34d4103b70 2024-09-21T02:45:47.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.488+0000 7f34d37fe640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34bc0b6580 0x7f34bc0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:45:47.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.488+0000 7f34d37fe640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34bc0b6580 0x7f34bc0b8a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009230 tx=0x7f34c8000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:45:47.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.490+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34cc016030 con 0x7f34d4103b70 2024-09-21T02:45:47.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:45:47.676+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f34d4063800 con 0x7f34bc0b6580 2024-09-21T02:45:47.691 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1[39455]: 2024-09-21T02:45:47.216+0000 7f78d509f640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T02:45:47.691 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1[39455]: 2024-09-21T02:45:47.221+0000 7f78cce95640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:45:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi165 2024-09-21T02:45:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi165 to 17088M 2024-09-21T02:45:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='osd.1 ' entity='osd.1' 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:45:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi165 2024-09-21T02:45:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi165 to 17088M 2024-09-21T02:45:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='osd.1 ' entity='osd.1' 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:45:48.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi165 2024-09-21T02:45:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi165 to 17088M 2024-09-21T02:45:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='osd.1 ' entity='osd.1' 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:45:48.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:45:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:45:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:45:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] boot 2024-09-21T02:45:49.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T02:45:49.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] boot 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T02:45:49.121 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: osd.1 [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] boot 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T02:45:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/3807126963' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"} : dispatch 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"} : dispatch 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"}]': finished 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T02:45:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:50 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/3807126963' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"} : dispatch 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"} : dispatch 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"}]': finished 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T02:45:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:50 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/3807126963' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"} : dispatch 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"} : dispatch 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eca4fd08-95dd-4326-a6e7-59be134cad7a"}]': finished 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T02:45:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:50 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:45:51.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:51 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/4179633592' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:51.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:51 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/4179633592' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:51.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:51 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/4179633592' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:45:52.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:52 smithi031 ceph-mon[31209]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:52 smithi165 ceph-mon[34818]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:52 smithi178 ceph-mon[34759]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:54.397 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:54 smithi178 ceph-mon[34759]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:54.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:54 smithi031 ceph-mon[31209]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:54.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:54 smithi165 ceph-mon[34818]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:56.442 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:56 smithi178 ceph-mon[34759]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:56.493 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:56 smithi165 ceph-mon[34818]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:56 smithi031 ceph-mon[31209]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:58.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:58 smithi178 ceph-mon[34759]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:58.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T02:45:58.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:58.335 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:45:58 smithi178 ceph-mon[34759]: Deploying daemon osd.2 on smithi178 2024-09-21T02:45:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:58 smithi031 ceph-mon[31209]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T02:45:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:45:58 smithi031 ceph-mon[31209]: Deploying daemon osd.2 on smithi178 2024-09-21T02:45:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:58 smithi165 ceph-mon[34818]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:45:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T02:45:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:45:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:45:58 smithi165 ceph-mon[34818]: Deploying daemon osd.2 on smithi178 2024-09-21T02:46:00.453 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:00 smithi178 ceph-mon[34759]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:00.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:00 smithi031 ceph-mon[31209]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:00.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:00 smithi165 ceph-mon[34818]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:02 smithi031 ceph-mon[31209]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:02.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:02 smithi178 ceph-mon[34759]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:02 smithi165 ceph-mon[34818]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:04.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:04 smithi031 ceph-mon[31209]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:04.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:04 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:04.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:04 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:04.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:04 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:04.613 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:04 smithi178 ceph-mon[34759]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:04.613 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:04 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:04.613 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:04 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:04.614 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:04 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:04.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:04 smithi165 ceph-mon[34818]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:04.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:04 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:04.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:04 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:04.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:04 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:06 smithi178 ceph-mon[34759]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:06 smithi031 ceph-mon[31209]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:06.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:06 smithi165 ceph-mon[34818]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:07.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.519+0000 7f34d17fa640 1 -- 172.21.15.178:0/3760931289 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34d4063800 con 0x7f34bc0b6580 2024-09-21T02:46:07.521 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34bc0b6580 msgr2=0x7f34bc0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34bc0b6580 0x7f34bc0b8a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009230 tx=0x7f34c8000bc0 comp rx=0 tx=0).stop 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 msgr2=0x7f34d419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 0x7f34d419a4f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00eab0 tx=0x7f34cc00ef70 comp rx=0 tx=0).stop 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 shutdown_connections 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34bc0b6580 0x7f34bc0b8a90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d4109e00 0x7f34d419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d41094b0 0x7f34d419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 --2- 172.21.15.178:0/3760931289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d4103b70 0x7f34d419a4f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 >> 172.21.15.178:0/3760931289 conn(0x7f34d40ff5e0 msgr2=0x7f34d4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 shutdown_connections 2024-09-21T02:46:07.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:46:07.522+0000 7f34da33d640 1 -- 172.21.15.178:0/3760931289 wait complete. 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:07 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:07.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:07.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:07.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:07.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:07 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:08.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:08.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:08.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:08.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:08.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:08.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:08.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:08.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:07 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:08.160 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T02:46:08.163 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-09-21T02:46:08.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T02:46:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:46:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:08 smithi178 ceph-mon[34759]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:08.824 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:08 smithi031 ceph-mon[31209]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:08 smithi165 ceph-mon[34818]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:10.217 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:09 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2[39921]: 2024-09-21T02:46:09.887+0000 7f1df5761640 -1 osd.2 0 log_to_monitors true 2024-09-21T02:46:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:10 smithi178 ceph-mon[34759]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:10 smithi178 ceph-mon[34759]: from='osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:46:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:10 smithi178 ceph-mon[34759]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:46:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:10 smithi031 ceph-mon[31209]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:10 smithi031 ceph-mon[31209]: from='osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:46:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:10 smithi031 ceph-mon[31209]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:46:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:46:10.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-09-21T02:46:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:10 smithi165 ceph-mon[34818]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:10 smithi165 ceph-mon[34818]: from='osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:46:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:10 smithi165 ceph-mon[34818]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:46:11.139 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:46:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.592+0000 7f0b633a4640 1 -- 172.21.15.31:0/2948195601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c110190 msgr2=0x7f0b5c1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.592+0000 7f0b633a4640 1 --2- 172.21.15.31:0/2948195601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c110190 0x7f0b5c1125b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009900 tx=0x7f0b5402a480 comp rx=0 tx=0).stop 2024-09-21T02:46:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 -- 172.21.15.31:0/2948195601 shutdown_connections 2024-09-21T02:46:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 --2- 172.21.15.31:0/2948195601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c110190 0x7f0b5c1125b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 --2- 172.21.15.31:0/2948195601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 0x7f0b5c10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 --2- 172.21.15.31:0/2948195601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 -- 172.21.15.31:0/2948195601 >> 172.21.15.31:0/2948195601 conn(0x7f0b5c079730 msgr2=0x7f0b5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 -- 172.21.15.31:0/2948195601 shutdown_connections 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 -- 172.21.15.31:0/2948195601 wait complete. 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 Processor -- start 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.593+0000 7f0b633a4640 1 -- start start 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 0x7f0b5c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b5c110190 0x7f0b5c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c115c80 con 0x7f0b5c1025e0 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c115b00 con 0x7f0b5c102f30 2024-09-21T02:46:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c115e00 con 0x7f0b5c110190 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50650/0 (socket says 172.21.15.31:50650) 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 -- 172.21.15.31:0/3122339635 learned_addr learned my addr 172.21.15.31:0/3122339635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b6093f640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 0x7f0b5c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61941640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b5c110190 0x7f0b5c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 -- 172.21.15.31:0/3122339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b5c110190 msgr2=0x7f0b5c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b5c110190 0x7f0b5c1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 -- 172.21.15.31:0/3122339635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 msgr2=0x7f0b5c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 0x7f0b5c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 -- 172.21.15.31:0/3122339635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c1a2580 con 0x7f0b5c1025e0 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b6093f640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 0x7f0b5c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b61140640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c19c610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800cce0 tx=0x7f0b5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:46:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58004b20 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b58004cc0 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c1a2870 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.594+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800f7c0 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.595+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c1a8fd0 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.595+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c06c1e0 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.597+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f0b58005560 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.598+0000 7f0b527fc640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b480b6550 0x7f0b480b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.598+0000 7f0b6093f640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b480b6550 0x7f0b480b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.598+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2660+0+0 (secure 0 0 0) 0x7f0b580e6180 con 0x7f0b5c1025e0 2024-09-21T02:46:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.598+0000 7f0b6093f640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b480b6550 0x7f0b480b8a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c1040b0 tx=0x7f0b4c00b5f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:46:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.601+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b58014030 con 0x7f0b5c1025e0 2024-09-21T02:46:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:11.801+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0b5c109310 con 0x7f0b480b6550 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi178 2024-09-21T02:46:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:11.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.978 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2[39921]: 2024-09-21T02:46:11.643+0000 7f1df16e3640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T02:46:11.978 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2[39921]: 2024-09-21T02:46:11.649+0000 7f1ded4e1640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi178 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-21T02:46:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi178 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-21T02:46:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:12.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:46:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:46:12.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:46:12.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:12.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:13 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:46:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:13 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:46:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:13 smithi165 ceph-mon[34818]: osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] boot 2024-09-21T02:46:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:13 smithi165 ceph-mon[34818]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T02:46:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:46:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:13 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:46:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:13 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:46:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:13 smithi178 ceph-mon[34759]: osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] boot 2024-09-21T02:46:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:13 smithi178 ceph-mon[34759]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T02:46:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:46:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:13 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:46:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:13 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:46:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:13 smithi031 ceph-mon[31209]: osd.2 [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] boot 2024-09-21T02:46:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:13 smithi031 ceph-mon[31209]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T02:46:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:46:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52403]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52403]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52406]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52406]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52406]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52406]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52391]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52391]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52391]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52391]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52394]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.533 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 02:46:14 smithi031 sudo[52394]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.666 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:46:14.666 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.666 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43399]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.666 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43399]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.666 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43402]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:46:14.666 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.667 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43402]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.667 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43402]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43405]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43405]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43408]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 sudo[43408]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.667 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43443]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43443]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43446]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43446]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.668 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43446]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.669 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43446]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.669 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43437]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43437]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43437]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43437]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43440]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43440]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43440]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:46:14.669 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 02:46:14 smithi165 sudo[43440]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3747883825' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "68acf89d-75fa-43bf-8181-73e5043cc7a5"} : dispatch 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3747883825' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68acf89d-75fa-43bf-8181-73e5043cc7a5"}]': finished 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:14.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.810 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.810 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:14 smithi031 ceph-mon[31209]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T02:46:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T02:46:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:46:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3747883825' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "68acf89d-75fa-43bf-8181-73e5043cc7a5"} : dispatch 2024-09-21T02:46:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3747883825' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68acf89d-75fa-43bf-8181-73e5043cc7a5"}]': finished 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:14 smithi165 ceph-mon[34818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3747883825' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "68acf89d-75fa-43bf-8181-73e5043cc7a5"} : dispatch 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3747883825' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68acf89d-75fa-43bf-8181-73e5043cc7a5"}]': finished 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:46:14.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:46:14.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:46:14.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:14 smithi178 ceph-mon[34759]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:46:15.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:15.678+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0b580a9cf0 con 0x7f0b5c1025e0 2024-09-21T02:46:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:15 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1961451463' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:46:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:15 smithi165 ceph-mon[34818]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T02:46:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:15 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1961451463' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:46:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:15 smithi178 ceph-mon[34759]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T02:46:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:15 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1961451463' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:46:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:15 smithi031 ceph-mon[31209]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T02:46:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:16 smithi165 ceph-mon[34818]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:16 smithi165 ceph-mon[34818]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T02:46:16.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:16 smithi178 ceph-mon[34759]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:16 smithi178 ceph-mon[34759]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T02:46:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:16 smithi031 ceph-mon[31209]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:16 smithi031 ceph-mon[31209]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T02:46:18.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:18 smithi165 ceph-mon[34818]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:18.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:18 smithi178 ceph-mon[34759]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:18 smithi031 ceph-mon[31209]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:20.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:20 smithi031 ceph-mon[31209]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:20.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:20 smithi165 ceph-mon[34818]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:20 smithi178 ceph-mon[34759]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:22.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:22 smithi031 ceph-mon[31209]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:22.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T02:46:22.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:22 smithi165 ceph-mon[34818]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T02:46:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:22 smithi178 ceph-mon[34759]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T02:46:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:23.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:23 smithi178 ceph-mon[34759]: Deploying daemon osd.3 on smithi031 2024-09-21T02:46:24.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:23 smithi031 ceph-mon[31209]: Deploying daemon osd.3 on smithi031 2024-09-21T02:46:24.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:23 smithi165 ceph-mon[34818]: Deploying daemon osd.3 on smithi031 2024-09-21T02:46:24.940 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:24 smithi031 ceph-mon[31209]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:24 smithi178 ceph-mon[34759]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:24 smithi165 ceph-mon[34818]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:26.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:26 smithi178 ceph-mon[34759]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:26.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:26 smithi165 ceph-mon[34818]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:26 smithi031 ceph-mon[31209]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:28 smithi178 ceph-mon[34759]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:28 smithi031 ceph-mon[31209]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:28 smithi165 ceph-mon[34818]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:29 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:29 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:29 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:29.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:29 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:29.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:29 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:29.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:29 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:30.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:29 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:30.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:29 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:30.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:29 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:30 smithi031 ceph-mon[31209]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:30 smithi165 ceph-mon[34818]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:31.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:30 smithi178 ceph-mon[34759]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:32 smithi031 ceph-mon[31209]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:32 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:32 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:32 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:32 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:32 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:32 smithi165 ceph-mon[34818]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:32 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:32 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:32 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:32 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:32 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:32 smithi178 ceph-mon[34759]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:32 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:32 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:32 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:32 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:32 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:32.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.945+0000 7f0b527fc640 1 -- 172.21.15.31:0/3122339635 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b5c109310 con 0x7f0b480b6550 2024-09-21T02:46:32.946 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b480b6550 msgr2=0x7f0b480b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b480b6550 0x7f0b480b8a60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c1040b0 tx=0x7f0b4c00b5f0 comp rx=0 tx=0).stop 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 msgr2=0x7f0b5c19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c19c610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800cce0 tx=0x7f0b5800a620 comp rx=0 tx=0).stop 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 shutdown_connections 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b480b6550 0x7f0b480b8a60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b5c110190 0x7f0b5c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b5c102f30 0x7f0b5c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 --2- 172.21.15.31:0/3122339635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c1025e0 0x7f0b5c19c610 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.947+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 >> 172.21.15.31:0/3122339635 conn(0x7f0b5c079730 msgr2=0x7f0b5c108c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:46:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.948+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 shutdown_connections 2024-09-21T02:46:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:46:32.948+0000 7f0b633a4640 1 -- 172.21.15.31:0/3122339635 wait complete. 2024-09-21T02:46:33.573 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:33 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:33.573 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:33 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:33.573 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:33 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:33.609 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T02:46:33.612 INFO:tasks.cephadm:Deploying osd.4 on smithi165 with /dev/vg_nvme/lv_3... 2024-09-21T02:46:33.612 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T02:46:33.640 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:33 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:33.640 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:33 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:33.640 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:33 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:33.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:33 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:33.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:33 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:33.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:33 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:33.864 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:46:34.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:34 smithi031 ceph-mon[31209]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:34.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:34 smithi165 ceph-mon[34818]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:34.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:34 smithi178 ceph-mon[34759]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:34.995 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 02:46:34 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T02:46:34.693+0000 7f159393b640 -1 osd.3 0 log_to_monitors true 2024-09-21T02:46:35.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:35 smithi165 ceph-mon[34818]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T02:46:35.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:35 smithi031 ceph-mon[31209]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T02:46:35.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:35 smithi178 ceph-mon[34759]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T02:46:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:46:36.265 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-09-21T02:46:36.552 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:46:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-mon[31209]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-mon[31209]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T02:46:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-mon[31209]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T02:46:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-mon[31209]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:46:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:36.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:36 smithi165 ceph-mon[34818]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:36.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:36 smithi165 ceph-mon[34818]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T02:46:36.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:36 smithi165 ceph-mon[34818]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T02:46:36.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:36 smithi165 ceph-mon[34818]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:46:36.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:36 smithi178 ceph-mon[34759]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:36 smithi178 ceph-mon[34759]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T02:46:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:36 smithi178 ceph-mon[34759]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T02:46:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:36 smithi178 ceph-mon[34759]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:46:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3390106505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 msgr2=0x7f68f81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:37.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3390106505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f81098b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009900 tx=0x7f68ec02a410 comp rx=0 tx=0).stop 2024-09-21T02:46:37.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3390106505 shutdown_connections 2024-09-21T02:46:37.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3390106505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 0x7f68f8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:37.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3390106505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f81098b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:37.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3390106505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f8103b60 0x7f68f8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:37.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3390106505 >> 172.21.15.165:0/3390106505 conn(0x7f68f807b850 msgr2=0x7f68f807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:46:37.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3390106505 shutdown_connections 2024-09-21T02:46:37.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.013+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3390106505 wait complete. 2024-09-21T02:46:37.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 Processor -- start 2024-09-21T02:46:37.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 -- start start 2024-09-21T02:46:37.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f8103b60 0x7f68f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:37.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:37.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 0x7f68f8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:37.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f68f8113ad0 con 0x7f68f8109df0 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68f8113950 con 0x7f68f81094a0 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.014+0000 7f68fc8f3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f68f8113c50 con 0x7f68f8103b60 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39270/0 (socket says 172.21.15.165:39270) 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 -- 172.21.15.165:0/3595627398 learned_addr learned my addr 172.21.15.165:0/3595627398 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f6d9d640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 0x7f68f8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f659c640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f8103b60 0x7f68f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 -- 172.21.15.165:0/3595627398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f8103b60 msgr2=0x7f68f8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f8103b60 0x7f68f8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:37.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 -- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 msgr2=0x7f68f8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 0x7f68f8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 -- 172.21.15.165:0/3595627398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f81a8780 con 0x7f68f81094a0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f6d9d640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 0x7f68f8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68f5d9b640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f8075c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68ec02a930 tx=0x7f68ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec03f030 con 0x7f68f81094a0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f81a8a10 con 0x7f68f81094a0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ec003aa0 con 0x7f68f81094a0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec035070 con 0x7f68f81094a0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.015+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f81a8fc0 con 0x7f68f81094a0 2024-09-21T02:46:37.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.016+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f8103f70 con 0x7f68f81094a0 2024-09-21T02:46:37.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.018+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f68ec042030 con 0x7f68f81094a0 2024-09-21T02:46:37.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.019+0000 7f68e77fe640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f68e00b6560 0x7f68e00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:46:37.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.019+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3637+0+0 (secure 0 0 0) 0x7f68ec106be0 con 0x7f68f81094a0 2024-09-21T02:46:37.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.019+0000 7f68f659c640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f68e00b6560 0x7f68e00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:46:37.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.019+0000 7f68f659c640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f68e00b6560 0x7f68e00b8a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68f8077160 tx=0x7f68e8009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:46:37.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.022+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68ec0ce6b0 con 0x7f68f81094a0 2024-09-21T02:46:37.031 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T02:46:36.862+0000 7f158f8bd640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T02:46:37.031 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 02:46:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T02:46:36.869+0000 7f15876b3640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:46:37.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:37.209+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f68f8063800 con 0x7f68e00b6560 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi031 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:46:37.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:37 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi031 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.588 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:46:37.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:37 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi031 2024-09-21T02:46:37.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073]' entity='osd.3' 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:46:37.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:37 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:38.537 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:46:38.537 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:46:38.537 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:38.537 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:46:38.537 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:38.537 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] boot 2024-09-21T02:46:38.538 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T02:46:38.538 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] boot 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T02:46:38.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: osd.3 [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] boot 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T02:46:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:46:39.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:39 smithi178 ceph-mon[34759]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T02:46:39.764 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:39 smithi165 ceph-mon[34818]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T02:46:39.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:39 smithi031 ceph-mon[31209]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/2248730229' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"} : dispatch 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"} : dispatch 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"}]': finished 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:46:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:40 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/1170882876' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/2248730229' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"} : dispatch 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"} : dispatch 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"}]': finished 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:46:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:40 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/1170882876' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/2248730229' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"} : dispatch 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"} : dispatch 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edf3d333-8a5b-4a48-81ac-50917b7593c6"}]': finished 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:46:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:40 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/1170882876' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:46:42.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:42 smithi178 ceph-mon[34759]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:42 smithi031 ceph-mon[31209]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:42.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:42 smithi165 ceph-mon[34818]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:44 smithi165 ceph-mon[34818]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:44 smithi178 ceph-mon[34759]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:44.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:44 smithi031 ceph-mon[31209]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:46 smithi165 ceph-mon[34818]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T02:46:46.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:46 smithi031 ceph-mon[31209]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T02:46:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:46 smithi178 ceph-mon[34759]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T02:46:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:48 smithi165 ceph-mon[34818]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:46:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T02:46:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:48 smithi031 ceph-mon[31209]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:46:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T02:46:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:48.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:48 smithi178 ceph-mon[34759]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:46:48.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T02:46:48.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:49.635 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:49 smithi165 ceph-mon[34818]: Deploying daemon osd.4 on smithi165 2024-09-21T02:46:49.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:49 smithi031 ceph-mon[31209]: Deploying daemon osd.4 on smithi165 2024-09-21T02:46:49.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:49 smithi178 ceph-mon[34759]: Deploying daemon osd.4 on smithi165 2024-09-21T02:46:50.636 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:50 smithi165 ceph-mon[34818]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T02:46:50.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:50 smithi031 ceph-mon[31209]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T02:46:50.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:50 smithi178 ceph-mon[34759]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T02:46:52.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:52 smithi031 ceph-mon[31209]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T02:46:52.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:52 smithi165 ceph-mon[34818]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T02:46:52.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:52 smithi178 ceph-mon[34759]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T02:46:54.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:54 smithi031 ceph-mon[31209]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:46:54.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:54.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:54.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:54.835 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:54 smithi165 ceph-mon[34818]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:46:54.835 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:54.835 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:54.835 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:54.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:54 smithi178 ceph-mon[34759]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:46:54.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:54.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:54.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:56.611 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:56 smithi165 ceph-mon[34818]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:46:56.780 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:56 smithi178 ceph-mon[34759]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:46:56.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:56 smithi031 ceph-mon[31209]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:46:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:57 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:57 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:57 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:57 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:57.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:57 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:57 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:57 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:57 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:58.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:57 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:58.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:57 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:57 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:57 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:57 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:46:58.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:57 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:46:58.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:57 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.148+0000 7f68e77fe640 1 -- 172.21.15.165:0/3595627398 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68f8063800 con 0x7f68e00b6560 2024-09-21T02:46:58.150 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 4 on host 'smithi165' 2024-09-21T02:46:58.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f68e00b6560 msgr2=0x7f68e00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f68e00b6560 0x7f68e00b8a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68f8077160 tx=0x7f68e8009110 comp rx=0 tx=0).stop 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 msgr2=0x7f68f8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f8075c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68ec02a930 tx=0x7f68ec004270 comp rx=0 tx=0).stop 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 shutdown_connections 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f68e00b6560 0x7f68e00b8a70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68f8109df0 0x7f68f8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f81094a0 0x7f68f8075c50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 --2- 172.21.15.165:0/3595627398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f8103b60 0x7f68f8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 >> 172.21.15.165:0/3595627398 conn(0x7f68f807b850 msgr2=0x7f68f8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 shutdown_connections 2024-09-21T02:46:58.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:46:58.151+0000 7f68fc8f3640 1 -- 172.21.15.165:0/3595627398 wait complete. 2024-09-21T02:46:58.778 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:58 smithi165 ceph-mon[34818]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:58.778 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:58.778 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.778 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:46:58 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:58.779 DEBUG:teuthology.orchestra.run.smithi165:osd.4> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T02:46:58.782 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-09-21T02:46:58.782 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T02:46:59.028 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:46:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:58 smithi031 ceph-mon[31209]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:46:58 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:59.149 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:58 smithi178 ceph-mon[34759]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:46:59.149 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:46:59.149 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:46:59.149 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:46:58 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:00.253 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 02:46:59 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4[46041]: 2024-09-21T02:46:59.940+0000 7fb14baeb640 -1 osd.4 0 log_to_monitors true 2024-09-21T02:47:00.959 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:00 smithi178 ceph-mon[34759]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:00.960 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:00 smithi178 ceph-mon[34759]: from='osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:47:00.960 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:00 smithi178 ceph-mon[34759]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:47:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:00 smithi031 ceph-mon[31209]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:00 smithi031 ceph-mon[31209]: from='osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:47:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:00 smithi031 ceph-mon[31209]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:47:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:00 smithi165 ceph-mon[34818]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:00 smithi165 ceph-mon[34818]: from='osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:47:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:00 smithi165 ceph-mon[34818]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:47:01.642 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:47:01.668 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-09-21T02:47:01.926 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:47:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:01 smithi178 ceph-mon[34759]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T02:47:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:01 smithi178 ceph-mon[34759]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T02:47:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:01 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:01 smithi178 ceph-mon[34759]: from='osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:47:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:01 smithi178 ceph-mon[34759]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:47:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:01 smithi031 ceph-mon[31209]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T02:47:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:01 smithi031 ceph-mon[31209]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T02:47:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:01 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:01 smithi031 ceph-mon[31209]: from='osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:47:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:01 smithi031 ceph-mon[31209]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:47:02.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:01 smithi165 ceph-mon[34818]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T02:47:02.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:01 smithi165 ceph-mon[34818]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T02:47:02.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:01 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:02.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:01 smithi165 ceph-mon[34818]: from='osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:47:02.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:01 smithi165 ceph-mon[34818]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:47:02.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 -- 172.21.15.178:0/1448830890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 msgr2=0x7f3478104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:02.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 --2- 172.21.15.178:0/1448830890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f3478104dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3468009900 tx=0x7f346802a410 comp rx=0 tx=0).stop 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 -- 172.21.15.178:0/1448830890 shutdown_connections 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 --2- 172.21.15.178:0/1448830890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f347810de10 0x7f3478110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 --2- 172.21.15.178:0/1448830890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3478105300 0x7f347810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 --2- 172.21.15.178:0/1448830890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f3478104dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 -- 172.21.15.178:0/1448830890 >> 172.21.15.178:0/1448830890 conn(0x7f3478079730 msgr2=0x7f3478079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 -- 172.21.15.178:0/1448830890 shutdown_connections 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.401+0000 7f347e1fa640 1 -- 172.21.15.178:0/1448830890 wait complete. 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 Processor -- start 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 -- start start 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f347819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3478105300 0x7f347819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f347810de10 0x7f347819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3478113a70 con 0x7f347810de10 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34781138f0 con 0x7f3478105300 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f347e1fa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3478113bf0 con 0x7f34781049b0 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f347819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f347819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48082/0 (socket says 172.21.15.178:48082) 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 -- 172.21.15.178:0/2529200542 learned_addr learned my addr 172.21.15.178:0/2529200542 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f3477fff640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f347810de10 0x7f347819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f346effd640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3478105300 0x7f347819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 -- 172.21.15.178:0/2529200542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3478105300 msgr2=0x7f347819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3478105300 0x7f347819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 -- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f347810de10 msgr2=0x7f347819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:02.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f347810de10 0x7f347819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.402+0000 7f34777fe640 1 -- 172.21.15.178:0/2529200542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34781a0490 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.403+0000 7f34777fe640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f347819a520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3468032fd0 tx=0x7f3468004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.403+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346803f030 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.403+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34781a0720 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.403+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3468004490 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.403+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468035070 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.403+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781a6ee0 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.404+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347806b560 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.406+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3468042030 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.407+0000 7f34757fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34640b6560 0x7f34640b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.407+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4036+0+0 (secure 0 0 0) 0x7f3468106b00 con 0x7f34781049b0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.407+0000 7f346effd640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34640b6560 0x7f34640b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.408+0000 7f346effd640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34640b6560 0x7f34640b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3478102470 tx=0x7f346000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:47:02.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.410+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34680d2820 con 0x7f34781049b0 2024-09-21T02:47:02.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:02.597+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3478063800 con 0x7f34640b6560 2024-09-21T02:47:02.691 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4[46041]: 2024-09-21T02:47:02.189+0000 7fb148282640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T02:47:02.691 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4[46041]: 2024-09-21T02:47:02.195+0000 7fb13f863640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:47:02.960 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:02.960 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi165 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi165 to 8544M 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:02.961 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='osd.4 ' entity='osd.4' 2024-09-21T02:47:02.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:02.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:02.962 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:47:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T02:47:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi165 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi165 to 8544M 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='osd.4 ' entity='osd.4' 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:03.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:47:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T02:47:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi165 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi165 to 8544M 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='osd.4 ' entity='osd.4' 2024-09-21T02:47:03.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:03.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:03.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] boot 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T02:47:03.966 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:03 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] boot 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T02:47:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:03 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: osd.4 [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] boot 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T02:47:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:03 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:47:04.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:04 smithi178 ceph-mon[34759]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:05.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:04 smithi031 ceph-mon[31209]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:04 smithi165 ceph-mon[34818]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/3364690611' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"} : dispatch 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"} : dispatch 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"}]': finished 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:05 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/2036047436' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/3364690611' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"} : dispatch 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"} : dispatch 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"}]': finished 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:05 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/2036047436' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:06.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T02:47:06.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/3364690611' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"} : dispatch 2024-09-21T02:47:06.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"} : dispatch 2024-09-21T02:47:06.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d68d7120-069f-428b-823a-5fbd7f74f03b"}]': finished 2024-09-21T02:47:06.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T02:47:06.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:06.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:05 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/2036047436' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:06 smithi031 ceph-mon[31209]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:06 smithi165 ceph-mon[34818]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:06 smithi178 ceph-mon[34759]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:09.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:08 smithi031 ceph-mon[31209]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:08 smithi165 ceph-mon[34818]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:08 smithi178 ceph-mon[34759]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:10.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:10 smithi178 ceph-mon[34759]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:11.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:10 smithi031 ceph-mon[31209]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:10 smithi165 ceph-mon[34818]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:12.879 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:12 smithi178 ceph-mon[34759]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:12 smithi031 ceph-mon[31209]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:12 smithi165 ceph-mon[34818]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:14.062 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T02:47:14.062 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T02:47:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T02:47:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:14 smithi031 ceph-mon[31209]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:14 smithi031 ceph-mon[31209]: Deploying daemon osd.5 on smithi178 2024-09-21T02:47:15.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:14 smithi178 ceph-mon[34759]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:15.105 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:14 smithi178 ceph-mon[34759]: Deploying daemon osd.5 on smithi178 2024-09-21T02:47:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:14 smithi165 ceph-mon[34818]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:14 smithi165 ceph-mon[34818]: Deploying daemon osd.5 on smithi178 2024-09-21T02:47:16.934 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:16 smithi178 ceph-mon[34759]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:16 smithi031 ceph-mon[31209]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:16 smithi165 ceph-mon[34818]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:18.933 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:18 smithi178 ceph-mon[34759]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:18 smithi031 ceph-mon[31209]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:18 smithi165 ceph-mon[34818]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:20 smithi178 ceph-mon[34759]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:20 smithi031 ceph-mon[31209]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:20 smithi165 ceph-mon[34818]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:22 smithi031 ceph-mon[31209]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:23.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:23.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:22 smithi165 ceph-mon[34818]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:23.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:23.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:22 smithi178 ceph-mon[34759]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:23.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:23.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:23.223 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:23.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.937+0000 7f34757fa640 1 -- 172.21.15.178:0/2529200542 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3478063800 con 0x7f34640b6560 2024-09-21T02:47:23.938 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-09-21T02:47:23.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34640b6560 msgr2=0x7f34640b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:23.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34640b6560 0x7f34640b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3478102470 tx=0x7f346000b410 comp rx=0 tx=0).stop 2024-09-21T02:47:23.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 msgr2=0x7f347819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f347819a520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3468032fd0 tx=0x7f3468004270 comp rx=0 tx=0).stop 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 shutdown_connections 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34640b6560 0x7f34640b8a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f347810de10 0x7f347819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3478105300 0x7f347819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 --2- 172.21.15.178:0/2529200542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34781049b0 0x7f347819a520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 >> 172.21.15.178:0/2529200542 conn(0x7f3478079730 msgr2=0x7f34781008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 shutdown_connections 2024-09-21T02:47:23.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:47:23.939+0000 7f347e1fa640 1 -- 172.21.15.178:0/2529200542 wait complete. 2024-09-21T02:47:24.578 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T02:47:24.581 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-09-21T02:47:24.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T02:47:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:47:24.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:24 smithi031 ceph-mon[31209]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:24.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:24.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:24.976 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:25.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:24 smithi178 ceph-mon[34759]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:25.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:25.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:25.095 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:24 smithi165 ceph-mon[34818]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:25.478 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 02:47:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T02:47:25.135+0000 7f01f5f7e640 -1 osd.5 0 log_to_monitors true 2024-09-21T02:47:25.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:25 smithi178 ceph-mon[34759]: from='osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:47:25.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:25 smithi178 ceph-mon[34759]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:47:26.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:25 smithi031 ceph-mon[31209]: from='osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:47:26.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:25 smithi031 ceph-mon[31209]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:47:26.092 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:25 smithi165 ceph-mon[34818]: from='osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:47:26.092 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:25 smithi165 ceph-mon[34818]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:47:27.080 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T02:47:26.827+0000 7f01f1f00640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T02:47:27.080 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T02:47:26.836+0000 7f01edcfe640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:47:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-mon[34759]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-mon[34759]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T02:47:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-mon[34759]: from='osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:47:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-mon[34759]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T02:47:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:26 smithi178 ceph-mon[34759]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:47:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:26 smithi031 ceph-mon[31209]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:26 smithi031 ceph-mon[31209]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T02:47:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:26 smithi031 ceph-mon[31209]: from='osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:47:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:26 smithi031 ceph-mon[31209]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T02:47:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:26 smithi031 ceph-mon[31209]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:47:27.105 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:26 smithi165 ceph-mon[34818]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:27.105 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:26 smithi165 ceph-mon[34818]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T02:47:27.105 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:26 smithi165 ceph-mon[34818]: from='osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:47:27.105 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:26 smithi165 ceph-mon[34818]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T02:47:27.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:27.106 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:26 smithi165 ceph-mon[34818]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:47:27.597 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:47:27.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-09-21T02:47:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:27.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:27.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:27.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:27.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:27 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:28.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:28.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:28.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:27 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:28.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:28.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:27 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 -- 172.21.15.31:0/624511275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 msgr2=0x7fdd9c10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 --2- 172.21.15.31:0/624511275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c10d8b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009900 tx=0x7fdd9002a410 comp rx=0 tx=0).stop 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 -- 172.21.15.31:0/624511275 shutdown_connections 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 --2- 172.21.15.31:0/624511275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c10ddf0 0x7fdd9c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 --2- 172.21.15.31:0/624511275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c10d8b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 --2- 172.21.15.31:0/624511275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c104840 0x7fdd9c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 -- 172.21.15.31:0/624511275 >> 172.21.15.31:0/624511275 conn(0x7fdd9c079730 msgr2=0x7fdd9c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:47:28.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 -- 172.21.15.31:0/624511275 shutdown_connections 2024-09-21T02:47:28.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.386+0000 7fdda206b640 1 -- 172.21.15.31:0/624511275 wait complete. 2024-09-21T02:47:28.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 Processor -- start 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 -- start start 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c104840 0x7fdd9c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c10ddf0 0x7fdd9c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113a70 con 0x7fdd9c105270 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c1138f0 con 0x7fdd9c10ddf0 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.387+0000 7fdda206b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113bf0 con 0x7fdd9c104840 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:28.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37784/0 (socket says 172.21.15.31:37784) 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 -- 172.21.15.31:0/1823530276 learned_addr learned my addr 172.21.15.31:0/1823530276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9b7fe640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c104840 0x7fdd9c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9bfff640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c10ddf0 0x7fdd9c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 -- 172.21.15.31:0/1823530276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c104840 msgr2=0x7fdd9c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c104840 0x7fdd9c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 -- 172.21.15.31:0/1823530276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c10ddf0 msgr2=0x7fdd9c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c10ddf0 0x7fdd9c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 -- 172.21.15.31:0/1823530276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c1a03d0 con 0x7fdd9c105270 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9b7fe640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c104840 0x7fdd9c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9bfff640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c10ddf0 0x7fdd9c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:47:28.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd9affd640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c19aa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd90006010 tx=0x7fdd90004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9003f030 con 0x7fdd9c105270 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd90003aa0 con 0x7fdd9c105270 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c1a0660 con 0x7fdd9c105270 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90035070 con 0x7fdd9c105270 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.388+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a6e20 con 0x7fdd9c105270 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.390+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c06b500 con 0x7fdd9c105270 2024-09-21T02:47:28.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.392+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fdd90042030 con 0x7fdd9c105270 2024-09-21T02:47:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.393+0000 7fdd98ff9640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdd840b6560 0x7fdd840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:28.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.393+0000 7fdd9b7fe640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdd840b6560 0x7fdd840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:28.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.393+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4781+0+0 (secure 0 0 0) 0x7fdd90106ed0 con 0x7fdd9c105270 2024-09-21T02:47:28.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.394+0000 7fdd9b7fe640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdd840b6560 0x7fdd840b8a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c06bc00 tx=0x7fdd8c009670 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:47:28.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.396+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd900ce490 con 0x7fdd9c105270 2024-09-21T02:47:28.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:28.595+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdd9c063800 con 0x7fdd840b6560 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi178 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] boot 2024-09-21T02:47:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T02:47:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi178 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] boot 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:28.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:28.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:47:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:47:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi178 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: osd.5 [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] boot 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:29.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:30.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:29 smithi031 ceph-mon[31209]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:30.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:29 smithi031 ceph-mon[31209]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T02:47:30.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:29 smithi165 ceph-mon[34818]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:30.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:29 smithi165 ceph-mon[34818]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T02:47:30.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:29 smithi178 ceph-mon[34759]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:30.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:29 smithi178 ceph-mon[34759]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T02:47:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:30 smithi031 ceph-mon[31209]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:30 smithi031 ceph-mon[31209]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T02:47:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:30 smithi165 ceph-mon[34818]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:30 smithi165 ceph-mon[34818]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T02:47:31.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:30 smithi178 ceph-mon[34759]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:31.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:30 smithi178 ceph-mon[34759]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T02:47:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3606570577' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a7e5212-2bd5-4861-8192-1ab5dcb592c0"} : dispatch 2024-09-21T02:47:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3606570577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7e5212-2bd5-4861-8192-1ab5dcb592c0"}]': finished 2024-09-21T02:47:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:31 smithi031 ceph-mon[31209]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T02:47:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/87302764' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3606570577' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a7e5212-2bd5-4861-8192-1ab5dcb592c0"} : dispatch 2024-09-21T02:47:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3606570577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7e5212-2bd5-4861-8192-1ab5dcb592c0"}]': finished 2024-09-21T02:47:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:31 smithi165 ceph-mon[34818]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T02:47:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/87302764' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3606570577' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3a7e5212-2bd5-4861-8192-1ab5dcb592c0"} : dispatch 2024-09-21T02:47:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3606570577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7e5212-2bd5-4861-8192-1ab5dcb592c0"}]': finished 2024-09-21T02:47:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:31 smithi178 ceph-mon[34759]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T02:47:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/87302764' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:32 smithi165 ceph-mon[34818]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:33.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:32 smithi178 ceph-mon[34759]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:33.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:32 smithi031 ceph-mon[31209]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:35.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:34 smithi031 ceph-mon[31209]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:34 smithi165 ceph-mon[34818]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:34 smithi178 ceph-mon[34759]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:36 smithi031 ceph-mon[31209]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-21T02:47:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:36 smithi165 ceph-mon[34818]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-21T02:47:37.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:36 smithi178 ceph-mon[34759]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-21T02:47:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:38 smithi031 ceph-mon[31209]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:47:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:38 smithi165 ceph-mon[34818]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:47:39.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:38 smithi178 ceph-mon[34759]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:47:40.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:39 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T02:47:40.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:39 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:40.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:39 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T02:47:40.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:39 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:40.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:39 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T02:47:40.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:39 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:41.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:40 smithi031 ceph-mon[31209]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T02:47:41.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:40 smithi031 ceph-mon[31209]: Deploying daemon osd.6 on smithi031 2024-09-21T02:47:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:40 smithi165 ceph-mon[34818]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T02:47:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:40 smithi165 ceph-mon[34818]: Deploying daemon osd.6 on smithi031 2024-09-21T02:47:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:40 smithi178 ceph-mon[34759]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T02:47:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:40 smithi178 ceph-mon[34759]: Deploying daemon osd.6 on smithi031 2024-09-21T02:47:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:42 smithi031 ceph-mon[31209]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:47:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:42 smithi165 ceph-mon[34818]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:47:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:42 smithi178 ceph-mon[34759]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:47:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:44 smithi031 ceph-mon[31209]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:47:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:44 smithi165 ceph-mon[34818]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:47:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:44 smithi178 ceph-mon[34759]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:47:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:46 smithi031 ceph-mon[31209]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:47:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:47.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:46 smithi165 ceph-mon[34818]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:47:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:46 smithi178 ceph-mon[34759]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:47:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:48 smithi165 ceph-mon[34818]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:49.212 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:48 smithi031 ceph-mon[31209]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:49.212 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:49.212 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:49.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:48 smithi178 ceph-mon[34759]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:50.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.039+0000 7fdd98ff9640 1 -- 172.21.15.31:0/1823530276 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdd9c063800 con 0x7fdd840b6560 2024-09-21T02:47:50.041 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.041+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdd840b6560 msgr2=0x7fdd840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.041+0000 7fdda206b640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdd840b6560 0x7fdd840b8a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c06bc00 tx=0x7fdd8c009670 comp rx=0 tx=0).stop 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.041+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 msgr2=0x7fdd9c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.041+0000 7fdda206b640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c19aa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd90006010 tx=0x7fdd90004060 comp rx=0 tx=0).stop 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 shutdown_connections 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdd840b6560 0x7fdd840b8a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd9c10ddf0 0x7fdd9c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:50.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd9c105270 0x7fdd9c19aa00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 --2- 172.21.15.31:0/1823530276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd9c104840 0x7fdd9c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 >> 172.21.15.31:0/1823530276 conn(0x7fdd9c079730 msgr2=0x7fdd9c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:47:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 shutdown_connections 2024-09-21T02:47:50.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:47:50.042+0000 7fdda206b640 1 -- 172.21.15.31:0/1823530276 wait complete. 2024-09-21T02:47:50.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:49 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:50.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:49 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:50.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:49 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:50.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:49 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:50.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:49 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:50.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:49 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:49 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:49 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:49 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:50.792 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T02:47:50.794 INFO:tasks.cephadm:Deploying osd.7 on smithi165 with /dev/vg_nvme/lv_2... 2024-09-21T02:47:50.794 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T02:47:51.050 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:47:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:50 smithi031 ceph-mon[31209]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:50 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:50 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:50 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:51.171 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:50 smithi165 ceph-mon[34818]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:51.171 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:50 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:51.171 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:50 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:51.171 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:50 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:50 smithi178 ceph-mon[34759]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:50 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:47:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:50 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:50 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:52.155 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:51 smithi031 ceph-mon[31209]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:52.155 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 02:47:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T02:47:52.140+0000 7fb2d4fb7640 -1 osd.6 0 log_to_monitors true 2024-09-21T02:47:52.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:51 smithi165 ceph-mon[34818]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:52.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:51 smithi178 ceph-mon[34759]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:53.126 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:52 smithi165 ceph-mon[34818]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T02:47:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:52 smithi178 ceph-mon[34759]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T02:47:53.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:52 smithi031 ceph-mon[31209]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T02:47:53.728 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:47:53.753 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-09-21T02:47:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi031 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-21T02:47:54.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:54.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:54.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:53 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi031 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.136 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.137 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.137 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-21T02:47:54.137 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.137 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:54.137 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:54.137 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:53 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T02:47:54.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T02:47:54.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:47:54.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi031 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:47:54.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:53 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 -- 172.21.15.165:0/4028358793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc102ae0 msgr2=0x7f3cfc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 --2- 172.21.15.165:0/4028358793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc102ef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009900 tx=0x7f3cec02a410 comp rx=0 tx=0).stop 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 -- 172.21.15.165:0/4028358793 shutdown_connections 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 --2- 172.21.15.165:0/4028358793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 --2- 172.21.15.165:0/4028358793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc108d20 0x7f3cfc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 --2- 172.21.15.165:0/4028358793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc102ef0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 -- 172.21.15.165:0/4028358793 >> 172.21.15.165:0/4028358793 conn(0x7f3cfc079760 msgr2=0x7f3cfc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 -- 172.21.15.165:0/4028358793 shutdown_connections 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.521+0000 7f3d0085d640 1 -- 172.21.15.165:0/4028358793 wait complete. 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 Processor -- start 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 -- start start 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc108d20 0x7f3cfc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc113ab0 con 0x7f3cfc102ae0 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc113930 con 0x7f3cfc1083d0 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3d0085d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc113c30 con 0x7f3cfc108d20 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3cf9d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3cf9d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41604/0 (socket says 172.21.15.165:41604) 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3cf9d9b640 1 -- 172.21.15.165:0/886315694 learned_addr learned my addr 172.21.15.165:0/886315694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.522+0000 7f3cfa59c640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cfad9d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc108d20 0x7f3cfc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cf9d9b640 1 -- 172.21.15.165:0/886315694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc108d20 msgr2=0x7f3cfc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cf9d9b640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc108d20 0x7f3cfc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cf9d9b640 1 -- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc102ae0 msgr2=0x7f3cfc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cf9d9b640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cf9d9b640 1 -- 172.21.15.165:0/886315694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc1a0430 con 0x7f3cfc1083d0 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cfad9d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc108d20 0x7f3cfc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cfa59c640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3cf9d9b640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc19aa60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000d740 tx=0x7f3cf000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:47:54.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0017070 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf000de50 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc1a0720 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0004ce0 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.523+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc1a6e80 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.524+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cfc103d10 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.526+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3cf0004e80 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.527+0000 7f3ceb7fe640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3ce40b6560 0x7f3ce40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.527+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4900+0+0 (secure 0 0 0) 0x7f3cf00e63b0 con 0x7f3cfc1083d0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.527+0000 7f3cfa59c640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3ce40b6560 0x7f3ce40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.527+0000 7f3cfa59c640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3ce40b6560 0x7f3ce40b8a70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3cec031440 tx=0x7f3cec032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:47:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.530+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3cf00ad990 con 0x7f3cfc1083d0 2024-09-21T02:47:54.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:47:54.715+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3cfc063800 con 0x7f3ce40b6560 2024-09-21T02:47:54.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T02:47:54.383+0000 7fb2d0f39640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T02:47:54.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T02:47:54.392+0000 7fb2c8d2f640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:47:55.093 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:47:55.093 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T02:47:55.093 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:55.093 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:55.093 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' 2024-09-21T02:47:55.093 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:55.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:55.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:55.094 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:54 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:55.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:54 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:55.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:47:55.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T02:47:55.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:55.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:55.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273]' entity='osd.6' 2024-09-21T02:47:55.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:47:55.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:47:55.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:47:55.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:54 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:47:56.052 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:47:56.052 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:47:56.052 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:56.052 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:56.052 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] boot 2024-09-21T02:47:56.052 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T02:47:56.053 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:55 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] boot 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T02:47:56.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:55 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: osd.6 [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] boot 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T02:47:56.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:55 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/2725317975' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"} : dispatch 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"} : dispatch 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"}]': finished 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:47:58.194 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:47:57 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/1669868148' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/2725317975' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"} : dispatch 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"} : dispatch 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"}]': finished 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:47:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:47:57 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/1669868148' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/2725317975' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"} : dispatch 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"} : dispatch 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a58534f6-f366-4d71-a057-272ba5874b0d"}]': finished 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:47:58.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:47:58.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:47:57 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/1669868148' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:00.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:00 smithi165 ceph-mon[34818]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:00.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:00 smithi178 ceph-mon[34759]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:00.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:00 smithi031 ceph-mon[31209]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:02.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:02 smithi165 ceph-mon[34818]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:02.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:02 smithi178 ceph-mon[34759]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:02 smithi031 ceph-mon[31209]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:04.387 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:04 smithi165 ceph-mon[34818]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:04.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:04 smithi178 ceph-mon[34759]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:04.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:04 smithi031 ceph-mon[31209]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:06 smithi165 ceph-mon[34818]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T02:48:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:06 smithi165 ceph-mon[34818]: Deploying daemon osd.7 on smithi165 2024-09-21T02:48:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:06 smithi178 ceph-mon[34759]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T02:48:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:06 smithi178 ceph-mon[34759]: Deploying daemon osd.7 on smithi165 2024-09-21T02:48:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:06 smithi031 ceph-mon[31209]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T02:48:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:06 smithi031 ceph-mon[31209]: Deploying daemon osd.7 on smithi165 2024-09-21T02:48:08.379 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:08 smithi165 ceph-mon[34818]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:08.427 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:08 smithi178 ceph-mon[34759]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:08.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:08 smithi031 ceph-mon[31209]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:10.291 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:10 smithi165 ceph-mon[34818]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:10 smithi178 ceph-mon[34759]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:10.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:10 smithi031 ceph-mon[31209]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:12.246 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:12 smithi165 ceph-mon[34818]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:12.246 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:12.246 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:12.246 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:12 smithi178 ceph-mon[34759]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:12.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:12 smithi031 ceph-mon[31209]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:12.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:12.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:12.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:14 smithi165 ceph-mon[34818]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:14.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:14 smithi178 ceph-mon[34759]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:14.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:14 smithi031 ceph-mon[31209]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.538 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 7 on host 'smithi165' 2024-09-21T02:48:15.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.536+0000 7f3ceb7fe640 1 -- 172.21.15.165:0/886315694 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3cfc063800 con 0x7f3ce40b6560 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3ce40b6560 msgr2=0x7f3ce40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3ce40b6560 0x7f3ce40b8a70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3cec031440 tx=0x7f3cec032ed0 comp rx=0 tx=0).stop 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 msgr2=0x7f3cfc19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc19aa60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000d740 tx=0x7f3cf000dc00 comp rx=0 tx=0).stop 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 shutdown_connections 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3ce40b6560 0x7f3ce40b8a70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cfc108d20 0x7f3cfc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.538+0000 7f3d0085d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3cfc1083d0 0x7f3cfc19aa60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.539+0000 7f3d0085d640 1 --2- 172.21.15.165:0/886315694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cfc102ae0 0x7f3cfc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:15.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.539+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 >> 172.21.15.165:0/886315694 conn(0x7f3cfc079760 msgr2=0x7f3cfc101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:48:15.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.539+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 shutdown_connections 2024-09-21T02:48:15.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:48:15.539+0000 7f3d0085d640 1 -- 172.21.15.165:0/886315694 wait complete. 2024-09-21T02:48:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:16.192 DEBUG:teuthology.orchestra.run.smithi165:osd.7> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T02:48:16.194 INFO:tasks.cephadm:Deploying osd.8 on smithi178 with /dev/vg_nvme/lv_2... 2024-09-21T02:48:16.194 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T02:48:16.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:16 smithi165 ceph-mon[34818]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:16.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:16.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:16.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:16.452 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:48:16.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:16 smithi178 ceph-mon[34759]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:16.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:16.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:16.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:16 smithi031 ceph-mon[31209]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:17.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:17 smithi031 ceph-mon[31209]: from='osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:48:17.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:17 smithi031 ceph-mon[31209]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:48:17.630 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:17 smithi165 ceph-mon[34818]: from='osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:48:17.630 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:17 smithi165 ceph-mon[34818]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:48:17.631 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 02:48:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T02:48:17.361+0000 7fcdab7e9640 -1 osd.7 0 log_to_monitors true 2024-09-21T02:48:17.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:17 smithi178 ceph-mon[34759]: from='osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:48:17.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:17 smithi178 ceph-mon[34759]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:48:18.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:18 smithi178 ceph-mon[34759]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:18.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:18 smithi178 ceph-mon[34759]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:48:18.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:18 smithi178 ceph-mon[34759]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:48:18.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:18 smithi178 ceph-mon[34759]: from='osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:48:18.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:18.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:18 smithi178 ceph-mon[34759]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:48:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-mon[34818]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-mon[34818]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:48:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-mon[34818]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:48:18.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-mon[34818]: from='osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:48:18.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:18.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-mon[34818]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:48:18.692 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T02:48:18.383+0000 7fcda776b640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T02:48:18.692 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 02:48:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T02:48:18.395+0000 7fcda3569640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:48:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:18 smithi031 ceph-mon[31209]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:18 smithi031 ceph-mon[31209]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:48:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:18 smithi031 ceph-mon[31209]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:48:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:18 smithi031 ceph-mon[31209]: from='osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:48:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:18 smithi031 ceph-mon[31209]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:48:18.988 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:48:19.022 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-09-21T02:48:19.285 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.680 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.681 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.681 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.681 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:19.681 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:19.681 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:19.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:19.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 -- 172.21.15.178:0/3436977662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa400103b60 msgr2=0x7fa400103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:19.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 --2- 172.21.15.178:0/3436977662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa400103b60 0x7fa400103f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009860 tx=0x7fa3f002a340 comp rx=0 tx=0).stop 2024-09-21T02:48:19.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 -- 172.21.15.178:0/3436977662 shutdown_connections 2024-09-21T02:48:19.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 --2- 172.21.15.178:0/3436977662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa400110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:19.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 --2- 172.21.15.178:0/3436977662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4001094a0 0x7fa4001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:19.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 --2- 172.21.15.178:0/3436977662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa400103b60 0x7fa400103f70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:19.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 -- 172.21.15.178:0/3436977662 >> 172.21.15.178:0/3436977662 conn(0x7fa40007b850 msgr2=0x7fa40007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:48:19.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 -- 172.21.15.178:0/3436977662 shutdown_connections 2024-09-21T02:48:19.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.771+0000 7fa405070640 1 -- 172.21.15.178:0/3436977662 wait complete. 2024-09-21T02:48:19.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.772+0000 7fa405070640 1 Processor -- start 2024-09-21T02:48:19.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 -- start start 2024-09-21T02:48:19.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa400103b60 0x7fa40019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:19.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4001094a0 0x7fa40019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:19.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa40019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa400113ab0 con 0x7fa4001094a0 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa400113930 con 0x7fa400103b60 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa405070640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa400113c30 con 0x7fa400109df0 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa40019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa40019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54658/0 (socket says 172.21.15.178:54658) 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 -- 172.21.15.178:0/3076860000 learned_addr learned my addr 172.21.15.178:0/3076860000 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3fe59c640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4001094a0 0x7fa40019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3fed9d640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa400103b60 0x7fa40019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 -- 172.21.15.178:0/3076860000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa400103b60 msgr2=0x7fa40019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa400103b60 0x7fa40019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 -- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4001094a0 msgr2=0x7fa40019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4001094a0 0x7fa40019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:19.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 -- 172.21.15.178:0/3076860000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4001a0470 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.773+0000 7fa3ff59e640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa40019fd40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800efc0 tx=0x7fa3f800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.774+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8003970 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.774+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f8004020 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.774+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4001a0760 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.774+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800b940 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.774+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4001a6ec0 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.775+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa400103f70 con 0x7fa400109df0 2024-09-21T02:48:19.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.776+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fa3f801d020 con 0x7fa400109df0 2024-09-21T02:48:19.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.777+0000 7fa3e7fff640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3e80b6560 0x7fa3e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:19.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.777+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5645+0+0 (secure 0 0 0) 0x7fa3f80e6770 con 0x7fa400109df0 2024-09-21T02:48:19.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.777+0000 7fa3fed9d640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3e80b6560 0x7fa3e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:19.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.778+0000 7fa3fed9d640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3e80b6560 0x7fa3e80b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009830 tx=0x7fa3f0030340 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:48:19.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.781+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3f80ada70 con 0x7fa400109df0 2024-09-21T02:48:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:48:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T02:48:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:19.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:19.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:19.967+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa40010bc30 con 0x7fa3e80b6560 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi165 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi165 to 5696M 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] boot 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:48:20.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:20.603 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:48:20.603 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:48:20.603 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:20.603 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:20 smithi178 ceph-mon[34759]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T02:48:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:48:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:48:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi165 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi165 to 5696M 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] boot 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:20.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:20 smithi165 ceph-mon[34818]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi165 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi165 to 5696M 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: osd.7 [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] boot 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:48:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:48:20.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:48:20.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:48:20.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:20.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:20 smithi031 ceph-mon[31209]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T02:48:21.645 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:21 smithi178 ceph-mon[34759]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:48:21.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:21 smithi165 ceph-mon[34818]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:48:21.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:21 smithi031 ceph-mon[31209]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:48:22.649 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:22 smithi178 ceph-mon[34759]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:22.649 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:22 smithi178 ceph-mon[34759]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T02:48:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:22 smithi165 ceph-mon[34818]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:22 smithi165 ceph-mon[34818]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T02:48:22.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:22 smithi031 ceph-mon[31209]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:22.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:22 smithi031 ceph-mon[31209]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T02:48:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:23 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/2514005338' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"} : dispatch 2024-09-21T02:48:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:23 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"} : dispatch 2024-09-21T02:48:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:23 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"}]': finished 2024-09-21T02:48:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:23 smithi165 ceph-mon[34818]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:48:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:23 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:23.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:23 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/263253529' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:23 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/2514005338' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"} : dispatch 2024-09-21T02:48:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:23 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"} : dispatch 2024-09-21T02:48:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:23 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"}]': finished 2024-09-21T02:48:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:23 smithi178 ceph-mon[34759]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:48:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:23 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:23.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:23 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/263253529' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:23 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/2514005338' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"} : dispatch 2024-09-21T02:48:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:23 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"} : dispatch 2024-09-21T02:48:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:23 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5831064f-dc15-4eab-8239-3648a77f4b4e"}]': finished 2024-09-21T02:48:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:23 smithi031 ceph-mon[31209]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:48:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:23 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:23.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:23 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/263253529' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:24.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:24 smithi165 ceph-mon[34818]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:24 smithi178 ceph-mon[34759]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:24 smithi031 ceph-mon[31209]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:26.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:26 smithi165 ceph-mon[34818]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 201 KiB/s, 0 objects/s recovering 2024-09-21T02:48:26.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:26 smithi178 ceph-mon[34759]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 201 KiB/s, 0 objects/s recovering 2024-09-21T02:48:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:26 smithi031 ceph-mon[31209]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 201 KiB/s, 0 objects/s recovering 2024-09-21T02:48:28.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:28 smithi165 ceph-mon[34818]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 171 KiB/s, 0 objects/s recovering 2024-09-21T02:48:28.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:28 smithi178 ceph-mon[34759]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 171 KiB/s, 0 objects/s recovering 2024-09-21T02:48:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:28 smithi031 ceph-mon[31209]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 171 KiB/s, 0 objects/s recovering 2024-09-21T02:48:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:30 smithi165 ceph-mon[34818]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T02:48:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:48:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:30 smithi178 ceph-mon[34759]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T02:48:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:48:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:30.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:30 smithi031 ceph-mon[31209]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-21T02:48:30.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:48:30.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:31.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:31 smithi165 ceph-mon[34818]: Deploying daemon osd.8 on smithi178 2024-09-21T02:48:31.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:31 smithi178 ceph-mon[34759]: Deploying daemon osd.8 on smithi178 2024-09-21T02:48:31.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:31 smithi031 ceph-mon[31209]: Deploying daemon osd.8 on smithi178 2024-09-21T02:48:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:32 smithi165 ceph-mon[34818]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-21T02:48:32.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:32 smithi178 ceph-mon[34759]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-21T02:48:32.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:32 smithi031 ceph-mon[31209]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-21T02:48:34.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:34 smithi165 ceph-mon[34818]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-21T02:48:34.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:34 smithi178 ceph-mon[34759]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-21T02:48:34.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:34 smithi031 ceph-mon[31209]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-21T02:48:36.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:36 smithi165 ceph-mon[34818]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:48:36.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:36.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:36.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:36 smithi178 ceph-mon[34759]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:48:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:36 smithi031 ceph-mon[31209]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:48:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:38.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:38 smithi165 ceph-mon[34818]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:38 smithi178 ceph-mon[34759]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:38 smithi031 ceph-mon[31209]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:40 smithi178 ceph-mon[34759]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.344+0000 7fa3e7fff640 1 -- 172.21.15.178:0/3076860000 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3f0004040 con 0x7fa3e80b6560 2024-09-21T02:48:40.345 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 8 on host 'smithi178' 2024-09-21T02:48:40.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3e80b6560 msgr2=0x7fa3e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:40.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3e80b6560 0x7fa3e80b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0009830 tx=0x7fa3f0030340 comp rx=0 tx=0).stop 2024-09-21T02:48:40.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 msgr2=0x7fa40019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa40019fd40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800efc0 tx=0x7fa3f800c800 comp rx=0 tx=0).stop 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 shutdown_connections 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3e80b6560 0x7fa3e80b8a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400109df0 0x7fa40019fd40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4001094a0 0x7fa40019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 --2- 172.21.15.178:0/3076860000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa400103b60 0x7fa40019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.346+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 >> 172.21.15.178:0/3076860000 conn(0x7fa40007b850 msgr2=0x7fa4001075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:48:40.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.347+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 shutdown_connections 2024-09-21T02:48:40.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:48:40.347+0000 7fa405070640 1 -- 172.21.15.178:0/3076860000 wait complete. 2024-09-21T02:48:40.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:40 smithi165 ceph-mon[34818]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:40.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:40.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:40.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:40 smithi031 ceph-mon[31209]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:40.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:41.031 DEBUG:teuthology.orchestra.run.smithi178:osd.8> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T02:48:41.033 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-09-21T02:48:41.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:48:41.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:41 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:41.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:41 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:41.255 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:41 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:41.306 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:48:41.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:41 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:41.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:41 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:41.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:41 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:41.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:41 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:48:41.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:41 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:41.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:41 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:42.285 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 02:48:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T02:48:42.002+0000 7fe9baa63640 -1 osd.8 0 log_to_monitors true 2024-09-21T02:48:42.543 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:42 smithi178 ceph-mon[34759]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:42.544 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:42 smithi178 ceph-mon[34759]: from='osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:48:42.544 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:42 smithi178 ceph-mon[34759]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:48:42.573 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:42 smithi031 ceph-mon[31209]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:42.573 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:42 smithi031 ceph-mon[31209]: from='osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:48:42.573 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:42 smithi031 ceph-mon[31209]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:48:42.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:42 smithi165 ceph-mon[34818]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:42.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:42 smithi165 ceph-mon[34818]: from='osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:48:42.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:42 smithi165 ceph-mon[34818]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:48:43.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-mon[34759]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:48:43.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-mon[34759]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:48:43.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-mon[34759]: from='osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:48:43.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:43.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-mon[34759]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:48:43.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-mon[34759]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:48:43.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:43 smithi031 ceph-mon[31209]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:48:43.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:43 smithi031 ceph-mon[31209]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:48:43.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:43 smithi031 ceph-mon[31209]: from='osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:48:43.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:43 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:43.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:43 smithi031 ceph-mon[31209]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:48:43.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:43 smithi031 ceph-mon[31209]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:48:43.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:43 smithi165 ceph-mon[34818]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:48:43.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:43 smithi165 ceph-mon[34818]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:48:43.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:43 smithi165 ceph-mon[34818]: from='osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:48:43.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:43 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:43.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:43 smithi165 ceph-mon[34818]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:48:43.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:43 smithi165 ceph-mon[34818]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:48:43.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T02:48:43.779+0000 7fe9b69e5640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T02:48:43.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 02:48:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T02:48:43.792+0000 7fe9ae7db640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:48:44.026 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:48:44.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-09-21T02:48:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:48:44.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:44.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T02:48:44.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='osd.8 ' entity='osd.8' 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:44.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T02:48:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='osd.8 ' entity='osd.8' 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:44.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:44.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='osd.8 ' entity='osd.8' 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:48:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:48:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.829+0000 7f4fb697b640 1 -- 172.21.15.31:0/1197907961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 msgr2=0x7f4fb0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 --2- 172.21.15.31:0/1197907961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb0110640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009be0 tx=0x7f4fa802a6c0 comp rx=0 tx=0).stop 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 -- 172.21.15.31:0/1197907961 shutdown_connections 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 --2- 172.21.15.31:0/1197907961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fb01094c0 0x7f4fb01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 --2- 172.21.15.31:0/1197907961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb0110640 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 --2- 172.21.15.31:0/1197907961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fb0103b80 0x7f4fb0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:44.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 -- 172.21.15.31:0/1197907961 >> 172.21.15.31:0/1197907961 conn(0x7f4fb00ff610 msgr2=0x7f4fb0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:48:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 -- 172.21.15.31:0/1197907961 shutdown_connections 2024-09-21T02:48:44.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 -- 172.21.15.31:0/1197907961 wait complete. 2024-09-21T02:48:44.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 Processor -- start 2024-09-21T02:48:44.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.830+0000 7f4fb697b640 1 -- start start 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb697b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fb0103b80 0x7f4fb019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb697b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fb01094c0 0x7f4fb019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb697b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb697b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0113ad0 con 0x7f4fb0109e10 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb697b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0113950 con 0x7f4fb01094c0 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb697b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0113c50 con 0x7f4fb0103b80 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb01a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57458/0 (socket says 172.21.15.31:57458) 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 -- 172.21.15.31:0/3263620578 learned_addr learned my addr 172.21.15.31:0/3263620578 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4faffff640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fb0103b80 0x7f4fb019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fa7fff640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fb01094c0 0x7f4fb019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 -- 172.21.15.31:0/3263620578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fb0103b80 msgr2=0x7f4fb019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:44.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fb0103b80 0x7f4fb019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 -- 172.21.15.31:0/3263620578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fb01094c0 msgr2=0x7f4fb019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fb01094c0 0x7f4fb019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 -- 172.21.15.31:0/3263620578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb01a25d0 con 0x7f4fb0109e10 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.831+0000 7f4fb4f18640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb01a1ea0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8004af0 tx=0x7f4fa8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.832+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8041020 con 0x7f4fb0109e10 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.832+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa8003e10 con 0x7f4fb0109e10 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.832+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8036040 con 0x7f4fb0109e10 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.832+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb01a2860 con 0x7f4fb0109e10 2024-09-21T02:48:44.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.832+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb01a9020 con 0x7f4fb0109e10 2024-09-21T02:48:44.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.833+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb0104e00 con 0x7f4fb0109e10 2024-09-21T02:48:44.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.834+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4fa8044020 con 0x7f4fb0109e10 2024-09-21T02:48:44.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.835+0000 7f4fadffb640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f9c0b6560 0x7f4f9c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:48:44.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.835+0000 7f4faffff640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f9c0b6560 0x7f4f9c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:48:44.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.835+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6077+0+0 (secure 0 0 0) 0x7f4fa81079b0 con 0x7f4fb0109e10 2024-09-21T02:48:44.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.836+0000 7f4faffff640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f9c0b6560 0x7f4f9c0b8a70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009bd0 tx=0x7f4fa0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:48:44.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:44.838+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4fa804b020 con 0x7f4fb0109e10 2024-09-21T02:48:45.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:48:45.038+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4fb010f910 con 0x7f4f9c0b6560 2024-09-21T02:48:45.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:48:45.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:48:45.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi178 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] boot 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:48:45.671 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:45 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi178 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] boot 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:48:45.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:45 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi178 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: osd.8 [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] boot 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:48:45.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:48:45.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:45 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:46.667 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:46 smithi031 ceph-mon[31209]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:48:46.667 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:46 smithi031 ceph-mon[31209]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:46.667 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:46 smithi031 ceph-mon[31209]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T02:48:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:46 smithi165 ceph-mon[34818]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:48:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:46 smithi165 ceph-mon[34818]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:46 smithi165 ceph-mon[34818]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T02:48:46.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:46 smithi178 ceph-mon[34759]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:48:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:46 smithi178 ceph-mon[34759]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:48:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:46 smithi178 ceph-mon[34759]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T02:48:47.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:47 smithi031 ceph-mon[31209]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T02:48:47.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:47 smithi165 ceph-mon[34818]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T02:48:47.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:47 smithi178 ceph-mon[34759]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T02:48:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:48 smithi165 ceph-mon[34818]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:48:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1831518076' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2d67c10d-2a66-4175-8ec4-f6178369fff1"} : dispatch 2024-09-21T02:48:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1831518076' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d67c10d-2a66-4175-8ec4-f6178369fff1"}]': finished 2024-09-21T02:48:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:48 smithi165 ceph-mon[34818]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:48:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:48 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:48:48.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/993200361' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:48 smithi178 ceph-mon[34759]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:48:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1831518076' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2d67c10d-2a66-4175-8ec4-f6178369fff1"} : dispatch 2024-09-21T02:48:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1831518076' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d67c10d-2a66-4175-8ec4-f6178369fff1"}]': finished 2024-09-21T02:48:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:48 smithi178 ceph-mon[34759]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:48:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:48 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:48:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/993200361' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:48 smithi031 ceph-mon[31209]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:48:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1831518076' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2d67c10d-2a66-4175-8ec4-f6178369fff1"} : dispatch 2024-09-21T02:48:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1831518076' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d67c10d-2a66-4175-8ec4-f6178369fff1"}]': finished 2024-09-21T02:48:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:48 smithi031 ceph-mon[31209]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:48:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:48 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:48:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/993200361' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:48:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:50 smithi165 ceph-mon[34818]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:48:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:50 smithi178 ceph-mon[34759]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:48:50.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:50 smithi031 ceph-mon[31209]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:48:52.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:52 smithi031 ceph-mon[31209]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:48:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:52 smithi165 ceph-mon[34818]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:48:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:52 smithi178 ceph-mon[34759]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:48:54.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:54 smithi165 ceph-mon[34818]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T02:48:54.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:54 smithi178 ceph-mon[34759]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T02:48:54.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:54 smithi031 ceph-mon[31209]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-21T02:48:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:56 smithi031 ceph-mon[31209]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T02:48:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:48:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:56 smithi031 ceph-mon[31209]: Deploying daemon osd.9 on smithi031 2024-09-21T02:48:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:56 smithi165 ceph-mon[34818]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T02:48:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:48:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:56 smithi165 ceph-mon[34818]: Deploying daemon osd.9 on smithi031 2024-09-21T02:48:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:56 smithi178 ceph-mon[34759]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-21T02:48:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:48:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:48:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:56 smithi178 ceph-mon[34759]: Deploying daemon osd.9 on smithi031 2024-09-21T02:48:58.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:48:58 smithi031 ceph-mon[31209]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:48:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:48:58 smithi165 ceph-mon[34818]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:48:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:48:58 smithi178 ceph-mon[34759]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:49:00.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:00 smithi165 ceph-mon[34818]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:49:00.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:00 smithi031 ceph-mon[31209]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:49:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:00 smithi178 ceph-mon[34759]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:49:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:02 smithi165 ceph-mon[34818]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:49:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:02 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:02.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:02 smithi178 ceph-mon[34759]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:49:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:02 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:02.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:02 smithi031 ceph-mon[31209]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:49:02.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:02.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:02.739 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:02 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:04.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:04 smithi031 ceph-mon[31209]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:04.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:04 smithi165 ceph-mon[34818]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:04 smithi178 ceph-mon[34759]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:05.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.919+0000 7f4fadffb640 1 -- 172.21.15.31:0/3263620578 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4fb010f910 con 0x7f4f9c0b6560 2024-09-21T02:49:05.921 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-09-21T02:49:05.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f9c0b6560 msgr2=0x7f4f9c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:05.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f9c0b6560 0x7f4f9c0b8a70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009bd0 tx=0x7f4fa0005f90 comp rx=0 tx=0).stop 2024-09-21T02:49:05.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 msgr2=0x7f4fb01a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:05.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb01a1ea0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8004af0 tx=0x7f4fa8003c20 comp rx=0 tx=0).stop 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 shutdown_connections 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f9c0b6560 0x7f4f9c0b8a70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fb0109e10 0x7f4fb01a1ea0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fb01094c0 0x7f4fb019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 --2- 172.21.15.31:0/3263620578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fb0103b80 0x7f4fb019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 >> 172.21.15.31:0/3263620578 conn(0x7f4fb00ff610 msgr2=0x7f4fb0108ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 shutdown_connections 2024-09-21T02:49:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:05.921+0000 7f4fb697b640 1 -- 172.21.15.31:0/3263620578 wait complete. 2024-09-21T02:49:06.283 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.283 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.283 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:06.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:06.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:06.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:06.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.284 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:06.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:06.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:06.711 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T02:49:06.713 INFO:tasks.cephadm:Deploying osd.10 on smithi165 with /dev/vg_nvme/lv_1... 2024-09-21T02:49:06.713 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:49:06.971 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:49:08.176 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 02:49:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T02:49:08.128+0000 7fb10cd58640 -1 osd.9 0 log_to_monitors true 2024-09-21T02:49:08.426 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:08 smithi031 ceph-mon[31209]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:08.426 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:08 smithi031 ceph-mon[31209]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:49:08.435 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:08 smithi165 ceph-mon[34818]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:08.435 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:08 smithi165 ceph-mon[34818]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:49:08.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:08 smithi178 ceph-mon[34759]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:08.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:08 smithi178 ceph-mon[34759]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T02:49:09.360 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T02:49:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T02:49:09.501 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:09.821 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-21T02:49:09.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T02:49:09.640+0000 7fb108cda640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T02:49:09.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 02:49:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T02:49:09.652+0000 7fb100ad0640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:49:09.847 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-09-21T02:49:10.107 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:49:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' 2024-09-21T02:49:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:10 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' 2024-09-21T02:49:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:10 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:10.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:10.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:10.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917]' entity='osd.9' 2024-09-21T02:49:10.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:10.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:10 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:10.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 -- 172.21.15.165:0/3653775996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 msgr2=0x7f76ec10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:10.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 --2- 172.21.15.165:0/3653775996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec10f540 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009ca0 tx=0x7f76dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:49:10.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 -- 172.21.15.165:0/3653775996 shutdown_connections 2024-09-21T02:49:10.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 --2- 172.21.15.165:0/3653775996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec10f540 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:10.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 --2- 172.21.15.165:0/3653775996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76ec1083c0 0x7f76ec1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:10.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 --2- 172.21.15.165:0/3653775996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76ec102ad0 0x7f76ec102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:10.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 -- 172.21.15.165:0/3653775996 >> 172.21.15.165:0/3653775996 conn(0x7f76ec0796d0 msgr2=0x7f76ec079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:10.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 -- 172.21.15.165:0/3653775996 shutdown_connections 2024-09-21T02:49:10.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.614+0000 7f76f347d640 1 -- 172.21.15.165:0/3653775996 wait complete. 2024-09-21T02:49:10.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 Processor -- start 2024-09-21T02:49:10.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 -- start start 2024-09-21T02:49:10.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76ec102ad0 0x7f76ec199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:10.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76ec1083c0 0x7f76ec199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:10.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:10.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1129d0 con 0x7f76ec102ad0 2024-09-21T02:49:10.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec112850 con 0x7f76ec108d10 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f347d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec112b50 con 0x7f76ec1083c0 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f1a1a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f1a1a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52252/0 (socket says 172.21.15.165:52252) 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.615+0000 7f76f1a1a640 1 -- 172.21.15.165:0/3476334869 learned_addr learned my addr 172.21.15.165:0/3476334869 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1219640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76ec102ad0 0x7f76ec199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f0a18640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76ec1083c0 0x7f76ec199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1a1a640 1 -- 172.21.15.165:0/3476334869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76ec1083c0 msgr2=0x7f76ec199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1a1a640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76ec1083c0 0x7f76ec199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1a1a640 1 -- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76ec102ad0 msgr2=0x7f76ec199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1a1a640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76ec102ad0 0x7f76ec199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1a1a640 1 -- 172.21.15.165:0/3476334869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec19f230 con 0x7f76ec108d10 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f1a1a640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec19ebc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f76dc005cc0 tx=0x7f76dc0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:49:10.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc04a070 con 0x7f76ec108d10 2024-09-21T02:49:10.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76dc003c80 con 0x7f76ec108d10 2024-09-21T02:49:10.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec19f4c0 con 0x7f76ec108d10 2024-09-21T02:49:10.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc02fa50 con 0x7f76ec108d10 2024-09-21T02:49:10.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.616+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ec19fa00 con 0x7f76ec108d10 2024-09-21T02:49:10.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.617+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ec103d00 con 0x7f76ec108d10 2024-09-21T02:49:10.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.619+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f76dc04d070 con 0x7f76ec108d10 2024-09-21T02:49:10.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.619+0000 7f76ea7fc640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76d40b6590 0x7f76d40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:10.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.620+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6196+0+0 (secure 0 0 0) 0x7f76dc109370 con 0x7f76ec108d10 2024-09-21T02:49:10.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.620+0000 7f76f1219640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76d40b6590 0x7f76d40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:10.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.620+0000 7f76f1219640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76d40b6590 0x7f76d40b8aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76e40058c0 tx=0x7f76e400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:49:10.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.623+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76dc084710 con 0x7f76ec108d10 2024-09-21T02:49:10.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:10.814+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f76ec10a650 con 0x7f76d40b6590 2024-09-21T02:49:11.928 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:49:11.928 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:49:11.928 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi031 2024-09-21T02:49:11.928 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-21T02:49:11.928 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] boot 2024-09-21T02:49:11.929 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T02:49:11.929 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:11.929 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:49:11.929 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:49:11.929 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:49:11.929 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi031 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] boot 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:49:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:49:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:49:11.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:49:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:49:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi031 2024-09-21T02:49:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-21T02:49:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: osd.9 [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] boot 2024-09-21T02:49:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T02:49:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:49:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:49:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:49:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:49:12.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:12.796 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:12 smithi165 ceph-mon[34818]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:12.796 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:12 smithi165 ceph-mon[34818]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T02:49:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:12 smithi178 ceph-mon[34759]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:12 smithi178 ceph-mon[34759]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T02:49:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:12 smithi031 ceph-mon[31209]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:49:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:12 smithi031 ceph-mon[31209]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T02:49:13.912 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:13 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/2290704535' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"} : dispatch 2024-09-21T02:49:13.912 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:13 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"} : dispatch 2024-09-21T02:49:13.913 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:13 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"}]': finished 2024-09-21T02:49:13.913 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:13 smithi165 ceph-mon[34818]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:49:13.913 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:13 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/2290704535' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"} : dispatch 2024-09-21T02:49:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:13 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"} : dispatch 2024-09-21T02:49:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:13 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"}]': finished 2024-09-21T02:49:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:13 smithi178 ceph-mon[34759]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:49:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:13 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/2290704535' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"} : dispatch 2024-09-21T02:49:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:13 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"} : dispatch 2024-09-21T02:49:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:13 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0ae22f5-028f-4723-9374-c88933908b49"}]': finished 2024-09-21T02:49:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:13 smithi031 ceph-mon[31209]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:49:14.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:14 smithi165 ceph-mon[34818]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:14 smithi165 ceph-mon[34818]: from='client.? 172.21.15.165:0/2635191735' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:49:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:14 smithi178 ceph-mon[34759]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:14 smithi178 ceph-mon[34759]: from='client.? 172.21.15.165:0/2635191735' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:49:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:14 smithi031 ceph-mon[31209]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:14 smithi031 ceph-mon[31209]: from='client.? 172.21.15.165:0/2635191735' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:49:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:16 smithi165 ceph-mon[34818]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:16 smithi178 ceph-mon[34759]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:16 smithi031 ceph-mon[31209]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:18.938 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:18 smithi165 ceph-mon[34818]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:18 smithi178 ceph-mon[34759]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:18 smithi031 ceph-mon[31209]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:20.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:20 smithi165 ceph-mon[34818]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:20.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:20 smithi178 ceph-mon[34759]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:20 smithi031 ceph-mon[31209]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:22 smithi165 ceph-mon[34818]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:49:22.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:22 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:22.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:22 smithi165 ceph-mon[34818]: Deploying daemon osd.10 on smithi165 2024-09-21T02:49:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:22 smithi178 ceph-mon[34759]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:49:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:22 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:22 smithi178 ceph-mon[34759]: Deploying daemon osd.10 on smithi165 2024-09-21T02:49:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:22 smithi031 ceph-mon[31209]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:49:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:22 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:22 smithi031 ceph-mon[31209]: Deploying daemon osd.10 on smithi165 2024-09-21T02:49:24.797 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:24 smithi165 ceph-mon[34818]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:24 smithi178 ceph-mon[34759]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:24 smithi031 ceph-mon[31209]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:26.764 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:26 smithi165 ceph-mon[34818]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:26 smithi178 ceph-mon[34759]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:26 smithi031 ceph-mon[31209]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:28.812 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:28 smithi165 ceph-mon[34818]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:28.812 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:28.812 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:28.812 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:28 smithi178 ceph-mon[34759]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:28 smithi031 ceph-mon[31209]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:30 smithi178 ceph-mon[34759]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:30 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.008 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:30 smithi165 ceph-mon[34818]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:31.008 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.008 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.008 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:31.008 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:31.009 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:30 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.052 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 10 on host 'smithi165' 2024-09-21T02:49:31.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.051+0000 7f76ea7fc640 1 -- 172.21.15.165:0/3476334869 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f76ec10a650 con 0x7f76d40b6590 2024-09-21T02:49:31.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76d40b6590 msgr2=0x7f76d40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:31.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76d40b6590 0x7f76d40b8aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76e40058c0 tx=0x7f76e400a2d0 comp rx=0 tx=0).stop 2024-09-21T02:49:31.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 msgr2=0x7f76ec19ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:31.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec19ebc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f76dc005cc0 tx=0x7f76dc0319f0 comp rx=0 tx=0).stop 2024-09-21T02:49:31.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 shutdown_connections 2024-09-21T02:49:31.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76d40b6590 0x7f76d40b8aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:31.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76ec1083c0 0x7f76ec199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:31.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76ec108d10 0x7f76ec19ebc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:31.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 --2- 172.21.15.165:0/3476334869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76ec102ad0 0x7f76ec199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:31.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.053+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 >> 172.21.15.165:0/3476334869 conn(0x7f76ec0796d0 msgr2=0x7f76ec104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:31.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.054+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 shutdown_connections 2024-09-21T02:49:31.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-21T02:49:31.054+0000 7f76f347d640 1 -- 172.21.15.165:0/3476334869 wait complete. 2024-09-21T02:49:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:30 smithi031 ceph-mon[31209]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:30 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.717 DEBUG:teuthology.orchestra.run.smithi165:osd.10> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T02:49:31.719 INFO:tasks.cephadm:Deploying osd.11 on smithi178 with /dev/vg_nvme/lv_1... 2024-09-21T02:49:31.719 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:49:31.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:31.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:31.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:31.978 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:49:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:32 smithi178 ceph-mon[34759]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:33.020 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:32 smithi165 ceph-mon[34818]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:32 smithi031 ceph-mon[31209]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:34.191 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 02:49:33 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T02:49:33.806+0000 7fbfef117640 -1 osd.10 0 log_to_monitors true 2024-09-21T02:49:34.644 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-21T02:49:34.674 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-09-21T02:49:34.919 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:34.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:34.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:34.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:49:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:49:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:35.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:35.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 -- 172.21.15.178:0/3184242314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 msgr2=0x7f45a81087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 --2- 172.21.15.178:0/3184242314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a81087c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4590009900 tx=0x7f459002a410 comp rx=0 tx=0).stop 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 -- 172.21.15.178:0/3184242314 shutdown_connections 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 --2- 172.21.15.178:0/3184242314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a81087c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 --2- 172.21.15.178:0/3184242314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a8108d00 0x7f45a810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 --2- 172.21.15.178:0/3184242314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 0x7f45a8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 -- 172.21.15.178:0/3184242314 >> 172.21.15.178:0/3184242314 conn(0x7f45a8079710 msgr2=0x7f45a8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 -- 172.21.15.178:0/3184242314 shutdown_connections 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 -- 172.21.15.178:0/3184242314 wait complete. 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.437+0000 7f45ae489640 1 Processor -- start 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 -- start start 2024-09-21T02:49:35.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 0x7f45a819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a8108d00 0x7f45a819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81129c0 con 0x7f45a8108d00 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8112840 con 0x7f45a8102a70 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45ae489640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8112b40 con 0x7f45a81083b0 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 0x7f45a819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43382/0 (socket says 172.21.15.178:43382) 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 -- 172.21.15.178:0/2472489053 learned_addr learned my addr 172.21.15.178:0/2472489053 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 0x7f45a819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.178:41078/0 (socket says 172.21.15.178:41078) 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f45aca26640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a8108d00 0x7f45a819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:35.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 -- 172.21.15.178:0/2472489053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 msgr2=0x7f45a819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 0x7f45a819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 -- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a8108d00 msgr2=0x7f45a819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a8108d00 0x7f45a819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 -- 172.21.15.178:0/2472489053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81a03f0 con 0x7f45a81083b0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.438+0000 7f459ffff640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a819aa20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f459002a930 tx=0x7f4590004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.439+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459003f030 con 0x7f45a81083b0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.439+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4590003aa0 con 0x7f45a81083b0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.439+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590035070 con 0x7f45a81083b0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.439+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a81a0680 con 0x7f45a81083b0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.439+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a6e40 con 0x7f45a81083b0 2024-09-21T02:49:35.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.440+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a8103cf0 con 0x7f45a81083b0 2024-09-21T02:49:35.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.441+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4590042030 con 0x7f45a81083b0 2024-09-21T02:49:35.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.442+0000 7f45a5ffb640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45940b65e0 0x7f45940b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:35.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.442+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6613+0+0 (secure 0 0 0) 0x7f45901076a0 con 0x7f45a81083b0 2024-09-21T02:49:35.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.442+0000 7f45a7fff640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45940b65e0 0x7f45940b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:35.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.443+0000 7f45a7fff640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45940b65e0 0x7f45940b8af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4598005fd0 tx=0x7f459800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:49:35.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.446+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45900d29b0 con 0x7f45a81083b0 2024-09-21T02:49:35.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:35.635+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f45a8063800 con 0x7f45940b65e0 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi165 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi165 to 4272M 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:49:35.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:49:35.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:35 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:36.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi165 2024-09-21T02:49:36.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi165 to 4272M 2024-09-21T02:49:36.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:49:36.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:49:36.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:49:36.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:49:36.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:49:36.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:49:36.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:35 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:36.191 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T02:49:35.714+0000 7fbfeb099640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T02:49:36.191 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T02:49:35.732+0000 7fbfe6e97640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:49:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi165 2024-09-21T02:49:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi165 to 4272M 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:49:36.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:35 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] boot 2024-09-21T02:49:36.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T02:49:36.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:36 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] boot 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T02:49:36.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:36 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: osd.10 [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] boot 2024-09-21T02:49:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T02:49:37.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:36 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:49:37.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:37 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:49:37.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:37 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:49:37.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:37 smithi178 ceph-mon[34759]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T02:49:38.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:37 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:49:38.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:37 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:49:38.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:37 smithi031 ceph-mon[31209]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T02:49:38.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:37 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:49:38.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:37 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:49:38.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:37 smithi165 ceph-mon[34818]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/1191617346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"} : dispatch 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"} : dispatch 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"}]': finished 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:49:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:38 smithi178 ceph-mon[34759]: from='client.? 172.21.15.178:0/2864296848' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/1191617346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"} : dispatch 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"} : dispatch 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"}]': finished 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:49:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.178:0/2864296848' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/1191617346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"} : dispatch 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"} : dispatch 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fb36d0d-9be8-49f1-b950-2e61cfcef7c5"}]': finished 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:49:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:38 smithi165 ceph-mon[34818]: from='client.? 172.21.15.178:0/2864296848' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:49:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:40 smithi031 ceph-mon[31209]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:40 smithi165 ceph-mon[34818]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:40 smithi178 ceph-mon[34759]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:42.998 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:42 smithi178 ceph-mon[34759]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T02:49:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:42 smithi031 ceph-mon[31209]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T02:49:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:42 smithi165 ceph-mon[34818]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-21T02:49:44.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:44 smithi178 ceph-mon[34759]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T02:49:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:44 smithi031 ceph-mon[31209]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T02:49:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:44 smithi165 ceph-mon[34818]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-21T02:49:46.828 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:46 smithi178 ceph-mon[34759]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:49:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:46 smithi031 ceph-mon[31209]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:49:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:46 smithi165 ceph-mon[34818]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T02:49:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:49:47.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:47 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:47.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:47 smithi178 ceph-mon[34759]: Deploying daemon osd.11 on smithi178 2024-09-21T02:49:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:49:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:47 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:48.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:47 smithi031 ceph-mon[31209]: Deploying daemon osd.11 on smithi178 2024-09-21T02:49:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:49:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:47 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:48.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:47 smithi165 ceph-mon[34818]: Deploying daemon osd.11 on smithi178 2024-09-21T02:49:48.823 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:48 smithi178 ceph-mon[34759]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T02:49:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:48 smithi031 ceph-mon[31209]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T02:49:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:48 smithi165 ceph-mon[34818]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T02:49:50.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:50 smithi178 ceph-mon[34759]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T02:49:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:50 smithi031 ceph-mon[31209]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T02:49:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:50 smithi165 ceph-mon[34818]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T02:49:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:52 smithi178 ceph-mon[34759]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:49:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:52 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:52 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:52 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:52 smithi031 ceph-mon[31209]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:49:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:52 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:52 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:52 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:52 smithi165 ceph-mon[34818]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:49:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:52 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:52 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:52 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:54 smithi031 ceph-mon[31209]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:54 smithi165 ceph-mon[34818]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:54 smithi178 ceph-mon[34759]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:56.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.350+0000 7f45a5ffb640 1 -- 172.21.15.178:0/2472489053 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f45a8063800 con 0x7f45940b65e0 2024-09-21T02:49:56.352 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 11 on host 'smithi178' 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45940b65e0 msgr2=0x7f45940b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45940b65e0 0x7f45940b8af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4598005fd0 tx=0x7f459800b390 comp rx=0 tx=0).stop 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 msgr2=0x7f45a819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a819aa20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f459002a930 tx=0x7f4590004270 comp rx=0 tx=0).stop 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 shutdown_connections 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45940b65e0 0x7f45940b8af0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a81083b0 0x7f45a819aa20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a8108d00 0x7f45a819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 --2- 172.21.15.178:0/2472489053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45a8102a70 0x7f45a819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 >> 172.21.15.178:0/2472489053 conn(0x7f45a8079710 msgr2=0x7f45a8101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 shutdown_connections 2024-09-21T02:49:56.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:49:56.353+0000 7f45ae489640 1 -- 172.21.15.178:0/2472489053 wait complete. 2024-09-21T02:49:56.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:56.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:56.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:56.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:56.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:56 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:56.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:56.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:56.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:56 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:49:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:49:57.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:57.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:49:57.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:57.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:56 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:49:57.091 DEBUG:teuthology.orchestra.run.smithi178:osd.11> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T02:49:57.093 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T02:49:57.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:49:57.370 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:49:57.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 -- 172.21.15.31:0/932655380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 msgr2=0x7f663010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 --2- 172.21.15.31:0/932655380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 0x7f663010f520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6628009ca0 tx=0x7f662802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 -- 172.21.15.31:0/932655380 shutdown_connections 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 --2- 172.21.15.31:0/932655380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66301083a0 0x7f66301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 --2- 172.21.15.31:0/932655380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 0x7f663010f520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 --2- 172.21.15.31:0/932655380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6630102a60 0x7f6630102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 -- 172.21.15.31:0/932655380 >> 172.21.15.31:0/932655380 conn(0x7f6630079730 msgr2=0x7f6630079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 -- 172.21.15.31:0/932655380 shutdown_connections 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.863+0000 7f6635549640 1 -- 172.21.15.31:0/932655380 wait complete. 2024-09-21T02:49:57.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.864+0000 7f6635549640 1 Processor -- start 2024-09-21T02:49:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.864+0000 7f6635549640 1 -- start start 2024-09-21T02:49:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6635549640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6630102a60 0x7f663019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6635549640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 0x7f663019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6635549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 0x7f663019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6635549640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66301129b0 con 0x7f6630108cf0 2024-09-21T02:49:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6635549640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6630112830 con 0x7f66301083a0 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6635549640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6630112b30 con 0x7f6630102a60 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6627fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 0x7f663019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 0x7f663019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6630102a60 0x7f663019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6627fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 0x7f663019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:41810/0 (socket says 172.21.15.31:41810) 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6627fff640 1 -- 172.21.15.31:0/1363684724 learned_addr learned my addr 172.21.15.31:0/1363684724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:49:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 -- 172.21.15.31:0/1363684724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 msgr2=0x7f663019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 0x7f663019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 -- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 msgr2=0x7f663019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 0x7f663019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 -- 172.21.15.31:0/1363684724 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66301a0360 con 0x7f6630102a60 2024-09-21T02:49:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f6627fff640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 0x7f663019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.865+0000 7f662effd640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6630102a60 0x7f663019a3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f662000d6d0 tx=0x7f662000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.866+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620017070 con 0x7f6630102a60 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.866+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66301a0650 con 0x7f6630102a60 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.866+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6620009820 con 0x7f6630102a60 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.866+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620004c40 con 0x7f6630102a60 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.866+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66301a6db0 con 0x7f6630102a60 2024-09-21T02:49:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.867+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6630102e70 con 0x7f6630102a60 2024-09-21T02:49:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.869+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f6620005220 con 0x7f6630102a60 2024-09-21T02:49:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.870+0000 7f662cff9640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f661c0b6560 0x7f661c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:49:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.870+0000 7f6627fff640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f661c0b6560 0x7f661c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:49:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.870+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7024+0+0 (secure 0 0 0) 0x7f66200e6ca0 con 0x7f6630102a60 2024-09-21T02:49:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.871+0000 7f6627fff640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f661c0b6560 0x7f661c0b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f663006a2b0 tx=0x7f661800b480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:49:57.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:57.872+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66200b1e20 con 0x7f6630102a60 2024-09-21T02:49:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.065+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f663002e640 con 0x7f6630102a60 2024-09-21T02:49:58.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.065+0000 7f662cff9640 1 -- 172.21.15.31:0/1363684724 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f66200b6d60 con 0x7f6630102a60 2024-09-21T02:49:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:49:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f661c0b6560 msgr2=0x7f661c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f661c0b6560 0x7f661c0b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f663006a2b0 tx=0x7f661800b480 comp rx=0 tx=0).stop 2024-09-21T02:49:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6630102a60 msgr2=0x7f663019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:49:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6630102a60 0x7f663019a3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f662000d6d0 tx=0x7f662000db90 comp rx=0 tx=0).stop 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 shutdown_connections 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f661c0b6560 0x7f661c0b8a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6630108cf0 0x7f663019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66301083a0 0x7f663019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 --2- 172.21.15.31:0/1363684724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6630102a60 0x7f663019a3f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 >> 172.21.15.31:0/1363684724 conn(0x7f6630079730 msgr2=0x7f663006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 shutdown_connections 2024-09-21T02:49:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:49:58.068+0000 7f6635549640 1 -- 172.21.15.31:0/1363684724 wait complete. 2024-09-21T02:49:58.478 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 02:49:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T02:49:58.267+0000 7f2e90de3640 -1 osd.11 0 log_to_monitors true 2024-09-21T02:49:58.787 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726886976,"num_in_osds":12,"osd_in_since":1726886978,"num_remapped_pgs":0} 2024-09-21T02:49:58.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:58 smithi165 ceph-mon[34818]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:58.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1363684724' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:49:58.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:58 smithi165 ceph-mon[34818]: from='osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:49:58.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:58 smithi165 ceph-mon[34818]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:49:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:58 smithi178 ceph-mon[34759]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1363684724' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:49:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:58 smithi178 ceph-mon[34759]: from='osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:49:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:58 smithi178 ceph-mon[34759]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:49:59.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:58 smithi031 ceph-mon[31209]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:49:59.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1363684724' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:49:59.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:58 smithi031 ceph-mon[31209]: from='osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:49:59.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:58 smithi031 ceph-mon[31209]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:49:59.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:49:59.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:59 smithi165 ceph-mon[34818]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:49:59.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:59 smithi165 ceph-mon[34818]: from='osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:49:59.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:59 smithi165 ceph-mon[34818]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:49:59.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:59 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:49:59.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:49:59 smithi165 ceph-mon[34818]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:49:59.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-mon[34759]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:49:59.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-mon[34759]: from='osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:49:59.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-mon[34759]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:49:59.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:49:59.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-mon[34759]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:49:59.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T02:49:59.659+0000 7f2e8cd65640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T02:49:59.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 02:49:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T02:49:59.671+0000 7f2e88b63640 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:50:00.055 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:00.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:59 smithi031 ceph-mon[31209]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:50:00.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:59 smithi031 ceph-mon[31209]: from='osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:50:00.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:59 smithi031 ceph-mon[31209]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:50:00.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:59 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:49:59 smithi031 ceph-mon[31209]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-21T02:50:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 -- 172.21.15.31:0/1010220376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 msgr2=0x7f33301125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 --2- 172.21.15.31:0/1010220376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301125c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3328009ca0 tx=0x7f332802a950 comp rx=0 tx=0).stop 2024-09-21T02:50:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 -- 172.21.15.31:0/1010220376 shutdown_connections 2024-09-21T02:50:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 --2- 172.21.15.31:0/1010220376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 0x7f333010fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 --2- 172.21.15.31:0/1010220376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301125c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 --2- 172.21.15.31:0/1010220376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33301025f0 0x7f3330102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 -- 172.21.15.31:0/1010220376 >> 172.21.15.31:0/1010220376 conn(0x7f3330079740 msgr2=0x7f3330079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 -- 172.21.15.31:0/1010220376 shutdown_connections 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 -- 172.21.15.31:0/1010220376 wait complete. 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.563+0000 7f3337932640 1 Processor -- start 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 -- start start 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33301025f0 0x7f333019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 0x7f333019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3330115c90 con 0x7f33301101a0 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3330115b10 con 0x7f33301025f0 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3337932640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3330115e10 con 0x7f3330102f40 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:00.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50994/0 (socket says 172.21.15.31:50994) 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 -- 172.21.15.31:0/347539475 learned_addr learned my addr 172.21.15.31:0/347539475 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f33356ce640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33301025f0 0x7f333019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3334ecd640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 0x7f333019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 -- 172.21.15.31:0/347539475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 msgr2=0x7f333019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 0x7f333019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 -- 172.21.15.31:0/347539475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33301025f0 msgr2=0x7f333019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33301025f0 0x7f333019c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3335ecf640 1 -- 172.21.15.31:0/347539475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33301a2530 con 0x7f33301101a0 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.564+0000 7f3334ecd640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 0x7f333019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.565+0000 7f3335ecf640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301a1e00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3328004af0 tx=0x7f332802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.565+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328034040 con 0x7f33301101a0 2024-09-21T02:50:00.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.565+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f332802fe80 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.565+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301a27c0 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.565+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280046a0 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.565+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301a8f80 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.566+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333006c240 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.567+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3328037020 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.568+0000 7f33267fc640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f331c0b6560 0x7f331c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.569+0000 7f33356ce640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f331c0b6560 0x7f331c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.569+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7062+0+0 (secure 0 0 0) 0x7f3328107c80 con 0x7f33301101a0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.569+0000 7f33356ce640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f331c0b6560 0x7f331c0b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f332c009bd0 tx=0x7f332c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.573+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33280ce9f0 con 0x7f33301101a0 2024-09-21T02:50:00.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.763+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f33301a91b0 con 0x7f33301101a0 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: overall HEALTH_OK 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.765 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:00.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:00 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.764+0000 7f33267fc640 1 -- 172.21.15.31:0/347539475 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f33280d2dd0 con 0x7f33301101a0 2024-09-21T02:50:00.767 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f331c0b6560 msgr2=0x7f331c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f331c0b6560 0x7f331c0b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f332c009bd0 tx=0x7f332c005f90 comp rx=0 tx=0).stop 2024-09-21T02:50:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 msgr2=0x7f33301a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:00.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301a1e00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3328004af0 tx=0x7f332802f5d0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 shutdown_connections 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f331c0b6560 0x7f331c0b8a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3330102f40 0x7f333019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33301101a0 0x7f33301a1e00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 --2- 172.21.15.31:0/347539475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33301025f0 0x7f333019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 >> 172.21.15.31:0/347539475 conn(0x7f3330079740 msgr2=0x7f3330109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 shutdown_connections 2024-09-21T02:50:00.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:00.766+0000 7f3337932640 1 -- 172.21.15.31:0/347539475 wait complete. 2024-09-21T02:50:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:50:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:50:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T02:50:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: overall HEALTH_OK 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:00.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:00 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: overall HEALTH_OK 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:00.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:00 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:01.399 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726887000,"num_in_osds":12,"osd_in_since":1726886978,"num_remapped_pgs":0} 2024-09-21T02:50:01.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:50:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: purged_snaps scrub starts 2024-09-21T02:50:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: purged_snaps scrub ok 2024-09-21T02:50:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: Detected new or changed devices on smithi178 2024-09-21T02:50:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-21T02:50:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] boot 2024-09-21T02:50:01.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:50:01.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:01.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:01 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/347539475' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: purged_snaps scrub starts 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: purged_snaps scrub ok 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: Detected new or changed devices on smithi178 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] boot 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:50:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:01.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:01 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/347539475' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: purged_snaps scrub starts 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: purged_snaps scrub ok 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: Detected new or changed devices on smithi178 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: osd.11 [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] boot 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:50:01.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:01 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/347539475' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 -- 172.21.15.31:0/3938658695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 msgr2=0x7fd2581087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 --2- 172.21.15.31:0/3938658695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd2581087b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd24802a410 comp rx=0 tx=0).stop 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 -- 172.21.15.31:0/3938658695 shutdown_connections 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 --2- 172.21.15.31:0/3938658695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd258108cf0 0x7fd25810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 --2- 172.21.15.31:0/3938658695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd2581087b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 --2- 172.21.15.31:0/3938658695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 0x7fd258102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 -- 172.21.15.31:0/3938658695 >> 172.21.15.31:0/3938658695 conn(0x7fd258079730 msgr2=0x7fd258079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:02.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 -- 172.21.15.31:0/3938658695 shutdown_connections 2024-09-21T02:50:02.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.130+0000 7fd2606ff640 1 -- 172.21.15.31:0/3938658695 wait complete. 2024-09-21T02:50:02.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 Processor -- start 2024-09-21T02:50:02.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 -- start start 2024-09-21T02:50:02.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 0x7fd25819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd25819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd258108cf0 0x7fd25819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd258113ab0 con 0x7fd2581083a0 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd258113930 con 0x7fd258102a60 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd2606ff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd258113c30 con 0x7fd258108cf0 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd25dc9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd25819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd25dc9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd25819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51016/0 (socket says 172.21.15.31:51016) 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.131+0000 7fd25dc9a640 1 -- 172.21.15.31:0/396168300 learned_addr learned my addr 172.21.15.31:0/396168300 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25e49b640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 0x7fd25819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25ec9c640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd258108cf0 0x7fd25819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25dc9a640 1 -- 172.21.15.31:0/396168300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd258108cf0 msgr2=0x7fd25819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:02.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25dc9a640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd258108cf0 0x7fd25819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25dc9a640 1 -- 172.21.15.31:0/396168300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 msgr2=0x7fd25819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25dc9a640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 0x7fd25819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25dc9a640 1 -- 172.21.15.31:0/396168300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2581a04f0 con 0x7fd2581083a0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25e49b640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 0x7fd25819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd25dc9a640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd25819ab20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd248031a90 tx=0x7fd248004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24803f030 con 0x7fd2581083a0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2581a0780 con 0x7fd2581083a0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd248003cb0 con 0x7fd2581083a0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248035070 con 0x7fd2581083a0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.132+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2581a6f40 con 0x7fd2581083a0 2024-09-21T02:50:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.133+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd258103ce0 con 0x7fd2581083a0 2024-09-21T02:50:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.135+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fd248042030 con 0x7fd2581083a0 2024-09-21T02:50:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.136+0000 7fd24f7fe640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd2440b6560 0x7fd2440b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.136+0000 7fd25e49b640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd2440b6560 0x7fd2440b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.136+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fd248107b90 con 0x7fd2581083a0 2024-09-21T02:50:02.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.136+0000 7fd25e49b640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd2440b6560 0x7fd2440b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2540058c0 tx=0x7fd254005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.139+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2480ce750 con 0x7fd2581083a0 2024-09-21T02:50:02.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.328+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd25810aae0 con 0x7fd2581083a0 2024-09-21T02:50:02.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.329+0000 7fd24f7fe640 1 -- 172.21.15.31:0/396168300 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18806 (secure 0 0 0) 0x7fd2480d2b30 con 0x7fd2581083a0 2024-09-21T02:50:02.331 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:02.331 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":71,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:50:01.663545+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd2440b6560 msgr2=0x7fd2440b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd2440b6560 0x7fd2440b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2540058c0 tx=0x7fd254005ec0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 msgr2=0x7fd25819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd25819ab20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd248031a90 tx=0x7fd248004270 comp rx=0 tx=0).stop 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 shutdown_connections 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd2440b6560 0x7fd2440b8a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd258108cf0 0x7fd25819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2581083a0 0x7fd25819ab20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 --2- 172.21.15.31:0/396168300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd258102a60 0x7fd25819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:02.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 >> 172.21.15.31:0/396168300 conn(0x7fd258079730 msgr2=0x7fd258104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:02.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 shutdown_connections 2024-09-21T02:50:02.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:02.332+0000 7fd2606ff640 1 -- 172.21.15.31:0/396168300 wait complete. 2024-09-21T02:50:02.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:02 smithi165 ceph-mon[34818]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:50:02.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:02 smithi165 ceph-mon[34818]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:50:02.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:02 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/396168300' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:02 smithi031 ceph-mon[31209]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:50:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:02 smithi031 ceph-mon[31209]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:50:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:02 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/396168300' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:02.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:02 smithi178 ceph-mon[34759]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:50:02.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:02 smithi178 ceph-mon[34759]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:50:02.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:02 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/396168300' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:02.998 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:46:13.535825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:50:02.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T02:50:03.255 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1930319421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c102a60 msgr2=0x7f3b3c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1930319421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c102a60 0x7f3b3c102e70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009900 tx=0x7f3b2c02a410 comp rx=0 tx=0).stop 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1930319421 shutdown_connections 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1930319421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1930319421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b3c1083a0 0x7f3b3c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1930319421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c102a60 0x7f3b3c102e70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1930319421 >> 172.21.15.31:0/1930319421 conn(0x7f3b3c079730 msgr2=0x7f3b3c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1930319421 shutdown_connections 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.708+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1930319421 wait complete. 2024-09-21T02:50:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 Processor -- start 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 -- start start 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b3c102a60 0x7f3b3c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b3c1083a0 0x7f3b3c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1129b0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c112830 con 0x7f3b3c1083a0 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b42f8f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c112b30 con 0x7f3b3c102a60 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.709+0000 7f3b4152c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51026/0 (socket says 172.21.15.31:51026) 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 -- 172.21.15.31:0/1144231571 learned_addr learned my addr 172.21.15.31:0/1144231571 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b3bfff640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b3c1083a0 0x7f3b3c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b40d2b640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b3c102a60 0x7f3b3c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 -- 172.21.15.31:0/1144231571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b3c102a60 msgr2=0x7f3b3c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b3c102a60 0x7f3b3c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 -- 172.21.15.31:0/1144231571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b3c1083a0 msgr2=0x7f3b3c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b3c1083a0 0x7f3b3c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 -- 172.21.15.31:0/1144231571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c1a03e0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b4152c640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c19fcb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400cad0 tx=0x7f3b3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400ae20 con 0x7f3b3c108cf0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b34005020 con 0x7f3b3c108cf0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c1a06d0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.710+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400f6b0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.711+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c1a6e30 con 0x7f3b3c108cf0 2024-09-21T02:50:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.712+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3c102e70 con 0x7f3b3c108cf0 2024-09-21T02:50:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.714+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3b340051d0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.715+0000 7f3b39ffb640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b280b6560 0x7f3b280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.715+0000 7f3b40d2b640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b280b6560 0x7f3b280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.715+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f3b340e6fd0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.715+0000 7f3b40d2b640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b280b6560 0x7f3b280b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000c00 tx=0x7f3b2c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:03.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.719+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b34014030 con 0x7f3b3c108cf0 2024-09-21T02:50:03.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.907+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b3c1a71a0 con 0x7f3b3c108cf0 2024-09-21T02:50:03.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.907+0000 7f3b39ffb640 1 -- 172.21.15.31:0/1144231571 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f3b340adc30 con 0x7f3b3c108cf0 2024-09-21T02:50:03.908 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b280b6560 msgr2=0x7f3b280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b280b6560 0x7f3b280b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000c00 tx=0x7f3b2c031400 comp rx=0 tx=0).stop 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 msgr2=0x7f3b3c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c19fcb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400cad0 tx=0x7f3b3400cf90 comp rx=0 tx=0).stop 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 shutdown_connections 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b280b6560 0x7f3b280b8a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b3c108cf0 0x7f3b3c19fcb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b3c1083a0 0x7f3b3c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 --2- 172.21.15.31:0/1144231571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b3c102a60 0x7f3b3c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 >> 172.21.15.31:0/1144231571 conn(0x7f3b3c079730 msgr2=0x7f3b3c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 shutdown_connections 2024-09-21T02:50:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:03.909+0000 7f3b42f8f640 1 -- 172.21.15.31:0/1144231571 wait complete. 2024-09-21T02:50:04.694 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.b on smithi165 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.e on smithi165 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.h on smithi165 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.k on smithi165 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.c on smithi178 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.f on smithi178 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.i on smithi178 2024-09-21T02:50:04.695 INFO:tasks.cephadm:Adding mds.l on smithi178 2024-09-21T02:50:04.695 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l' 2024-09-21T02:50:04.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:04 smithi165 ceph-mon[34818]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:04.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:04 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1144231571' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:50:04.954 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:50:04.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:04 smithi178 ceph-mon[34759]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:04.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:04 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1144231571' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:50:05.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:04 smithi031 ceph-mon[31209]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:05.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:04 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1144231571' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:50:05.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 -- 172.21.15.178:0/263588277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228103b60 msgr2=0x7f6228103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:05.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 --2- 172.21.15.178:0/263588277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228103b60 0x7f6228103f70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6218009900 tx=0x7f621802a410 comp rx=0 tx=0).stop 2024-09-21T02:50:05.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 -- 172.21.15.178:0/263588277 shutdown_connections 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 --2- 172.21.15.178:0/263588277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6228109df0 0x7f6228110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 --2- 172.21.15.178:0/263588277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62281094a0 0x7f62281098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 --2- 172.21.15.178:0/263588277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228103b60 0x7f6228103f70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 -- 172.21.15.178:0/263588277 >> 172.21.15.178:0/263588277 conn(0x7f622807b850 msgr2=0x7f622807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 -- 172.21.15.178:0/263588277 shutdown_connections 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.418+0000 7f622f3f4640 1 -- 172.21.15.178:0/263588277 wait complete. 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 Processor -- start 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 -- start start 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6228103b60 0x7f622819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:05.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281094a0 0x7f622819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 0x7f622819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6228113ab0 con 0x7f62281094a0 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6228113930 con 0x7f6228103b60 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622f3f4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6228113c30 con 0x7f6228109df0 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 0x7f622819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 0x7f622819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48926/0 (socket says 172.21.15.178:48926) 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 -- 172.21.15.178:0/2558494967 learned_addr learned my addr 172.21.15.178:0/2558494967 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d190640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6228103b60 0x7f622819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622c98f640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281094a0 0x7f622819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 -- 172.21.15.178:0/2558494967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6228103b60 msgr2=0x7f622819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6228103b60 0x7f622819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 -- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281094a0 msgr2=0x7f622819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:05.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.419+0000 7f622d991640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281094a0 0x7f622819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f622d991640 1 -- 172.21.15.178:0/2558494967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62281a0430 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f622d991640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 0x7f622819fd00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f622000eaa0 tx=0x7f622000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622000cdb0 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62281a0720 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6220005020 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200116d0 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.420+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281a6e80 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.421+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6228103f70 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.422+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f62200051d0 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.423+0000 7f62267fc640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62100b6560 0x7f62100b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.423+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f62200e74b0 con 0x7f6228109df0 2024-09-21T02:50:05.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.423+0000 7f622d190640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62100b6560 0x7f62100b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.425+0000 7f622d190640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62100b6560 0x7f62100b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6218031440 tx=0x7f6218009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:05.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.426+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6220016030 con 0x7f6228109df0 2024-09-21T02:50:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.615+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}) -- 0x7f6228063800 con 0x7f62100b6560 2024-09-21T02:50:05.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.625+0000 7f62267fc640 1 -- 172.21.15.178:0/2558494967 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6228063800 con 0x7f62100b6560 2024-09-21T02:50:05.627 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mds.all update... 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62100b6560 msgr2=0x7f62100b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62100b6560 0x7f62100b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6218031440 tx=0x7f6218009560 comp rx=0 tx=0).stop 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 msgr2=0x7f622819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 0x7f622819fd00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f622000eaa0 tx=0x7f622000ef60 comp rx=0 tx=0).stop 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 shutdown_connections 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f62100b6560 0x7f62100b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6228109df0 0x7f622819fd00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281094a0 0x7f622819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 --2- 172.21.15.178:0/2558494967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6228103b60 0x7f622819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 >> 172.21.15.178:0/2558494967 conn(0x7f622807b850 msgr2=0x7f6228107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 shutdown_connections 2024-09-21T02:50:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-21T02:50:05.627+0000 7f622f3f4640 1 -- 172.21.15.178:0/2558494967 wait complete. 2024-09-21T02:50:06.351 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T02:50:06.353 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T02:50:06.396 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T02:50:06.439 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T02:50:06.482 DEBUG:teuthology.orchestra.run.smithi165:mds.b> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T02:50:06.484 DEBUG:teuthology.orchestra.run.smithi165:mds.e> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T02:50:06.527 DEBUG:teuthology.orchestra.run.smithi165:mds.h> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T02:50:06.570 DEBUG:teuthology.orchestra.run.smithi165:mds.k> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T02:50:06.614 DEBUG:teuthology.orchestra.run.smithi178:mds.c> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T02:50:06.616 DEBUG:teuthology.orchestra.run.smithi178:mds.f> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T02:50:06.659 DEBUG:teuthology.orchestra.run.smithi178:mds.i> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T02:50:06.703 DEBUG:teuthology.orchestra.run.smithi178:mds.l> sudo journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T02:50:06.746 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T02:50:06.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:06.902 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:06.903 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:06 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-21T02:50:06.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:06.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:06.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:06.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:06.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:06.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:06.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:06.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:06 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='client.24571 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:07.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:06 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 -- 172.21.15.31:0/4179762263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 msgr2=0x7f78481098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 --2- 172.21.15.31:0/4179762263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f78481098d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f783c009900 tx=0x7f783c02a410 comp rx=0 tx=0).stop 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 -- 172.21.15.31:0/4179762263 shutdown_connections 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 --2- 172.21.15.31:0/4179762263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848109e10 0x7f7848110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 --2- 172.21.15.31:0/4179762263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f78481098d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 --2- 172.21.15.31:0/4179762263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7848101fe0 0x7f78481023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 -- 172.21.15.31:0/4179762263 >> 172.21.15.31:0/4179762263 conn(0x7f7848077bc0 msgr2=0x7f7848078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 -- 172.21.15.31:0/4179762263 shutdown_connections 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.446+0000 7f784f438640 1 -- 172.21.15.31:0/4179762263 wait complete. 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 Processor -- start 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 -- start start 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848101fe0 0x7f784819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f784819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7848109e10 0x7f784819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113a80 con 0x7f78481094c0 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113900 con 0x7f7848109e10 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784f438640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113c00 con 0x7f7848101fe0 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784c9d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f784819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784c9d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f784819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51050/0 (socket says 172.21.15.31:51050) 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.447+0000 7f784c9d3640 1 -- 172.21.15.31:0/2455688255 learned_addr learned my addr 172.21.15.31:0/2455688255 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784d1d4640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848101fe0 0x7f784819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784d9d5640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7848109e10 0x7f784819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784c9d3640 1 -- 172.21.15.31:0/2455688255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848101fe0 msgr2=0x7f784819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784c9d3640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848101fe0 0x7f784819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784c9d3640 1 -- 172.21.15.31:0/2455688255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7848109e10 msgr2=0x7f784819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784c9d3640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7848109e10 0x7f784819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784c9d3640 1 -- 172.21.15.31:0/2455688255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481a0400 con 0x7f78481094c0 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784d1d4640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848101fe0 0x7f784819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784c9d3640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f784819aa30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f783c02a930 tx=0x7f783c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c03f030 con 0x7f78481094c0 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c003aa0 con 0x7f78481094c0 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c035070 con 0x7f78481094c0 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78481a0690 con 0x7f78481094c0 2024-09-21T02:50:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.448+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a6e50 con 0x7f78481094c0 2024-09-21T02:50:07.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.449+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784806b970 con 0x7f78481094c0 2024-09-21T02:50:07.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.451+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f783c042030 con 0x7f78481094c0 2024-09-21T02:50:07.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.452+0000 7f78467fc640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78300b6560 0x7f78300b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:07.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.452+0000 7f784d1d4640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78300b6560 0x7f78300b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:07.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.452+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f783c107b80 con 0x7f78481094c0 2024-09-21T02:50:07.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.452+0000 7f784d1d4640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78300b6560 0x7f78300b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7838005c40 tx=0x7f783800a430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.455+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f783c0ce740 con 0x7f78481094c0 2024-09-21T02:50:07.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.679+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f784810bc90 con 0x7f78481094c0 2024-09-21T02:50:07.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:07 smithi031 ceph-mon[31209]: Deploying daemon mds.l on smithi178 2024-09-21T02:50:07.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.680+0000 7f78467fc640 1 -- 172.21.15.31:0/2455688255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f783c0d2b20 con 0x7f78481094c0 2024-09-21T02:50:07.682 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:07.682 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"btime":"2024-09-21T02:42:18:754388+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T02:50:07.682 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78300b6560 msgr2=0x7f78300b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78300b6560 0x7f78300b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7838005c40 tx=0x7f783800a430 comp rx=0 tx=0).stop 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 msgr2=0x7f784819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f784819aa30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f783c02a930 tx=0x7f783c004270 comp rx=0 tx=0).stop 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 shutdown_connections 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78300b6560 0x7f78300b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7848109e10 0x7f784819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78481094c0 0x7f784819aa30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 --2- 172.21.15.31:0/2455688255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7848101fe0 0x7f784819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 >> 172.21.15.31:0/2455688255 conn(0x7f7848077bc0 msgr2=0x7f7848107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 shutdown_connections 2024-09-21T02:50:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:07.682+0000 7f784f438640 1 -- 172.21.15.31:0/2455688255 wait complete. 2024-09-21T02:50:07.846 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:07 smithi178 ceph-mon[34759]: Deploying daemon mds.l on smithi178 2024-09-21T02:50:07.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:07 smithi165 ceph-mon[34818]: Deploying daemon mds.l on smithi178 2024-09-21T02:50:08.228 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 systemd[1]: Starting Ceph mds.l for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:08.489 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 podman[63391]: 2024-09-21 02:50:08.140456545 +0000 UTC m=+0.015992712 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:08.489 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 podman[63391]: 2024-09-21 02:50:08.249459824 +0000 UTC m=+0.124996020 container create 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T02:50:08.489 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 podman[63391]: 2024-09-21 02:50:08.485702491 +0000 UTC m=+0.361238644 container init 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T02:50:08.489 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 podman[63391]: 2024-09-21 02:50:08.488501401 +0000 UTC m=+0.364037555 container start 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T02:50:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:08 smithi031 ceph-mon[31209]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:08 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2455688255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:08 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6832/1642580204,v1:172.21.15.178:6833/1642580204] up:boot 2024-09-21T02:50:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:08 smithi031 ceph-mon[31209]: fsmap 1 up:standby 2024-09-21T02:50:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:08 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:50:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:08 smithi165 ceph-mon[34818]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:08 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2455688255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:08 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6832/1642580204,v1:172.21.15.178:6833/1642580204] up:boot 2024-09-21T02:50:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:08 smithi165 ceph-mon[34818]: fsmap 1 up:standby 2024-09-21T02:50:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:08 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-mon[34759]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2455688255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6832/1642580204,v1:172.21.15.178:6833/1642580204] up:boot 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-mon[34759]: fsmap 1 up:standby 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-mds[63410]: -- 172.21.15.178:0/2203521831 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1f7d31c20 con 0x55e1fbed9600 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l[63406]: starting mds.l at 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 bash[63391]: 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc 2024-09-21T02:50:08.978 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 02:50:08 smithi178 systemd[1]: Started Ceph mds.l for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:09.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:09.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:09.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:09.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:09.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:09.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:09.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:09 smithi165 ceph-mon[34818]: Deploying daemon mds.k on smithi165 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:09.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:09 smithi178 ceph-mon[34759]: Deploying daemon mds.k on smithi165 2024-09-21T02:50:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:10.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:09 smithi031 ceph-mon[31209]: Deploying daemon mds.k on smithi165 2024-09-21T02:50:10.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:10.665 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:10 smithi165 systemd[1]: Starting Ceph mds.k for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:10.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:10 smithi031 ceph-mon[31209]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:10.924 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:10 smithi165 ceph-mon[34818]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:10.924 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:10 smithi165 podman[62916]: 2024-09-21 02:50:10.650779251 +0000 UTC m=+0.015746706 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:10.924 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:10 smithi165 podman[62916]: 2024-09-21 02:50:10.777148063 +0000 UTC m=+0.142115510 container create 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:50:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:10 smithi178 ceph-mon[34759]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:11.191 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:11 smithi165 podman[62916]: 2024-09-21 02:50:11.029891917 +0000 UTC m=+0.394859364 container init 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T02:50:11.191 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:11 smithi165 podman[62916]: 2024-09-21 02:50:11.032700148 +0000 UTC m=+0.397667596 container start 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:50:11.191 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:11 smithi165 ceph-mds[62935]: -- 172.21.15.165:0/1544304960 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a68dd5d860 con 0x55a691f05180 2024-09-21T02:50:11.191 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k[62931]: starting mds.k at 2024-09-21T02:50:11.191 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:11 smithi165 bash[62916]: 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc 2024-09-21T02:50:11.191 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 02:50:11 smithi165 systemd[1]: Started Ceph mds.k for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:11.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 -- 172.21.15.31:0/2009717637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c108cf0 msgr2=0x7f322c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 --2- 172.21.15.31:0/2009717637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c108cf0 0x7f322c10f520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3224009ca0 tx=0x7f322402a8f0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 -- 172.21.15.31:0/2009717637 shutdown_connections 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 --2- 172.21.15.31:0/2009717637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c1083a0 0x7f322c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 --2- 172.21.15.31:0/2009717637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c108cf0 0x7f322c10f520 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 --2- 172.21.15.31:0/2009717637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f322c102a60 0x7f322c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 -- 172.21.15.31:0/2009717637 >> 172.21.15.31:0/2009717637 conn(0x7f322c079730 msgr2=0x7f322c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 -- 172.21.15.31:0/2009717637 shutdown_connections 2024-09-21T02:50:11.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.197+0000 7f3232024640 1 -- 172.21.15.31:0/2009717637 wait complete. 2024-09-21T02:50:11.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.198+0000 7f3232024640 1 Processor -- start 2024-09-21T02:50:11.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 -- start start 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c102a60 0x7f322c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 0x7f322c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f322c108cf0 0x7f322c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f322c112a60 con 0x7f322c1083a0 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1128e0 con 0x7f322c108cf0 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f3232024640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f322c112be0 con 0x7f322c102a60 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 0x7f322c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 0x7f322c199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34282/0 (socket says 172.21.15.31:34282) 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 -- 172.21.15.31:0/2823441257 learned_addr learned my addr 172.21.15.31:0/2823441257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:11.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322b7fe640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c102a60 0x7f322c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322bfff640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f322c108cf0 0x7f322c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 -- 172.21.15.31:0/2823441257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c102a60 msgr2=0x7f322c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c102a60 0x7f322c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 -- 172.21.15.31:0/2823441257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f322c108cf0 msgr2=0x7f322c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f322c108cf0 0x7f322c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.199+0000 7f322affd640 1 -- 172.21.15.31:0/2823441257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c19f520 con 0x7f322c1083a0 2024-09-21T02:50:11.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.200+0000 7f322affd640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 0x7f322c199b50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f322000b470 tx=0x7f322000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:11.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.201+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000e830 con 0x7f322c1083a0 2024-09-21T02:50:11.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.201+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3220018400 con 0x7f322c1083a0 2024-09-21T02:50:11.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.201+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c19f810 con 0x7f322c1083a0 2024-09-21T02:50:11.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.201+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000f660 con 0x7f322c1083a0 2024-09-21T02:50:11.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.202+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c06ae60 con 0x7f322c1083a0 2024-09-21T02:50:11.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.203+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c102ef0 con 0x7f322c1083a0 2024-09-21T02:50:11.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.204+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f322000e9d0 con 0x7f322c1083a0 2024-09-21T02:50:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.205+0000 7f3228ff9640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32140b65e0 0x7f32140b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.205+0000 7f322b7fe640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32140b65e0 0x7f32140b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:11.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.205+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f32200e6830 con 0x7f322c1083a0 2024-09-21T02:50:11.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.206+0000 7f322b7fe640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32140b65e0 0x7f32140b8af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f321c006fd0 tx=0x7f321c00f040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.208+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3220014030 con 0x7f322c1083a0 2024-09-21T02:50:11.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.436+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322c19a6e0 con 0x7f322c1083a0 2024-09-21T02:50:11.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.437+0000 7f3228ff9640 1 -- 172.21.15.31:0/2823441257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f322c19a6e0 con 0x7f322c1083a0 2024-09-21T02:50:11.439 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:11.439 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"btime":"2024-09-21T02:50:08:527015+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:50:11.439 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32140b65e0 msgr2=0x7f32140b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32140b65e0 0x7f32140b8af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f321c006fd0 tx=0x7f321c00f040 comp rx=0 tx=0).stop 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 msgr2=0x7f322c199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 0x7f322c199b50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f322000b470 tx=0x7f322000b930 comp rx=0 tx=0).stop 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 shutdown_connections 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32140b65e0 0x7f32140b8af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f322c108cf0 0x7f322c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f322c1083a0 0x7f322c199b50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 --2- 172.21.15.31:0/2823441257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f322c102a60 0x7f322c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:11.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.439+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 >> 172.21.15.31:0/2823441257 conn(0x7f322c079730 msgr2=0x7f322c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:11.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.440+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 shutdown_connections 2024-09-21T02:50:11.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:11.440+0000 7f3232024640 1 -- 172.21.15.31:0/2823441257 wait complete. 2024-09-21T02:50:12.435 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: Deploying daemon mds.j on smithi031 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2823441257' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6832/3390770693,v1:172.21.15.165:6833/3390770693] up:boot 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: fsmap 2 up:standby 2024-09-21T02:50:12.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:12.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: Deploying daemon mds.j on smithi031 2024-09-21T02:50:12.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2823441257' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:12.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6832/3390770693,v1:172.21.15.165:6833/3390770693] up:boot 2024-09-21T02:50:12.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: fsmap 2 up:standby 2024-09-21T02:50:12.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:50:12.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:12.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: Deploying daemon mds.j on smithi031 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2823441257' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6832/3390770693,v1:172.21.15.165:6833/3390770693] up:boot 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: fsmap 2 up:standby 2024-09-21T02:50:12.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:50:14.084 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:13 smithi031 systemd[1]: Starting Ceph mds.j for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:14.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:14.399 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:14 smithi031 ceph-mon[31209]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:14.399 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 podman[72745]: 2024-09-21 02:50:14.030738056 +0000 UTC m=+0.016302467 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:14.399 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 podman[72745]: 2024-09-21 02:50:14.154285654 +0000 UTC m=+0.139850054 container create 1ad677833a1885cdc7162fff5647042fa2978429ff48003730b0742442f40d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T02:50:14.399 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 podman[72745]: 2024-09-21 02:50:14.398476526 +0000 UTC m=+0.384040933 container init 1ad677833a1885cdc7162fff5647042fa2978429ff48003730b0742442f40d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:50:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:14 smithi165 ceph-mon[34818]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:14.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:14 smithi178 ceph-mon[34759]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:14.569 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:14.691 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 podman[72745]: 2024-09-21 02:50:14.401908068 +0000 UTC m=+0.387472465 container start 1ad677833a1885cdc7162fff5647042fa2978429ff48003730b0742442f40d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T02:50:14.692 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 ceph-mds[72782]: -- 172.21.15.31:0/2672538549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fed71df860 con 0x55fedb387600 2024-09-21T02:50:14.692 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j[72764]: starting mds.j at 2024-09-21T02:50:14.692 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 bash[72745]: 1ad677833a1885cdc7162fff5647042fa2978429ff48003730b0742442f40d84 2024-09-21T02:50:14.692 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 02:50:14 smithi031 systemd[1]: Started Ceph mds.j for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 -- 172.21.15.31:0/1674113337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 msgr2=0x7f2bec102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/1674113337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec102ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009900 tx=0x7f2bdc02a410 comp rx=0 tx=0).stop 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 -- 172.21.15.31:0/1674113337 shutdown_connections 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/1674113337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec1083d0 0x7f2bec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/1674113337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bec108d20 0x7f2bec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/1674113337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec102ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 -- 172.21.15.31:0/1674113337 >> 172.21.15.31:0/1674113337 conn(0x7f2bec079760 msgr2=0x7f2bec079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 -- 172.21.15.31:0/1674113337 shutdown_connections 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 -- 172.21.15.31:0/1674113337 wait complete. 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 Processor -- start 2024-09-21T02:50:15.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.051+0000 7f2bf23f2640 1 -- start start 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf23f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf23f2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bec1083d0 0x7f2bec1047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf23f2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec108d20 0x7f2bec104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf23f2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec112990 con 0x7f2bec102ae0 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf23f2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec112810 con 0x7f2bec1083d0 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf23f2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec112b10 con 0x7f2bec108d20 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34326/0 (socket says 172.21.15.31:34326) 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 -- 172.21.15.31:0/3900716641 learned_addr learned my addr 172.21.15.31:0/3900716641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bf098f640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec108d20 0x7f2bec104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2beb7fe640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bec1083d0 0x7f2bec1047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 -- 172.21.15.31:0/3900716641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec108d20 msgr2=0x7f2bec104d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec108d20 0x7f2bec104d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 -- 172.21.15.31:0/3900716641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bec1083d0 msgr2=0x7f2bec1047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bec1083d0 0x7f2bec1047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.052+0000 7f2bebfff640 1 -- 172.21.15.31:0/3900716641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec1a8750 con 0x7f2bec102ae0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.053+0000 7f2bebfff640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec104280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005940 tx=0x7f2bdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.053+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc03f030 con 0x7f2bec102ae0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.053+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec1a89e0 con 0x7f2bec102ae0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.053+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bdc003cb0 con 0x7f2bec102ae0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.053+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc035070 con 0x7f2bec102ae0 2024-09-21T02:50:15.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.053+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec1a8f10 con 0x7f2bec102ae0 2024-09-21T02:50:15.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.054+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec103d10 con 0x7f2bec102ae0 2024-09-21T02:50:15.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.056+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f2bdc042030 con 0x7f2bec102ae0 2024-09-21T02:50:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.057+0000 7f2be97fa640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2bd40b6560 0x7f2bd40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.057+0000 7f2beb7fe640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2bd40b6560 0x7f2bd40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.057+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f2bdc107940 con 0x7f2bec102ae0 2024-09-21T02:50:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.057+0000 7f2beb7fe640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2bd40b6560 0x7f2bd40b8a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2be0005d00 tx=0x7f2be0005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:15.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.061+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2bdc0ce5a0 con 0x7f2bec102ae0 2024-09-21T02:50:15.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.306+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bec105580 con 0x7f2bec102ae0 2024-09-21T02:50:15.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.306+0000 7f2be97fa640 1 -- 172.21.15.31:0/3900716641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f2bdc0d2980 con 0x7f2bec102ae0 2024-09-21T02:50:15.308 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:15.308 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"btime":"2024-09-21T02:50:15:183238+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T02:50:15.308 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.308+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2bd40b6560 msgr2=0x7f2bd40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.308+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2bd40b6560 0x7f2bd40b8a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2be0005d00 tx=0x7f2be0005c50 comp rx=0 tx=0).stop 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.308+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 msgr2=0x7f2bec104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.308+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec104280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005940 tx=0x7f2bdc0042f0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 shutdown_connections 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2bd40b6560 0x7f2bd40b8a70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec108d20 0x7f2bec104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bec1083d0 0x7f2bec1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 --2- 172.21.15.31:0/3900716641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec102ae0 0x7f2bec104280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 >> 172.21.15.31:0/3900716641 conn(0x7f2bec079760 msgr2=0x7f2bec101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 shutdown_connections 2024-09-21T02:50:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:15.309+0000 7f2bf23f2640 1 -- 172.21.15.31:0/3900716641 wait complete. 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:15.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: Deploying daemon mds.d on smithi031 2024-09-21T02:50:15.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.31:6834/3355921594,v1:172.21.15.31:6835/3355921594] up:boot 2024-09-21T02:50:15.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: fsmap 3 up:standby 2024-09-21T02:50:15.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:50:15.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:15 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3900716641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:15.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:15.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:15.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: Deploying daemon mds.d on smithi031 2024-09-21T02:50:15.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.31:6834/3355921594,v1:172.21.15.31:6835/3355921594] up:boot 2024-09-21T02:50:15.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: fsmap 3 up:standby 2024-09-21T02:50:15.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:50:15.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:15 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3900716641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: Deploying daemon mds.d on smithi031 2024-09-21T02:50:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.31:6834/3355921594,v1:172.21.15.31:6835/3355921594] up:boot 2024-09-21T02:50:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: fsmap 3 up:standby 2024-09-21T02:50:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:50:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:15 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3900716641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:16.709 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:16 smithi031 ceph-mon[31209]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:16 smithi165 ceph-mon[34818]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:16 smithi178 ceph-mon[34759]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:18.084 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 02:50:17 smithi031 systemd[1]: Started Ceph mds.d for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:18.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:18.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:18 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:18.733 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:18 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:18 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:19.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/1077746650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc110230 msgr2=0x7fe5dc112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:19.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/1077746650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc110230 0x7fe5dc112650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009ca0 tx=0x7fe5d402a950 comp rx=0 tx=0).stop 2024-09-21T02:50:19.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/1077746650 shutdown_connections 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/1077746650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc110230 0x7fe5dc112650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/1077746650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5dc102fd0 0x7fe5dc10fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/1077746650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5dc102680 0x7fe5dc102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/1077746650 >> 172.21.15.31:0/1077746650 conn(0x7fe5dc0796d0 msgr2=0x7fe5dc079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/1077746650 shutdown_connections 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.260+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/1077746650 wait complete. 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 Processor -- start 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 -- start start 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 0x7fe5dc19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5dc102fd0 0x7fe5dc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5dc110230 0x7fe5dc1a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc115d20 con 0x7fe5dc102680 2024-09-21T02:50:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc115ba0 con 0x7fe5dc110230 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e2ddc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc115ea0 con 0x7fe5dc102fd0 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e0b78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 0x7fe5dc19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e0b78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 0x7fe5dc19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43498/0 (socket says 172.21.15.31:43498) 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e0b78640 1 -- 172.21.15.31:0/4069478345 learned_addr learned my addr 172.21.15.31:0/4069478345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5e1379640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5dc110230 0x7fe5dc1a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.261+0000 7fe5dbfff640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5dc102fd0 0x7fe5dc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e0b78640 1 -- 172.21.15.31:0/4069478345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5dc102fd0 msgr2=0x7fe5dc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e0b78640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5dc102fd0 0x7fe5dc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e0b78640 1 -- 172.21.15.31:0/4069478345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5dc110230 msgr2=0x7fe5dc1a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e0b78640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5dc110230 0x7fe5dc1a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e0b78640 1 -- 172.21.15.31:0/4069478345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc1a2680 con 0x7fe5dc102680 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e0b78640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 0x7fe5dc19c710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d770 tx=0x7fe5d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0017070 con 0x7fe5dc102680 2024-09-21T02:50:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc1a2970 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d00054d0 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0004c30 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.262+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a90d0 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.263+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc102a90 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.265+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fe5d0004dd0 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.266+0000 7fe5d9ffb640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5cc0b6560 0x7fe5cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.266+0000 7fe5dbfff640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5cc0b6560 0x7fe5cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.266+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fe5d00e6d90 con 0x7fe5dc102680 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.266+0000 7fe5dbfff640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5cc0b6560 0x7fe5cc0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc110e80 tx=0x7fe5c800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.269+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5d00b1dd0 con 0x7fe5dc102680 2024-09-21T02:50:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.497+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5dc1a9370 con 0x7fe5dc102680 2024-09-21T02:50:19.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.498+0000 7fe5d9ffb640 1 -- 172.21.15.31:0/4069478345 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7fe5d00b6d10 con 0x7fe5dc102680 2024-09-21T02:50:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:19.500 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"btime":"2024-09-21T02:50:18:575925+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T02:50:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2024-09-21T02:50:19.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5cc0b6560 msgr2=0x7fe5cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5cc0b6560 0x7fe5cc0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc110e80 tx=0x7fe5c800b410 comp rx=0 tx=0).stop 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 msgr2=0x7fe5dc19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 0x7fe5dc19c710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d770 tx=0x7fe5d000dc30 comp rx=0 tx=0).stop 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 shutdown_connections 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5cc0b6560 0x7fe5cc0b8a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5dc102fd0 0x7fe5dc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5dc110230 0x7fe5dc1a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 --2- 172.21.15.31:0/4069478345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5dc102680 0x7fe5dc19c710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 >> 172.21.15.31:0/4069478345 conn(0x7fe5dc0796d0 msgr2=0x7fe5dc109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 shutdown_connections 2024-09-21T02:50:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:19.500+0000 7fe5e2ddc640 1 -- 172.21.15.31:0/4069478345 wait complete. 2024-09-21T02:50:19.792 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:19 smithi165 ceph-mon[34818]: Deploying daemon mds.e on smithi165 2024-09-21T02:50:19.792 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:19 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.31:6836/3179894266,v1:172.21.15.31:6837/3179894266] up:boot 2024-09-21T02:50:19.792 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:19 smithi165 ceph-mon[34818]: fsmap 4 up:standby 2024-09-21T02:50:19.792 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:19 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:50:19.793 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:19 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/4069478345' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:19 smithi031 ceph-mon[31209]: Deploying daemon mds.e on smithi165 2024-09-21T02:50:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:19 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.31:6836/3179894266,v1:172.21.15.31:6837/3179894266] up:boot 2024-09-21T02:50:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:19 smithi031 ceph-mon[31209]: fsmap 4 up:standby 2024-09-21T02:50:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:19 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:50:19.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:19 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4069478345' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:19.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:19 smithi178 ceph-mon[34759]: Deploying daemon mds.e on smithi165 2024-09-21T02:50:19.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:19 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.31:6836/3179894266,v1:172.21.15.31:6837/3179894266] up:boot 2024-09-21T02:50:19.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:19 smithi178 ceph-mon[34759]: fsmap 4 up:standby 2024-09-21T02:50:19.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:19 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:50:19.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:19 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/4069478345' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:20.187 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:19 smithi165 systemd[1]: Starting Ceph mds.e for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:20.441 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 podman[63197]: 2024-09-21 02:50:20.088895562 +0000 UTC m=+0.015646701 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:20.441 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 podman[63197]: 2024-09-21 02:50:20.206302072 +0000 UTC m=+0.133053236 container create 34ee203c202183ac7c48112688cc74f553e90e0974e57e849ebbba0c7c659984 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:50:20.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:20 smithi165 ceph-mon[34818]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:20.941 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 podman[63197]: 2024-09-21 02:50:20.450035097 +0000 UTC m=+0.376786216 container init 34ee203c202183ac7c48112688cc74f553e90e0974e57e849ebbba0c7c659984 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:50:20.942 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 podman[63197]: 2024-09-21 02:50:20.453091238 +0000 UTC m=+0.379842375 container start 34ee203c202183ac7c48112688cc74f553e90e0974e57e849ebbba0c7c659984 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T02:50:20.942 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 ceph-mds[63216]: -- 172.21.15.165:0/4117190911 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558bc9449860 con 0x558bcd5f1180 2024-09-21T02:50:20.942 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e[63212]: starting mds.e at 2024-09-21T02:50:20.942 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 bash[63197]: 34ee203c202183ac7c48112688cc74f553e90e0974e57e849ebbba0c7c659984 2024-09-21T02:50:20.942 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 02:50:20 smithi165 systemd[1]: Started Ceph mds.e for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:20 smithi178 ceph-mon[34759]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:20 smithi031 ceph-mon[31209]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:21.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:21.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] up:boot 2024-09-21T02:50:21.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: fsmap 5 up:standby 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:21.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:21 smithi165 ceph-mon[34818]: Deploying daemon mds.f on smithi178 2024-09-21T02:50:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] up:boot 2024-09-21T02:50:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: fsmap 5 up:standby 2024-09-21T02:50:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:50:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:21.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:21.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:21.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:21.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:21 smithi178 ceph-mon[34759]: Deploying daemon mds.f on smithi178 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] up:boot 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: fsmap 5 up:standby 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:22.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:21 smithi031 ceph-mon[31209]: Deploying daemon mds.f on smithi178 2024-09-21T02:50:22.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:22.530 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:22.620 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:22 smithi178 systemd[1]: Starting Ceph mds.f for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:22.883 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:22 smithi178 ceph-mon[34759]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:22.884 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:22 smithi178 podman[63658]: 2024-09-21 02:50:22.624578504 +0000 UTC m=+0.016367126 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:22.884 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:22 smithi178 podman[63658]: 2024-09-21 02:50:22.731275057 +0000 UTC m=+0.123063679 container create 3de6889f53ff89eaac1143c8aa72ad8944b584b73b5e4325e6a80c98ca400de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:50:22.908 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:22 smithi165 ceph-mon[34818]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:22.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:22 smithi031 ceph-mon[31209]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 -- 172.21.15.31:0/1534881250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc103b80 msgr2=0x7f01fc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 --2- 172.21.15.31:0/1534881250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc103b80 0x7f01fc103f90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009900 tx=0x7f01ec02a410 comp rx=0 tx=0).stop 2024-09-21T02:50:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 -- 172.21.15.31:0/1534881250 shutdown_connections 2024-09-21T02:50:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 --2- 172.21.15.31:0/1534881250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 0x7f01fc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 --2- 172.21.15.31:0/1534881250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01fc1094c0 0x7f01fc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 --2- 172.21.15.31:0/1534881250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc103b80 0x7f01fc103f90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.049+0000 7f02024b4640 1 -- 172.21.15.31:0/1534881250 >> 172.21.15.31:0/1534881250 conn(0x7f01fc0ff5d0 msgr2=0x7f01fc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.050+0000 7f02024b4640 1 -- 172.21.15.31:0/1534881250 shutdown_connections 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.050+0000 7f02024b4640 1 -- 172.21.15.31:0/1534881250 wait complete. 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.050+0000 7f02024b4640 1 Processor -- start 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 -- start start 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01fc103b80 0x7f01fc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 0x7f01fc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 0x7f01fc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc113ad0 con 0x7f01fc1094c0 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc113950 con 0x7f01fc103b80 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f02024b4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc113c50 con 0x7f01fc109e10 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f01fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 0x7f01fc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f01fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 0x7f01fc19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43508/0 (socket says 172.21.15.31:43508) 2024-09-21T02:50:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f01fb7fe640 1 -- 172.21.15.31:0/318102649 learned_addr learned my addr 172.21.15.31:0/318102649 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.051+0000 7f01fbfff640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01fc103b80 0x7f01fc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f0200a51640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 0x7f01fc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01fb7fe640 1 -- 172.21.15.31:0/318102649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 msgr2=0x7f01fc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01fb7fe640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 0x7f01fc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01fb7fe640 1 -- 172.21.15.31:0/318102649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01fc103b80 msgr2=0x7f01fc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01fb7fe640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01fc103b80 0x7f01fc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01fb7fe640 1 -- 172.21.15.31:0/318102649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc1a0410 con 0x7f01fc1094c0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f0200a51640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 0x7f01fc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01fb7fe640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 0x7f01fc19aa40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01f000d740 tx=0x7f01f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0017070 con 0x7f01fc1094c0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc1a0700 con 0x7f01fc1094c0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f000de50 con 0x7f01fc1094c0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f000f620 con 0x7f01fc1094c0 2024-09-21T02:50:23.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.052+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc1a6e60 con 0x7f01fc1094c0 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.053+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc103f90 con 0x7f01fc1094c0 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.055+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f01f001a070 con 0x7f01fc1094c0 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.056+0000 7f01f97fa640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f01e40b6560 0x7f01e40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.056+0000 7f01fbfff640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f01e40b6560 0x7f01e40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.056+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f01f00e6960 con 0x7f01fc1094c0 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.056+0000 7f01fbfff640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f01e40b6560 0x7f01e40b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01ec031440 tx=0x7f01ec009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:23.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.059+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01f00ad5c0 con 0x7f01fc1094c0 2024-09-21T02:50:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.289+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01fc19b5d0 con 0x7f01fc1094c0 2024-09-21T02:50:23.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.289+0000 7f01f97fa640 1 -- 172.21.15.31:0/318102649 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7f01fc19b5d0 con 0x7f01fc1094c0 2024-09-21T02:50:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:23.291 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"btime":"2024-09-21T02:50:20:587802+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T02:50:23.291 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-09-21T02:50:23.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.291+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f01e40b6560 msgr2=0x7f01e40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:23.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.291+0000 7f02024b4640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f01e40b6560 0x7f01e40b8a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01ec031440 tx=0x7f01ec009560 comp rx=0 tx=0).stop 2024-09-21T02:50:23.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.291+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 msgr2=0x7f01fc19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:23.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.291+0000 7f02024b4640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 0x7f01fc19aa40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01f000d740 tx=0x7f01f000dc00 comp rx=0 tx=0).stop 2024-09-21T02:50:23.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 shutdown_connections 2024-09-21T02:50:23.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f01e40b6560 0x7f01e40b8a70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01fc109e10 0x7f01fc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01fc1094c0 0x7f01fc19aa40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 --2- 172.21.15.31:0/318102649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01fc103b80 0x7f01fc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 >> 172.21.15.31:0/318102649 conn(0x7f01fc0ff5d0 msgr2=0x7f01fc1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 shutdown_connections 2024-09-21T02:50:23.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:23.292+0000 7f02024b4640 1 -- 172.21.15.31:0/318102649 wait complete. 2024-09-21T02:50:23.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:23 smithi178 podman[63658]: 2024-09-21 02:50:23.001319597 +0000 UTC m=+0.393108217 container init 3de6889f53ff89eaac1143c8aa72ad8944b584b73b5e4325e6a80c98ca400de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T02:50:23.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:23 smithi178 podman[63658]: 2024-09-21 02:50:23.004343921 +0000 UTC m=+0.396132543 container start 3de6889f53ff89eaac1143c8aa72ad8944b584b73b5e4325e6a80c98ca400de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-21T02:50:23.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:23 smithi178 ceph-mds[63677]: -- 172.21.15.178:0/1694725971 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9cc671c20 con 0x55d9d0819a80 2024-09-21T02:50:23.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f[63673]: starting mds.f at 2024-09-21T02:50:23.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:23 smithi178 bash[63658]: 3de6889f53ff89eaac1143c8aa72ad8944b584b73b5e4325e6a80c98ca400de2 2024-09-21T02:50:23.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 02:50:23 smithi178 systemd[1]: Started Ceph mds.f for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:24.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:24.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:24.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:24.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:24.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: Deploying daemon mds.c on smithi178 2024-09-21T02:50:24.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/318102649' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:24.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6834/178089911,v1:172.21.15.178:6835/178089911] up:boot 2024-09-21T02:50:24.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: fsmap 6 up:standby 2024-09-21T02:50:24.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:24 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:24.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:24.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: Deploying daemon mds.c on smithi178 2024-09-21T02:50:24.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/318102649' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:24.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6834/178089911,v1:172.21.15.178:6835/178089911] up:boot 2024-09-21T02:50:24.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: fsmap 6 up:standby 2024-09-21T02:50:24.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:24 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:50:24.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:24.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:24.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:24.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:24.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:24.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: Deploying daemon mds.c on smithi178 2024-09-21T02:50:24.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/318102649' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:24.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6834/178089911,v1:172.21.15.178:6835/178089911] up:boot 2024-09-21T02:50:24.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: fsmap 6 up:standby 2024-09-21T02:50:24.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:24 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:50:25.228 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 systemd[1]: Starting Ceph mds.c for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:25.544 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 podman[63926]: 2024-09-21 02:50:25.164568903 +0000 UTC m=+0.015947671 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:25.544 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 podman[63926]: 2024-09-21 02:50:25.271408114 +0000 UTC m=+0.122786935 container create abf086330aad1b541846e81e1044cc9bf87460a869d5bd2fd46bacac7dfec62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:50:25.544 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 podman[63926]: 2024-09-21 02:50:25.507756157 +0000 UTC m=+0.359134916 container init abf086330aad1b541846e81e1044cc9bf87460a869d5bd2fd46bacac7dfec62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T02:50:25.544 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 podman[63926]: 2024-09-21 02:50:25.510485847 +0000 UTC m=+0.361864616 container start abf086330aad1b541846e81e1044cc9bf87460a869d5bd2fd46bacac7dfec62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:50:25.978 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 ceph-mds[63945]: -- 172.21.15.178:0/1610424159 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56363e705860 con 0x5636428ad180 2024-09-21T02:50:25.978 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c[63941]: starting mds.c at 2024-09-21T02:50:25.978 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 bash[63926]: abf086330aad1b541846e81e1044cc9bf87460a869d5bd2fd46bacac7dfec62b 2024-09-21T02:50:25.978 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 02:50:25 smithi178 systemd[1]: Started Ceph mds.c for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:25.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:26.252 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:26.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: Deploying daemon mds.h on smithi165 2024-09-21T02:50:26.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] up:boot 2024-09-21T02:50:26.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: fsmap 7 up:standby 2024-09-21T02:50:26.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:26 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:50:26.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:26.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:26.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:26.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:26.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: Deploying daemon mds.h on smithi165 2024-09-21T02:50:26.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] up:boot 2024-09-21T02:50:26.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: fsmap 7 up:standby 2024-09-21T02:50:26.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:26 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:50:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 -- 172.21.15.31:0/3304400980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 msgr2=0x7fa984112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/3304400980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa984112620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009900 tx=0x7fa97c02a480 comp rx=0 tx=0).stop 2024-09-21T02:50:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 -- 172.21.15.31:0/3304400980 shutdown_connections 2024-09-21T02:50:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/3304400980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984102fa0 0x7fa98410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/3304400980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa984112620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/3304400980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 0x7fa984102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 -- 172.21.15.31:0/3304400980 >> 172.21.15.31:0/3304400980 conn(0x7fa984079760 msgr2=0x7fa984079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 -- 172.21.15.31:0/3304400980 shutdown_connections 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 -- 172.21.15.31:0/3304400980 wait complete. 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 Processor -- start 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.970+0000 7fa98b4f1640 1 -- start start 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98b4f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 0x7fa98419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98b4f1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984102fa0 0x7fa98419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98b4f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa9841a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98b4f1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa984115ca0 con 0x7fa984110200 2024-09-21T02:50:26.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98b4f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa984115b20 con 0x7fa984102650 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98b4f1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa984115e20 con 0x7fa984102fa0 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa9841a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98928d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 0x7fa98419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa9841a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43516/0 (socket says 172.21.15.31:43516) 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 -- 172.21.15.31:0/1906251624 learned_addr learned my addr 172.21.15.31:0/1906251624 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa988a8c640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984102fa0 0x7fa98419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 -- 172.21.15.31:0/1906251624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984102fa0 msgr2=0x7fa98419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984102fa0 0x7fa98419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 -- 172.21.15.31:0/1906251624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 msgr2=0x7fa98419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 0x7fa98419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa989a8e640 1 -- 172.21.15.31:0/1906251624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9841a25c0 con 0x7fa984110200 2024-09-21T02:50:26.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.971+0000 7fa98928d640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 0x7fa98419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:50:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.972+0000 7fa989a8e640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa9841a1e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa97c02a9f0 tx=0x7fa97c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.972+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c03f070 con 0x7fa984110200 2024-09-21T02:50:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.972+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9841a2850 con 0x7fa984110200 2024-09-21T02:50:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.972+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa97c031b50 con 0x7fa984110200 2024-09-21T02:50:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.972+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c035030 con 0x7fa984110200 2024-09-21T02:50:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.972+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9841a9010 con 0x7fa984110200 2024-09-21T02:50:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.973+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98406c1f0 con 0x7fa984110200 2024-09-21T02:50:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.975+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fa97c042070 con 0x7fa984110200 2024-09-21T02:50:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.975+0000 7fa9767fc640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9700b65a0 0x7fa9700b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.975+0000 7fa98928d640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9700b65a0 0x7fa9700b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.976+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fa97c107f10 con 0x7fa984110200 2024-09-21T02:50:26.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.976+0000 7fa98928d640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9700b65a0 0x7fa9700b8ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa980005fd0 tx=0x7fa980009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:26.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: Deploying daemon mds.h on smithi165 2024-09-21T02:50:26.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] up:boot 2024-09-21T02:50:26.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: fsmap 7 up:standby 2024-09-21T02:50:26.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:26 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:50:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:26.979+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa97c0d2f50 con 0x7fa984110200 2024-09-21T02:50:27.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.206+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa984102a60 con 0x7fa984110200 2024-09-21T02:50:27.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.206+0000 7fa9767fc640 1 -- 172.21.15.31:0/1906251624 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7fa984102a60 con 0x7fa984110200 2024-09-21T02:50:27.208 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:27.208 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"btime":"2024-09-21T02:50:26:177991+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T02:50:27.208 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9700b65a0 msgr2=0x7fa9700b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9700b65a0 0x7fa9700b8ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa980005fd0 tx=0x7fa980009500 comp rx=0 tx=0).stop 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 msgr2=0x7fa9841a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa9841a1e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa97c02a9f0 tx=0x7fa97c003c00 comp rx=0 tx=0).stop 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 shutdown_connections 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa9700b65a0 0x7fa9700b8ab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa984102fa0 0x7fa98419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa984110200 0x7fa9841a1e90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 --2- 172.21.15.31:0/1906251624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa984102650 0x7fa98419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 >> 172.21.15.31:0/1906251624 conn(0x7fa984079760 msgr2=0x7fa984109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:27.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 shutdown_connections 2024-09-21T02:50:27.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:27.208+0000 7fa98b4f1640 1 -- 172.21.15.31:0/1906251624 wait complete. 2024-09-21T02:50:27.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:27 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1906251624' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:27.899 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:27 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1906251624' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:27.899 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:27 smithi165 systemd[1]: Starting Ceph mds.h for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:27.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:27 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1906251624' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:28.172 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:27 smithi165 podman[63482]: 2024-09-21 02:50:27.800964046 +0000 UTC m=+0.015864956 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:28.172 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:27 smithi165 podman[63482]: 2024-09-21 02:50:27.931800072 +0000 UTC m=+0.146700974 container create 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T02:50:28.172 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:28 smithi165 podman[63482]: 2024-09-21 02:50:28.171682292 +0000 UTC m=+0.386583194 container init 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T02:50:28.441 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:28 smithi165 podman[63482]: 2024-09-21 02:50:28.174446195 +0000 UTC m=+0.389347099 container start 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T02:50:28.441 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mds[63501]: -- 172.21.15.165:0/2109275198 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f19dd9860 con 0x556f1df81600 2024-09-21T02:50:28.441 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h[63497]: starting mds.h at 2024-09-21T02:50:28.441 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:28 smithi165 bash[63482]: 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e 2024-09-21T02:50:28.442 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 02:50:28 smithi165 systemd[1]: Started Ceph mds.h for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:28 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:28 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:28 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:29.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:29.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:29 smithi165 ceph-mon[34818]: Deploying daemon mds.i on smithi178 2024-09-21T02:50:29.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:29 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] up:boot 2024-09-21T02:50:29.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:29 smithi165 ceph-mon[34818]: fsmap 8 up:standby 2024-09-21T02:50:29.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:29 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:50:29.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:29 smithi031 ceph-mon[31209]: Deploying daemon mds.i on smithi178 2024-09-21T02:50:29.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:29 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] up:boot 2024-09-21T02:50:29.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:29 smithi031 ceph-mon[31209]: fsmap 8 up:standby 2024-09-21T02:50:29.956 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:29 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:50:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:29 smithi178 ceph-mon[34759]: Deploying daemon mds.i on smithi178 2024-09-21T02:50:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:29 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] up:boot 2024-09-21T02:50:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:29 smithi178 ceph-mon[34759]: fsmap 8 up:standby 2024-09-21T02:50:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:29 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:50:30.197 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:30.454 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 systemd[1]: Starting Ceph mds.i for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:30.711 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:30 smithi178 ceph-mon[34759]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:30.712 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 podman[64206]: 2024-09-21 02:50:30.355824205 +0000 UTC m=+0.015836485 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:30.712 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 podman[64206]: 2024-09-21 02:50:30.461837862 +0000 UTC m=+0.121850128 container create 17789cc7aef87995d915eb06482f4f947c781650ae6f820db3ac3627ae2b9d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T02:50:30.712 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 podman[64206]: 2024-09-21 02:50:30.710983769 +0000 UTC m=+0.370996034 container init 17789cc7aef87995d915eb06482f4f947c781650ae6f820db3ac3627ae2b9d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 -- 172.21.15.31:0/647896480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4102aa0 msgr2=0x7fd9d4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/647896480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4102eb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009900 tx=0x7fd9c402a410 comp rx=0 tx=0).stop 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 -- 172.21.15.31:0/647896480 shutdown_connections 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/647896480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d41083e0 0x7fd9d41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/647896480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4108d30 0x7fd9d410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/647896480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4102eb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 -- 172.21.15.31:0/647896480 >> 172.21.15.31:0/647896480 conn(0x7fd9d40797e0 msgr2=0x7fd9d4079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.817+0000 7fd9db0f3640 1 -- 172.21.15.31:0/647896480 shutdown_connections 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.818+0000 7fd9db0f3640 1 -- 172.21.15.31:0/647896480 wait complete. 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.818+0000 7fd9db0f3640 1 Processor -- start 2024-09-21T02:50:30.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.818+0000 7fd9db0f3640 1 -- start start 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.818+0000 7fd9db0f3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9db0f3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d41083e0 0x7fd9d41047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9db0f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4108d30 0x7fd9d4104d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9db0f3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d41129a0 con 0x7fd9d4108d30 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9db0f3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4112820 con 0x7fd9d41083e0 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9db0f3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4112b20 con 0x7fd9d4102aa0 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4104290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54790/0 (socket says 172.21.15.31:54790) 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d9690640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4108d30 0x7fd9d4104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 -- 172.21.15.31:0/1543997799 learned_addr learned my addr 172.21.15.31:0/1543997799 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d3fff640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d41083e0 0x7fd9d41047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 -- 172.21.15.31:0/1543997799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d41083e0 msgr2=0x7fd9d41047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:30.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d41083e0 0x7fd9d41047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 -- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4108d30 msgr2=0x7fd9d4104d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4108d30 0x7fd9d4104d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 -- 172.21.15.31:0/1543997799 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d41a8760 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.819+0000 7fd9d8e8f640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4104290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4005940 tx=0x7fd9c4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.820+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c403f030 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.820+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d41a89f0 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.820+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c4003aa0 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.820+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4035070 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.820+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d41a8ed0 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.821+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d4102eb0 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.823+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fd9c4042030 con 0x7fd9d4102aa0 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.824+0000 7fd9d1ffb640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b6560 0x7fd9bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:30.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.824+0000 7fd9d3fff640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b6560 0x7fd9bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.824+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fd9c4107910 con 0x7fd9d4102aa0 2024-09-21T02:50:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.825+0000 7fd9d3fff640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b6560 0x7fd9bc0b8a70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4100b70 tx=0x7fd9c8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:30.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:30.826+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9c40d2930 con 0x7fd9d4102aa0 2024-09-21T02:50:30.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:30 smithi031 ceph-mon[31209]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:30.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:30 smithi165 ceph-mon[34818]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:30.978 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 podman[64206]: 2024-09-21 02:50:30.714151545 +0000 UTC m=+0.374163824 container start 17789cc7aef87995d915eb06482f4f947c781650ae6f820db3ac3627ae2b9d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T02:50:30.978 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 ceph-mds[64225]: -- 172.21.15.178:0/1428328966 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646e0bd1860 con 0x5646e4d79a80 2024-09-21T02:50:30.978 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i[64221]: starting mds.i at 2024-09-21T02:50:30.979 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 bash[64206]: 17789cc7aef87995d915eb06482f4f947c781650ae6f820db3ac3627ae2b9d12 2024-09-21T02:50:30.979 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 02:50:30 smithi178 systemd[1]: Started Ceph mds.i for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.069+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d4105590 con 0x7fd9d4102aa0 2024-09-21T02:50:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.070+0000 7fd9d1ffb640 1 -- 172.21.15.31:0/1543997799 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7fd9c40d7870 con 0x7fd9d4102aa0 2024-09-21T02:50:31.071 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:31.072 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"btime":"2024-09-21T02:50:28:669006+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T02:50:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b6560 msgr2=0x7fd9bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b6560 0x7fd9bc0b8a70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4100b70 tx=0x7fd9c8005f90 comp rx=0 tx=0).stop 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 msgr2=0x7fd9d4104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4104290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4005940 tx=0x7fd9c4004270 comp rx=0 tx=0).stop 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 shutdown_connections 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b6560 0x7fd9bc0b8a70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4108d30 0x7fd9d4104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d41083e0 0x7fd9d41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 --2- 172.21.15.31:0/1543997799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4102aa0 0x7fd9d4104290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 >> 172.21.15.31:0/1543997799 conn(0x7fd9d40797e0 msgr2=0x7fd9d4101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 shutdown_connections 2024-09-21T02:50:31.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:31.072+0000 7fd9db0f3640 1 -- 172.21.15.31:0/1543997799 wait complete. 2024-09-21T02:50:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: Deploying daemon mds.g on smithi031 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1543997799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] up:boot 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: fsmap 9 up:standby 2024-09-21T02:50:32.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:31 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: Deploying daemon mds.g on smithi031 2024-09-21T02:50:32.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1543997799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:32.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] up:boot 2024-09-21T02:50:32.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: fsmap 9 up:standby 2024-09-21T02:50:32.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:31 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: Deploying daemon mds.g on smithi031 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1543997799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] up:boot 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: fsmap 9 up:standby 2024-09-21T02:50:32.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:31 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:50:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:32 smithi165 ceph-mon[34818]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:33.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:32 smithi178 ceph-mon[34759]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:33.293 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:32 smithi031 ceph-mon[31209]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:33.834 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:33 smithi031 systemd[1]: Starting Ceph mds.g for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:34.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:34.307 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:33 smithi031 podman[73870]: 2024-09-21 02:50:33.898362407 +0000 UTC m=+0.017078371 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:34.307 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 podman[73870]: 2024-09-21 02:50:34.045220408 +0000 UTC m=+0.163936347 container create 49d3dbaf1837b15d1c710d452251c94111f1175e28239c3b81ef433165700a4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:50:34.307 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 podman[73870]: 2024-09-21 02:50:34.306740439 +0000 UTC m=+0.425456392 container init 49d3dbaf1837b15d1c710d452251c94111f1175e28239c3b81ef433165700a4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T02:50:34.515 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:34.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 podman[73870]: 2024-09-21 02:50:34.310136714 +0000 UTC m=+0.428852661 container start 49d3dbaf1837b15d1c710d452251c94111f1175e28239c3b81ef433165700a4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:50:34.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mds[73919]: -- 172.21.15.31:0/1945334207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5830d9860 con 0x55c587281180 2024-09-21T02:50:34.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g[73903]: starting mds.g at 2024-09-21T02:50:34.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 bash[73870]: 49d3dbaf1837b15d1c710d452251c94111f1175e28239c3b81ef433165700a4b 2024-09-21T02:50:34.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 02:50:34 smithi031 systemd[1]: Started Ceph mds.g for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.020+0000 7f947cff2640 1 -- 172.21.15.31:0/2379957357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94781094a0 msgr2=0x7f94781098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.020+0000 7f947cff2640 1 --2- 172.21.15.31:0/2379957357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94781094a0 0x7f94781098b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9460009900 tx=0x7f946002a410 comp rx=0 tx=0).stop 2024-09-21T02:50:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.020+0000 7f947cff2640 1 -- 172.21.15.31:0/2379957357 shutdown_connections 2024-09-21T02:50:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.020+0000 7f947cff2640 1 --2- 172.21.15.31:0/2379957357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 0x7f9478110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.020+0000 7f947cff2640 1 --2- 172.21.15.31:0/2379957357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94781094a0 0x7f94781098b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.020+0000 7f947cff2640 1 --2- 172.21.15.31:0/2379957357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9478103b60 0x7f9478103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.021+0000 7f947cff2640 1 -- 172.21.15.31:0/2379957357 >> 172.21.15.31:0/2379957357 conn(0x7f947807b850 msgr2=0x7f947807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.021+0000 7f947cff2640 1 -- 172.21.15.31:0/2379957357 shutdown_connections 2024-09-21T02:50:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.021+0000 7f947cff2640 1 -- 172.21.15.31:0/2379957357 wait complete. 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.021+0000 7f947cff2640 1 Processor -- start 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 -- start start 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 0x7f947819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94781094a0 0x7f947819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 0x7f947819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9478113ab0 con 0x7f9478103b60 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9478113930 con 0x7f94781094a0 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947cff2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9478113c30 con 0x7f9478109df0 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 0x7f947819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 0x7f947819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39634/0 (socket says 172.21.15.31:39634) 2024-09-21T02:50:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 -- 172.21.15.31:0/550076255 learned_addr learned my addr 172.21.15.31:0/550076255 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f9475d9b640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94781094a0 0x7f947819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f9476d9d640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 0x7f947819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 -- 172.21.15.31:0/550076255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 msgr2=0x7f947819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 0x7f947819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 -- 172.21.15.31:0/550076255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94781094a0 msgr2=0x7f947819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94781094a0 0x7f947819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 -- 172.21.15.31:0/550076255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94781a03f0 con 0x7f9478103b60 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f9476d9d640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 0x7f947819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.022+0000 7f947659c640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 0x7f947819a480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f946800d950 tx=0x7f946800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.023+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468017070 con 0x7f9478103b60 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.023+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9468009c30 con 0x7f9478103b60 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.023+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800f620 con 0x7f9478103b60 2024-09-21T02:50:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.023+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781a06e0 con 0x7f9478103b60 2024-09-21T02:50:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.023+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a6e40 con 0x7f9478103b60 2024-09-21T02:50:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.024+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9478103f70 con 0x7f9478103b60 2024-09-21T02:50:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.026+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f9468005030 con 0x7f9478103b60 2024-09-21T02:50:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.027+0000 7f946f7fe640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94640b6560 0x7f94640b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.027+0000 7f9475d9b640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94640b6560 0x7f94640b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.027+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f94680e6c50 con 0x7f9478103b60 2024-09-21T02:50:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.027+0000 7f9475d9b640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94640b6560 0x7f94640b8a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9460009230 tx=0x7f9460031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.030+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94680ad8b0 con 0x7f9478103b60 2024-09-21T02:50:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:35.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:35.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:34 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:35.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:34 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:34 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:35.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.269+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f947819b5b0 con 0x7f9478103b60 2024-09-21T02:50:35.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.269+0000 7f946f7fe640 1 -- 172.21.15.31:0/550076255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7f9468009de0 con 0x7f9478103b60 2024-09-21T02:50:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:35.271 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"btime":"2024-09-21T02:50:34:891098+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T02:50:35.272 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94640b6560 msgr2=0x7f94640b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94640b6560 0x7f94640b8a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9460009230 tx=0x7f9460031400 comp rx=0 tx=0).stop 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 msgr2=0x7f947819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 0x7f947819a480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f946800d950 tx=0x7f946800de10 comp rx=0 tx=0).stop 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 shutdown_connections 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94640b6560 0x7f94640b8a70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9478109df0 0x7f947819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94781094a0 0x7f947819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 --2- 172.21.15.31:0/550076255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9478103b60 0x7f947819a480 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 >> 172.21.15.31:0/550076255 conn(0x7f947807b850 msgr2=0x7f9478107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 shutdown_connections 2024-09-21T02:50:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:35.272+0000 7f947cff2640 1 -- 172.21.15.31:0/550076255 wait complete. 2024-09-21T02:50:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:35 smithi165 ceph-mon[34818]: Deploying daemon mds.a on smithi031 2024-09-21T02:50:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:35 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.31:6838/3653301851,v1:172.21.15.31:6839/3653301851] up:boot 2024-09-21T02:50:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:35 smithi165 ceph-mon[34818]: fsmap 10 up:standby 2024-09-21T02:50:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:35 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:50:36.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:35 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/550076255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:36.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:35 smithi178 ceph-mon[34759]: Deploying daemon mds.a on smithi031 2024-09-21T02:50:36.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:35 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.31:6838/3653301851,v1:172.21.15.31:6839/3653301851] up:boot 2024-09-21T02:50:36.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:35 smithi178 ceph-mon[34759]: fsmap 10 up:standby 2024-09-21T02:50:36.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:35 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:50:36.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:35 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/550076255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:36.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:35 smithi031 ceph-mon[31209]: Deploying daemon mds.a on smithi031 2024-09-21T02:50:36.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:35 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.31:6838/3653301851,v1:172.21.15.31:6839/3653301851] up:boot 2024-09-21T02:50:36.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:35 smithi031 ceph-mon[31209]: fsmap 10 up:standby 2024-09-21T02:50:36.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:35 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:50:36.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:35 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/550076255' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:36.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:36 smithi031 ceph-mon[31209]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:36.967 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:36 smithi031 systemd[1]: Starting Ceph mds.a for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:36 smithi165 ceph-mon[34818]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:37.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:36 smithi178 ceph-mon[34759]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:37.334 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 podman[74252]: 2024-09-21 02:50:37.084691054 +0000 UTC m=+0.016196030 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:37.334 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 podman[74252]: 2024-09-21 02:50:37.211912049 +0000 UTC m=+0.143417016 container create f3a818248bedad7a3aa3e9312367b6b0ee2c90d40fd2ed8e6aba78c1f5c9293e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:50:37.834 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 podman[74252]: 2024-09-21 02:50:37.448072392 +0000 UTC m=+0.379577368 container init f3a818248bedad7a3aa3e9312367b6b0ee2c90d40fd2ed8e6aba78c1f5c9293e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:50:37.834 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 podman[74252]: 2024-09-21 02:50:37.450978538 +0000 UTC m=+0.382483505 container start f3a818248bedad7a3aa3e9312367b6b0ee2c90d40fd2ed8e6aba78c1f5c9293e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T02:50:37.834 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 ceph-mds[74271]: -- 172.21.15.31:0/2765922910 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56012e3c7860 con 0x56013256f180 2024-09-21T02:50:37.834 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a[74267]: starting mds.a at 2024-09-21T02:50:37.834 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 bash[74252]: f3a818248bedad7a3aa3e9312367b6b0ee2c90d40fd2ed8e6aba78c1f5c9293e 2024-09-21T02:50:37.835 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 02:50:37 smithi031 systemd[1]: Started Ceph mds.a for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: Deploying daemon mds.b on smithi165 2024-09-21T02:50:38.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.31:6840/2144690594,v1:172.21.15.31:6841/2144690594] up:boot 2024-09-21T02:50:38.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: fsmap 11 up:standby 2024-09-21T02:50:38.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:38 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:50:38.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:38.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:38.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:38.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: Deploying daemon mds.b on smithi165 2024-09-21T02:50:38.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.31:6840/2144690594,v1:172.21.15.31:6841/2144690594] up:boot 2024-09-21T02:50:38.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: fsmap 11 up:standby 2024-09-21T02:50:38.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:38 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:50:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:50:39.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:50:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: Deploying daemon mds.b on smithi165 2024-09-21T02:50:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.31:6840/2144690594,v1:172.21.15.31:6841/2144690594] up:boot 2024-09-21T02:50:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: fsmap 11 up:standby 2024-09-21T02:50:39.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:38 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:50:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:39.691 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:39 smithi165 systemd[1]: Starting Ceph mds.b for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T02:50:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 -- 172.21.15.31:0/143799563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3901094a0 msgr2=0x7ff3901098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 --2- 172.21.15.31:0/143799563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3901094a0 0x7ff3901098b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff384009900 tx=0x7ff38402a410 comp rx=0 tx=0).stop 2024-09-21T02:50:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 -- 172.21.15.31:0/143799563 shutdown_connections 2024-09-21T02:50:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 --2- 172.21.15.31:0/143799563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff390109df0 0x7ff390110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 --2- 172.21.15.31:0/143799563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3901094a0 0x7ff3901098b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 --2- 172.21.15.31:0/143799563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390103b60 0x7ff390103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.770+0000 7ff396946640 1 -- 172.21.15.31:0/143799563 >> 172.21.15.31:0/143799563 conn(0x7ff39007b850 msgr2=0x7ff39007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.771+0000 7ff396946640 1 -- 172.21.15.31:0/143799563 shutdown_connections 2024-09-21T02:50:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.771+0000 7ff396946640 1 -- 172.21.15.31:0/143799563 wait complete. 2024-09-21T02:50:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.771+0000 7ff396946640 1 Processor -- start 2024-09-21T02:50:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.771+0000 7ff396946640 1 -- start start 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff396946640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390103b60 0x7ff39019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff396946640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3901094a0 0x7ff39019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff396946640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 0x7ff39019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff396946640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff390113ab0 con 0x7ff390109df0 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff396946640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff390113930 con 0x7ff3901094a0 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff396946640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff390113c30 con 0x7ff390103b60 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 0x7ff39019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 0x7ff39019fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58212/0 (socket says 172.21.15.31:58212) 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 -- 172.21.15.31:0/832837004 learned_addr learned my addr 172.21.15.31:0/832837004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff38f7fe640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3901094a0 0x7ff39019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff38ffff640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390103b60 0x7ff39019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 -- 172.21.15.31:0/832837004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390103b60 msgr2=0x7ff39019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390103b60 0x7ff39019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 -- 172.21.15.31:0/832837004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3901094a0 msgr2=0x7ff39019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3901094a0 0x7ff39019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.772+0000 7ff394ee3640 1 -- 172.21.15.31:0/832837004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3901a0510 con 0x7ff390109df0 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.773+0000 7ff394ee3640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 0x7ff39019fde0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff38800cad0 tx=0x7ff38800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.773+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800ae20 con 0x7ff390109df0 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.773+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff388005020 con 0x7ff390109df0 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.773+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3901a0800 con 0x7ff390109df0 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.773+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800f6b0 con 0x7ff390109df0 2024-09-21T02:50:39.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.773+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3901a6f60 con 0x7ff390109df0 2024-09-21T02:50:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.774+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff390103f70 con 0x7ff390109df0 2024-09-21T02:50:39.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.776+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7ff3880051d0 con 0x7ff390109df0 2024-09-21T02:50:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.777+0000 7ff38d7fa640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3780b6560 0x7ff3780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.777+0000 7ff38ffff640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3780b6560 0x7ff3780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:39.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.777+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7ff3880e7280 con 0x7ff390109df0 2024-09-21T02:50:39.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.777+0000 7ff38ffff640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3780b6560 0x7ff3780b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff380005fd0 tx=0x7ff38000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:39.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:39.780+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff388014030 con 0x7ff390109df0 2024-09-21T02:50:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.008+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff39019b6d0 con 0x7ff390109df0 2024-09-21T02:50:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.008+0000 7ff38d7fa640 1 -- 172.21.15.31:0/832837004 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9704 (secure 0 0 0) 0x7ff39019b6d0 con 0x7ff390109df0 2024-09-21T02:50:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:40.010 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"btime":"2024-09-21T02:50:37:905566+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T02:50:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3780b6560 msgr2=0x7ff3780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3780b6560 0x7ff3780b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff380005fd0 tx=0x7ff38000b390 comp rx=0 tx=0).stop 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 msgr2=0x7ff39019fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 0x7ff39019fde0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff38800cad0 tx=0x7ff38800cf90 comp rx=0 tx=0).stop 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 shutdown_connections 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3780b6560 0x7ff3780b8a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff390109df0 0x7ff39019fde0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3901094a0 0x7ff39019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 --2- 172.21.15.31:0/832837004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390103b60 0x7ff39019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.011+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 >> 172.21.15.31:0/832837004 conn(0x7ff39007b850 msgr2=0x7ff390107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.012+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 shutdown_connections 2024-09-21T02:50:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:40.012+0000 7ff396946640 1 -- 172.21.15.31:0/832837004 wait complete. 2024-09-21T02:50:40.051 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:39 smithi165 podman[63752]: 2024-09-21 02:50:39.68589365 +0000 UTC m=+0.016123815 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:50:40.051 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:39 smithi165 podman[63752]: 2024-09-21 02:50:39.806805294 +0000 UTC m=+0.137035447 container create 49ce0d5da4b8b51b53caf19c9fe9f29e9367f2e2c6c2fdc3bae5ca3ffb91c616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T02:50:40.052 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:40 smithi165 podman[63752]: 2024-09-21 02:50:40.051022322 +0000 UTC m=+0.381252489 container init 49ce0d5da4b8b51b53caf19c9fe9f29e9367f2e2c6c2fdc3bae5ca3ffb91c616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:50:40.441 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:40 smithi165 podman[63752]: 2024-09-21 02:50:40.05383678 +0000 UTC m=+0.384066943 container start 49ce0d5da4b8b51b53caf19c9fe9f29e9367f2e2c6c2fdc3bae5ca3ffb91c616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T02:50:40.441 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mds[63771]: -- 172.21.15.165:0/3793747270 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5652a195b860 con 0x5652a5b03a80 2024-09-21T02:50:40.441 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: starting mds.b at 2024-09-21T02:50:40.441 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:40 smithi165 bash[63752]: 49ce0d5da4b8b51b53caf19c9fe9f29e9367f2e2c6c2fdc3bae5ca3ffb91c616 2024-09-21T02:50:40.441 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:40 smithi165 systemd[1]: Started Ceph mds.b for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T02:50:40.669 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:40.669 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/832837004' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:40.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:40 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/832837004' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:40.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:40 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/832837004' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:40.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:40.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:40 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:41.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:41 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] up:boot 2024-09-21T02:50:41.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:41 smithi165 ceph-mon[34818]: fsmap 12 up:standby 2024-09-21T02:50:41.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:41 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:50:41.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:41 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] up:boot 2024-09-21T02:50:41.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:41 smithi178 ceph-mon[34759]: fsmap 12 up:standby 2024-09-21T02:50:41.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:41 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:50:42.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:41 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] up:boot 2024-09-21T02:50:42.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:41 smithi031 ceph-mon[31209]: fsmap 12 up:standby 2024-09-21T02:50:42.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:41 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:50:42.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:50:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:42.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:42.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:42 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:42.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:42.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:42 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:42.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:42.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:42.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:42 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:43.070 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 -- 172.21.15.31:0/1167965258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc01094a0 msgr2=0x7fbdc01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/1167965258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc01094a0 0x7fbdc01098b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009900 tx=0x7fbdb402a410 comp rx=0 tx=0).stop 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 -- 172.21.15.31:0/1167965258 shutdown_connections 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/1167965258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc0109df0 0x7fbdc0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/1167965258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc01094a0 0x7fbdc01098b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/1167965258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc0103b60 0x7fbdc0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 -- 172.21.15.31:0/1167965258 >> 172.21.15.31:0/1167965258 conn(0x7fbdc007b850 msgr2=0x7fbdc007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 -- 172.21.15.31:0/1167965258 shutdown_connections 2024-09-21T02:50:43.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 -- 172.21.15.31:0/1167965258 wait complete. 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.583+0000 7fbdc69fb640 1 Processor -- start 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 -- start start 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc0103b60 0x7fbdc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc01094a0 0x7fbdc0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 0x7fbdc0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc0113ab0 con 0x7fbdc0109df0 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc0113930 con 0x7fbdc01094a0 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc69fb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc0113c30 con 0x7fbdc0103b60 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 0x7fbdc0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 0x7fbdc0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58230/0 (socket says 172.21.15.31:58230) 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 -- 172.21.15.31:0/3935300297 learned_addr learned my addr 172.21.15.31:0/3935300297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdbffff640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc0103b60 0x7fbdc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:43.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdbf7fe640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc01094a0 0x7fbdc0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 -- 172.21.15.31:0/3935300297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc0103b60 msgr2=0x7fbdc0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc0103b60 0x7fbdc0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 -- 172.21.15.31:0/3935300297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc01094a0 msgr2=0x7fbdc0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc01094a0 0x7fbdc0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdc4f98640 1 -- 172.21.15.31:0/3935300297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc01a8780 con 0x7fbdc0109df0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.584+0000 7fbdbf7fe640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc01094a0 0x7fbdc0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.585+0000 7fbdc4f98640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 0x7fbdc0076190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009fd0 tx=0x7fbdb800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.585+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8017070 con 0x7fbdc0109df0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.585+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb8003e20 con 0x7fbdc0109df0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.585+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc01a8a70 con 0x7fbdc0109df0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.585+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8010460 con 0x7fbdc0109df0 2024-09-21T02:50:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.585+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc01a8f30 con 0x7fbdc0109df0 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.586+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc0103f70 con 0x7fbdc0109df0 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.587+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fbdb8003970 con 0x7fbdc0109df0 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.589+0000 7fbdbd7fa640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbda80b6560 0x7fbda80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.589+0000 7fbdbffff640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbda80b6560 0x7fbda80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.589+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fbdb80e6db0 con 0x7fbdc0109df0 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.589+0000 7fbdbffff640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbda80b6560 0x7fbda80b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0077160 tx=0x7fbdb0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.592+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbdb80ad9b0 con 0x7fbdc0109df0 2024-09-21T02:50:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.851+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdc0076a10 con 0x7fbdc0109df0 2024-09-21T02:50:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.852+0000 7fbdbd7fa640 1 -- 172.21.15.31:0/3935300297 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7fbdc0076a10 con 0x7fbdc0109df0 2024-09-21T02:50:43.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:43.854 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"btime":"2024-09-21T02:50:40:605177+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24583,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24625,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24632,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24655,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24659,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24695,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T02:50:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-09-21T02:50:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbda80b6560 msgr2=0x7fbda80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbda80b6560 0x7fbda80b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0077160 tx=0x7fbdb0005f90 comp rx=0 tx=0).stop 2024-09-21T02:50:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 msgr2=0x7fbdc0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 0x7fbdc0076190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009fd0 tx=0x7fbdb800b260 comp rx=0 tx=0).stop 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 shutdown_connections 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbda80b6560 0x7fbda80b8a70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdc0109df0 0x7fbdc0076190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc01094a0 0x7fbdc0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 --2- 172.21.15.31:0/3935300297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc0103b60 0x7fbdc0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 >> 172.21.15.31:0/3935300297 conn(0x7fbdc007b850 msgr2=0x7fbdc0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 shutdown_connections 2024-09-21T02:50:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:43.855+0000 7fbdc69fb640 1 -- 172.21.15.31:0/3935300297 wait complete. 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3935300297' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:44.498 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:44.499 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T02:50:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3935300297' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:44.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3935300297' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:50:44.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:50:44.787 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 -- 172.21.15.31:0/3932777063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c101ba0 msgr2=0x7f742c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 --2- 172.21.15.31:0/3932777063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c101ba0 0x7f742c101fb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f741c009900 tx=0x7f741c02a410 comp rx=0 tx=0).stop 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 -- 172.21.15.31:0/3932777063 shutdown_connections 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 --2- 172.21.15.31:0/3932777063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109570 0x7f742c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 --2- 172.21.15.31:0/3932777063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f742c109ec0 0x7f742c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 --2- 172.21.15.31:0/3932777063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c101ba0 0x7f742c101fb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 -- 172.21.15.31:0/3932777063 >> 172.21.15.31:0/3932777063 conn(0x7f742c077800 msgr2=0x7f742c077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 -- 172.21.15.31:0/3932777063 shutdown_connections 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.291+0000 7f7430878640 1 -- 172.21.15.31:0/3932777063 wait complete. 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 Processor -- start 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 -- start start 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f742c101ba0 0x7f742c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 0x7f742c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109ec0 0x7f742c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f742c113ab0 con 0x7f742c109570 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f742c113930 con 0x7f742c101ba0 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7430878640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f742c113c30 con 0x7f742c109ec0 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7429d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 0x7f742c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7429d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 0x7f742c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58252/0 (socket says 172.21.15.31:58252) 2024-09-21T02:50:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f7429d9b640 1 -- 172.21.15.31:0/1020638110 learned_addr learned my addr 172.21.15.31:0/1020638110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f742a59c640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f742c101ba0 0x7f742c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.292+0000 7f742ad9d640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109ec0 0x7f742c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7429d9b640 1 -- 172.21.15.31:0/1020638110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109ec0 msgr2=0x7f742c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7429d9b640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109ec0 0x7f742c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7429d9b640 1 -- 172.21.15.31:0/1020638110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f742c101ba0 msgr2=0x7f742c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7429d9b640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f742c101ba0 0x7f742c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7429d9b640 1 -- 172.21.15.31:0/1020638110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c1a0450 con 0x7f742c109570 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f742ad9d640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109ec0 0x7f742c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7429d9b640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 0x7f742c19aa80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f742000c970 tx=0x7f742000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004220 con 0x7f742c109570 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200043c0 con 0x7f742c109570 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c1a0740 con 0x7f742c109570 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004a00 con 0x7f742c109570 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.293+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1a6ea0 con 0x7f742c109570 2024-09-21T02:50:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.294+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c101fb0 con 0x7f742c109570 2024-09-21T02:50:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.296+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f74200079b0 con 0x7f742c109570 2024-09-21T02:50:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.296+0000 7f741b7fe640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74140b6560 0x7f74140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.296+0000 7f742a59c640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74140b6560 0x7f74140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.297+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f74200e7120 con 0x7f742c109570 2024-09-21T02:50:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.297+0000 7f742a59c640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74140b6560 0x7f74140b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f741c031440 tx=0x7f741c009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.299+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7420014030 con 0x7f742c109570 2024-09-21T02:50:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.491+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f742c19b610 con 0x7f742c109570 2024-09-21T02:50:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.492+0000 7f741b7fe640 1 -- 172.21.15.31:0/1020638110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f742c19b610 con 0x7f742c109570 2024-09-21T02:50:45.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:45.495 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-09-21T02:50:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T02:50:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-09-21T02:50:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-09-21T02:50:45.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:50:45.495 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74140b6560 msgr2=0x7f74140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74140b6560 0x7f74140b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f741c031440 tx=0x7f741c009560 comp rx=0 tx=0).stop 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 msgr2=0x7f742c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 0x7f742c19aa80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f742000c970 tx=0x7f742000ce30 comp rx=0 tx=0).stop 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 shutdown_connections 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74140b6560 0x7f74140b8a70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f742c109ec0 0x7f742c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f742c109570 0x7f742c19aa80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 --2- 172.21.15.31:0/1020638110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f742c101ba0 0x7f742c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 >> 172.21.15.31:0/1020638110 conn(0x7f742c077800 msgr2=0x7f742c100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 shutdown_connections 2024-09-21T02:50:45.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:45.494+0000 7f7430878640 1 -- 172.21.15.31:0/1020638110 wait complete. 2024-09-21T02:50:46.211 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:50:46.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:50:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:46.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:46 smithi031 ceph-mon[31209]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:46.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:46 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1020638110' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:50:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:46 smithi165 ceph-mon[34818]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:46 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1020638110' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:50:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:46 smithi178 ceph-mon[34759]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:46 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1020638110' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:50:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 -- 172.21.15.31:0/657942193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe774105190 msgr2=0x7fe7741055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/657942193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe774105190 0x7fe7741055a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe768009900 tx=0x7fe76802a410 comp rx=0 tx=0).stop 2024-09-21T02:50:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 -- 172.21.15.31:0/657942193 shutdown_connections 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/657942193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe774105ae0 0x7fe77410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/657942193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe774105190 0x7fe7741055a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/657942193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7741031c0 0x7fe7741035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 -- 172.21.15.31:0/657942193 >> 172.21.15.31:0/657942193 conn(0x7fe774077c60 msgr2=0x7fe7740780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 -- 172.21.15.31:0/657942193 shutdown_connections 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.971+0000 7fe77c5b6640 1 -- 172.21.15.31:0/657942193 wait complete. 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 Processor -- start 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 -- start start 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 0x7fe77419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe774105190 0x7fe77419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe774105ae0 0x7fe77419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe774113aa0 con 0x7fe7741031c0 2024-09-21T02:50:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe774113920 con 0x7fe774105ae0 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77c5b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe774113c20 con 0x7fe774105190 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77a352640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 0x7fe77419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77a352640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 0x7fe77419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58266/0 (socket says 172.21.15.31:58266) 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77a352640 1 -- 172.21.15.31:0/3136734388 learned_addr learned my addr 172.21.15.31:0/3136734388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe779b51640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe774105190 0x7fe77419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77ab53640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe774105ae0 0x7fe77419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.972+0000 7fe77a352640 1 -- 172.21.15.31:0/3136734388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe774105190 msgr2=0x7fe77419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77a352640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe774105190 0x7fe77419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77a352640 1 -- 172.21.15.31:0/3136734388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe774105ae0 msgr2=0x7fe77419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77a352640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe774105ae0 0x7fe77419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77a352640 1 -- 172.21.15.31:0/3136734388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741a0340 con 0x7fe7741031c0 2024-09-21T02:50:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77a352640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 0x7fe77419a490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c920 tx=0x7fe76c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c007c70 con 0x7fe7741031c0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a0630 con 0x7fe7741031c0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe76c007e10 con 0x7fe7741031c0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c018820 con 0x7fe7741031c0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.973+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a6e50 con 0x7fe7741031c0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.974+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7741035d0 con 0x7fe7741031c0 2024-09-21T02:50:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.977+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fe76c00f450 con 0x7fe7741031c0 2024-09-21T02:50:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.977+0000 7fe7737fe640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe75c0b6590 0x7fe75c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.977+0000 7fe779b51640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe75c0b6590 0x7fe75c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.977+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fe76c0e72b0 con 0x7fe7741031c0 2024-09-21T02:50:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.978+0000 7fe779b51640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe75c0b6590 0x7fe75c0b8aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe768000c00 tx=0x7fe768031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:46.981+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe76c014030 con 0x7fe7741031c0 2024-09-21T02:50:47.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.169+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe77410aaf0 con 0x7fe7741031c0 2024-09-21T02:50:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.315+0000 7fe7737fe640 1 -- 172.21.15.31:0/3136734388 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fe76c0adfa0 con 0x7fe7741031c0 2024-09-21T02:50:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-09-21T02:50:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.318+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe75c0b6590 msgr2=0x7fe75c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.318+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe75c0b6590 0x7fe75c0b8aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe768000c00 tx=0x7fe768031400 comp rx=0 tx=0).stop 2024-09-21T02:50:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.318+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 msgr2=0x7fe77419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.318+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 0x7fe77419a490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c920 tx=0x7fe76c00cde0 comp rx=0 tx=0).stop 2024-09-21T02:50:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 shutdown_connections 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe75c0b6590 0x7fe75c0b8aa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe774105190 0x7fe77419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe774105ae0 0x7fe77419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 --2- 172.21.15.31:0/3136734388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7741031c0 0x7fe77419a490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 >> 172.21.15.31:0/3136734388 conn(0x7fe774077c60 msgr2=0x7fe7741064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 shutdown_connections 2024-09-21T02:50:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:47.319+0000 7fe77c5b6640 1 -- 172.21.15.31:0/3136734388 wait complete. 2024-09-21T02:50:47.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:47 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3136734388' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:50:47.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:47 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3136734388' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:50:47.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:47 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3136734388' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:50:48.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:50:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:48.329 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:48 smithi031 ceph-mon[31209]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:48.329 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3136734388' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:50:48.329 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:48 smithi031 ceph-mon[31209]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:50:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:48 smithi165 ceph-mon[34818]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3136734388' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:50:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:48 smithi165 ceph-mon[34818]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:50:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:48 smithi178 ceph-mon[34759]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3136734388' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:50:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:48 smithi178 ceph-mon[34759]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:50:48.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 -- 172.21.15.31:0/2209609752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28102fa0 msgr2=0x7f0b2810fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:48.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2209609752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28102fa0 0x7f0b2810fcc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009900 tx=0x7f0b1802a480 comp rx=0 tx=0).stop 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 -- 172.21.15.31:0/2209609752 shutdown_connections 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2209609752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b28110200 0x7f0b28112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2209609752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28102fa0 0x7f0b2810fcc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2209609752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b28102650 0x7f0b28102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 -- 172.21.15.31:0/2209609752 >> 172.21.15.31:0/2209609752 conn(0x7f0b28079760 msgr2=0x7f0b28079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 -- 172.21.15.31:0/2209609752 shutdown_connections 2024-09-21T02:50:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.787+0000 7f0b301f4640 1 -- 172.21.15.31:0/2209609752 wait complete. 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 Processor -- start 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 -- start start 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b28102650 0x7f0b2819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b28102fa0 0x7f0b2819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 0x7f0b281a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28115ca0 con 0x7f0b28110200 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28115b20 con 0x7f0b28102fa0 2024-09-21T02:50:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b301f4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28115e20 con 0x7f0b28102650 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b2e791640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 0x7f0b281a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b2df90640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b28102650 0x7f0b2819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.788+0000 7f0b2e791640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 0x7f0b281a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58294/0 (socket says 172.21.15.31:58294) 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 -- 172.21.15.31:0/2686862043 learned_addr learned my addr 172.21.15.31:0/2686862043 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2d78f640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b28102fa0 0x7f0b2819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 -- 172.21.15.31:0/2686862043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b28102650 msgr2=0x7f0b2819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b28102650 0x7f0b2819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 -- 172.21.15.31:0/2686862043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b28102fa0 msgr2=0x7f0b2819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b28102fa0 0x7f0b2819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 -- 172.21.15.31:0/2686862043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b281a25c0 con 0x7f0b28110200 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b2e791640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 0x7f0b281a1e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000eaa0 tx=0x7f0b2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000cdb0 con 0x7f0b28110200 2024-09-21T02:50:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b20005020 con 0x7f0b28110200 2024-09-21T02:50:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b200116d0 con 0x7f0b28110200 2024-09-21T02:50:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b281a28b0 con 0x7f0b28110200 2024-09-21T02:50:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.789+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b281a9010 con 0x7f0b28110200 2024-09-21T02:50:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.790+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2806c1f0 con 0x7f0b28110200 2024-09-21T02:50:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.792+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0b200051d0 con 0x7f0b28110200 2024-09-21T02:50:48.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.793+0000 7f0b1effd640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b140b65e0 0x7f0b140b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:48.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.793+0000 7f0b2df90640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b140b65e0 0x7f0b140b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:48.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.794+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7712+0+0 (secure 0 0 0) 0x7f0b200e7580 con 0x7f0b28110200 2024-09-21T02:50:48.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.794+0000 7f0b2df90640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b140b65e0 0x7f0b140b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400bbe0 tx=0x7f0b2400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:48.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.797+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b20016030 con 0x7f0b28110200 2024-09-21T02:50:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:48.988+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0b28102a60 con 0x7f0b28110200 2024-09-21T02:50:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.331+0000 7f0b1effd640 1 -- 172.21.15.31:0/2686862043 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) ==== 170+0+0 (secure 0 0 0) 0x7f0b28102a60 con 0x7f0b28110200 2024-09-21T02:50:49.334 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b140b65e0 msgr2=0x7f0b140b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b140b65e0 0x7f0b140b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400bbe0 tx=0x7f0b2400b390 comp rx=0 tx=0).stop 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 msgr2=0x7f0b281a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 0x7f0b281a1e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000eaa0 tx=0x7f0b2000ef60 comp rx=0 tx=0).stop 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 shutdown_connections 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0b140b65e0 0x7f0b140b8af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b28110200 0x7f0b281a1e90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b28102fa0 0x7f0b2819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 --2- 172.21.15.31:0/2686862043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b28102650 0x7f0b2819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.334+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 >> 172.21.15.31:0/2686862043 conn(0x7f0b28079760 msgr2=0x7f0b28109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:49.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.335+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 shutdown_connections 2024-09-21T02:50:49.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:49.335+0000 7f0b301f4640 1 -- 172.21.15.31:0/2686862043 wait complete. 2024-09-21T02:50:49.660 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:49 smithi031 ceph-mon[31209]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:50:49.660 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:49 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2686862043' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:50:49.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:49 smithi165 ceph-mon[34818]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:50:49.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:49 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2686862043' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:50:49.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:49 smithi178 ceph-mon[34759]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:50:49.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:49 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2686862043' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:50:49.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:50:50.245 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:50.640 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:50 smithi031 ceph-mon[31209]: pgmap v258: 33 pgs: 1 creating+activating, 18 unknown, 14 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:50.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:50 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2686862043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:50:50.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:50 smithi031 ceph-mon[31209]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:50:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:50 smithi165 ceph-mon[34818]: pgmap v258: 33 pgs: 1 creating+activating, 18 unknown, 14 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:50 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2686862043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:50:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:50 smithi165 ceph-mon[34818]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:50:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:50 smithi178 ceph-mon[34759]: pgmap v258: 33 pgs: 1 creating+activating, 18 unknown, 14 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2686862043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:50:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:50 smithi178 ceph-mon[34759]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 -- 172.21.15.31:0/3171046972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c102ae0 msgr2=0x7ffa5c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 --2- 172.21.15.31:0/3171046972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c102ae0 0x7ffa5c102ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009900 tx=0x7ffa4c02a410 comp rx=0 tx=0).stop 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 -- 172.21.15.31:0/3171046972 shutdown_connections 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 --2- 172.21.15.31:0/3171046972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 --2- 172.21.15.31:0/3171046972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa5c108d20 0x7ffa5c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 --2- 172.21.15.31:0/3171046972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c102ae0 0x7ffa5c102ef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 -- 172.21.15.31:0/3171046972 >> 172.21.15.31:0/3171046972 conn(0x7ffa5c079760 msgr2=0x7ffa5c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 -- 172.21.15.31:0/3171046972 shutdown_connections 2024-09-21T02:50:50.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 -- 172.21.15.31:0/3171046972 wait complete. 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.734+0000 7ffa61503640 1 Processor -- start 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 -- start start 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c102ae0 0x7ffa5c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa5c108d20 0x7ffa5c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c06c670 con 0x7ffa5c1083d0 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c06c4f0 con 0x7ffa5c108d20 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa61503640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c06c7f0 con 0x7ffa5c102ae0 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48150/0 (socket says 172.21.15.31:48150) 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 -- 172.21.15.31:0/1421604038 learned_addr learned my addr 172.21.15.31:0/1421604038 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5b7fe640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa5c108d20 0x7ffa5c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5affd640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c102ae0 0x7ffa5c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 -- 172.21.15.31:0/1421604038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c102ae0 msgr2=0x7ffa5c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c102ae0 0x7ffa5c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 -- 172.21.15.31:0/1421604038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa5c108d20 msgr2=0x7ffa5c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa5c108d20 0x7ffa5c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.735+0000 7ffa5a7fc640 1 -- 172.21.15.31:0/1421604038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1a0400 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.736+0000 7ffa5a7fc640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c19aa30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400d740 tx=0x7ffa4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.736+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44017070 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.736+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1a06f0 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.736+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4400de50 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.736+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44004ce0 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.736+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1a6e50 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.737+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c103d10 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.739+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7ffa44004e80 con 0x7ffa5c1083d0 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.740+0000 7ffa53fff640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ffa480b6560 0x7ffa480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.740+0000 7ffa5affd640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ffa480b6560 0x7ffa480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.740+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8094+0+0 (secure 0 0 0) 0x7ffa440e7040 con 0x7ffa5c1083d0 2024-09-21T02:50:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.741+0000 7ffa5affd640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ffa480b6560 0x7ffa480b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c031440 tx=0x7ffa4c005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:50.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.743+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffa440ad9b0 con 0x7ffa5c1083d0 2024-09-21T02:50:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:50.971+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ffa5c19b5c0 con 0x7ffa5c1083d0 2024-09-21T02:50:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.352+0000 7ffa53fff640 1 -- 172.21.15.31:0/1421604038 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ffa5c19b5c0 con 0x7ffa5c1083d0 2024-09-21T02:50:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T02:50:51.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ffa480b6560 msgr2=0x7ffa480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ffa480b6560 0x7ffa480b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c031440 tx=0x7ffa4c005d50 comp rx=0 tx=0).stop 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 msgr2=0x7ffa5c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c19aa30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400d740 tx=0x7ffa4400dc00 comp rx=0 tx=0).stop 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 shutdown_connections 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ffa480b6560 0x7ffa480b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa5c108d20 0x7ffa5c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c1083d0 0x7ffa5c19aa30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 --2- 172.21.15.31:0/1421604038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa5c102ae0 0x7ffa5c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 >> 172.21.15.31:0/1421604038 conn(0x7ffa5c079760 msgr2=0x7ffa5c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:51.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 shutdown_connections 2024-09-21T02:50:51.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:51.355+0000 7ffa61503640 1 -- 172.21.15.31:0/1421604038 wait complete. 2024-09-21T02:50:51.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:51 smithi031 ceph-mon[31209]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:50:51.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:51 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1421604038' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:50:51.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:51 smithi031 ceph-mon[31209]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T02:50:51.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:51 smithi165 ceph-mon[34818]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:50:51.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:51 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1421604038' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:50:51.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:51 smithi165 ceph-mon[34818]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T02:50:51.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:51 smithi178 ceph-mon[34759]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:50:51.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:51 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1421604038' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:50:51.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:51 smithi178 ceph-mon[34759]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T02:50:52.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:50:52.322 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:52.357 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: pgmap v261: 97 pgs: 16 creating+peering, 1 creating+activating, 44 unknown, 36 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:52.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: pgmap v261: 97 pgs: 16 creating+peering, 1 creating+activating, 44 unknown, 36 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:52.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:50:52.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1421604038' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:50:52.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T02:50:52.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T02:50:52.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] up:active 2024-09-21T02:50:52.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:52 smithi031 ceph-mon[31209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:50:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1421604038' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:50:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T02:50:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T02:50:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] up:active 2024-09-21T02:50:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:52 smithi165 ceph-mon[34818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: pgmap v261: 97 pgs: 16 creating+peering, 1 creating+activating, 44 unknown, 36 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1421604038' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] up:active 2024-09-21T02:50:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:52 smithi178 ceph-mon[34759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:52.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.814+0000 7f94a5246640 1 -- 172.21.15.31:0/216125325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0109e10 msgr2=0x7f94a0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:52.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.814+0000 7f94a5246640 1 --2- 172.21.15.31:0/216125325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0109e10 0x7f94a0110640 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ca0 tx=0x7f949802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:50:52.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 -- 172.21.15.31:0/216125325 shutdown_connections 2024-09-21T02:50:52.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 --2- 172.21.15.31:0/216125325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0109e10 0x7f94a0110640 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 --2- 172.21.15.31:0/216125325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a01094c0 0x7f94a01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 --2- 172.21.15.31:0/216125325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 0x7f94a0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 -- 172.21.15.31:0/216125325 >> 172.21.15.31:0/216125325 conn(0x7f94a00ff5d0 msgr2=0x7f94a0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 -- 172.21.15.31:0/216125325 shutdown_connections 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 -- 172.21.15.31:0/216125325 wait complete. 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.815+0000 7f94a5246640 1 Processor -- start 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 -- start start 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 0x7f94a019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 0x7f94a019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a0109e10 0x7f94a019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0113ad0 con 0x7f94a01094c0 2024-09-21T02:50:52.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0113950 con 0x7f94a0109e10 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f94a5246640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0113c50 con 0x7f94a0103b80 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 0x7f94a019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 0x7f94a019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48176/0 (socket says 172.21.15.31:48176) 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 -- 172.21.15.31:0/2738209267 learned_addr learned my addr 172.21.15.31:0/2738209267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949ed9d640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 0x7f94a019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 -- 172.21.15.31:0/2738209267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 msgr2=0x7f94a019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 0x7f94a019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949f59e640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a0109e10 0x7f94a019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 -- 172.21.15.31:0/2738209267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a0109e10 msgr2=0x7f94a019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a0109e10 0x7f94a019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949e59c640 1 -- 172.21.15.31:0/2738209267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a01a03b0 con 0x7f94a01094c0 2024-09-21T02:50:52.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.816+0000 7f949ed9d640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 0x7f94a019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.817+0000 7f949e59c640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 0x7f94a019a9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9f0 tx=0x7f949400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.817+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949400ccc0 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.817+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94940108e0 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.817+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a01a06a0 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.817+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494018860 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.817+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01a6e00 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.818+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a0103f90 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.820+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f9494010430 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.820+0000 7f9487fff640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94880b65e0 0x7f94880b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.820+0000 7f949ed9d640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94880b65e0 0x7f94880b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.821+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f94940e7ba0 con 0x7f94a01094c0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.821+0000 7f949ed9d640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94880b65e0 0x7f94880b8af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9490009b40 tx=0x7f9490005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:52.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:52.824+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94940ae450 con 0x7f94a01094c0 2024-09-21T02:50:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.012+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94a019b570 con 0x7f94a01094c0 2024-09-21T02:50:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.013+0000 7f9487fff640 1 -- 172.21.15.31:0/2738209267 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22617 (secure 0 0 0) 0x7f94a019b570 con 0x7f94a01094c0 2024-09-21T02:50:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:53.016 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:50:50.973567+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:50:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.015+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94880b65e0 msgr2=0x7f94880b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.015+0000 7f94a5246640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94880b65e0 0x7f94880b8af0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9490009b40 tx=0x7f9490005e50 comp rx=0 tx=0).stop 2024-09-21T02:50:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.015+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 msgr2=0x7f94a019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 0x7f94a019a9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9f0 tx=0x7f949400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:50:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 shutdown_connections 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94880b65e0 0x7f94880b8af0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94a0109e10 0x7f94a019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a01094c0 0x7f94a019a9e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 --2- 172.21.15.31:0/2738209267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94a0103b80 0x7f94a019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 >> 172.21.15.31:0/2738209267 conn(0x7f94a00ff5d0 msgr2=0x7f94a01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 shutdown_connections 2024-09-21T02:50:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:53.016+0000 7f94a5246640 1 -- 172.21.15.31:0/2738209267 wait complete. 2024-09-21T02:50:53.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:53 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2738209267' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:53.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:50:53.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:53 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2738209267' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:53.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:53 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2738209267' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:54.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 -- 172.21.15.31:0/1954922527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 msgr2=0x7f8500110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 --2- 172.21.15.31:0/1954922527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f8500110300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009ca0 tx=0x7f84f802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 -- 172.21.15.31:0/1954922527 shutdown_connections 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 --2- 172.21.15.31:0/1954922527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8500105300 0x7f850010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 --2- 172.21.15.31:0/1954922527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f8500110300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 --2- 172.21.15.31:0/1954922527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85001049b0 0x7f8500104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 -- 172.21.15.31:0/1954922527 >> 172.21.15.31:0/1954922527 conn(0x7f8500079730 msgr2=0x7f8500079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 -- 172.21.15.31:0/1954922527 shutdown_connections 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.424+0000 7f85079e6640 1 -- 172.21.15.31:0/1954922527 wait complete. 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 Processor -- start 2024-09-21T02:50:54.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 -- start start 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85001049b0 0x7f850019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500105300 0x7f850019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f850019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8500113a90 con 0x7f850010de10 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8500113910 con 0x7f8500105300 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f85079e6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8500113c10 con 0x7f85001049b0 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f8505f83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f850019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f8505f83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f850019fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48200/0 (socket says 172.21.15.31:48200) 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.425+0000 7f8505f83640 1 -- 172.21.15.31:0/647872279 learned_addr learned my addr 172.21.15.31:0/647872279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8504f81640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500105300 0x7f850019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:54.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505782640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85001049b0 0x7f850019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505f83640 1 -- 172.21.15.31:0/647872279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85001049b0 msgr2=0x7f850019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505f83640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85001049b0 0x7f850019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505f83640 1 -- 172.21.15.31:0/647872279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500105300 msgr2=0x7f850019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505f83640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500105300 0x7f850019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505f83640 1 -- 172.21.15.31:0/647872279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a0510 con 0x7f850010de10 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8504f81640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500105300 0x7f850019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f8505f83640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f850019fde0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f84f8005cc0 tx=0x7f84f802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8003bf0 con 0x7f850010de10 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f8003d90 con 0x7f850010de10 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a07a0 con 0x7f850010de10 2024-09-21T02:50:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f802fbc0 con 0x7f850010de10 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.426+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a6f60 con 0x7f850010de10 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.427+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500104dc0 con 0x7f850010de10 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.430+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f84f802fd60 con 0x7f850010de10 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.430+0000 7f84f67fc640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84ec0b6560 0x7f84ec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.430+0000 7f8505782640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84ec0b6560 0x7f84ec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.431+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f84f8108ad0 con 0x7f850010de10 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.431+0000 7f8505782640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84ec0b6560 0x7f84ec0b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84fc005fd0 tx=0x7f84fc005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.434+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84f80cf320 con 0x7f850010de10 2024-09-21T02:50:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.623+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f850019b6d0 con 0x7f850010de10 2024-09-21T02:50:54.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:54 smithi031 ceph-mon[31209]: pgmap v263: 97 pgs: 16 creating+peering, 1 creating+activating, 25 unknown, 55 active+clean; 578 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 637 B/s wr, 1 op/s 2024-09-21T02:50:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.625+0000 7f84f67fc640 1 -- 172.21.15.31:0/647872279 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22617 (secure 0 0 0) 0x7f850019b6d0 con 0x7f850010de10 2024-09-21T02:50:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:50:54.627 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:50:50.973567+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:50:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.627+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84ec0b6560 msgr2=0x7f84ec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.627+0000 7f85079e6640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84ec0b6560 0x7f84ec0b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84fc005fd0 tx=0x7f84fc005850 comp rx=0 tx=0).stop 2024-09-21T02:50:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.627+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 msgr2=0x7f850019fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.627+0000 7f85079e6640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f850019fde0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f84f8005cc0 tx=0x7f84f802f380 comp rx=0 tx=0).stop 2024-09-21T02:50:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 shutdown_connections 2024-09-21T02:50:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84ec0b6560 0x7f84ec0b8a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f850010de10 0x7f850019fde0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8500105300 0x7f850019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 --2- 172.21.15.31:0/647872279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85001049b0 0x7f850019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 >> 172.21.15.31:0/647872279 conn(0x7f8500079730 msgr2=0x7f85000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 shutdown_connections 2024-09-21T02:50:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:54.628+0000 7f85079e6640 1 -- 172.21.15.31:0/647872279 wait complete. 2024-09-21T02:50:54.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:54 smithi165 ceph-mon[34818]: pgmap v263: 97 pgs: 16 creating+peering, 1 creating+activating, 25 unknown, 55 active+clean; 578 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 637 B/s wr, 1 op/s 2024-09-21T02:50:54.727 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:54 smithi178 ceph-mon[34759]: pgmap v263: 97 pgs: 16 creating+peering, 1 creating+activating, 25 unknown, 55 active+clean; 578 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 637 B/s wr, 1 op/s 2024-09-21T02:50:55.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:50:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:55.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:55 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/647872279' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:55.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:55 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/647872279' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:55.691 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:50:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: 2024-09-21T02:50:55.618+0000 7f821a441640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:50:55.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:55 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/647872279' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:50:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 -- 172.21.15.31:0/129415471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 msgr2=0x7f3d5010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 --2- 172.21.15.31:0/129415471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5010f530 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009900 tx=0x7f3d4802a410 comp rx=0 tx=0).stop 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 -- 172.21.15.31:0/129415471 shutdown_connections 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 --2- 172.21.15.31:0/129415471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5010f530 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 --2- 172.21.15.31:0/129415471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d50105170 0x7f3d50105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 --2- 172.21.15.31:0/129415471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d50103160 0x7f3d50103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 -- 172.21.15.31:0/129415471 >> 172.21.15.31:0/129415471 conn(0x7f3d50077c60 msgr2=0x7f3d500780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 -- 172.21.15.31:0/129415471 shutdown_connections 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.071+0000 7f3d5569b640 1 -- 172.21.15.31:0/129415471 wait complete. 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 Processor -- start 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 -- start start 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d50103160 0x7f3d5019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d50105170 0x7f3d5019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:56.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50112a20 con 0x7f3d50105ac0 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d501128a0 con 0x7f3d50105170 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d5569b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d50112ba0 con 0x7f3d50103160 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d4f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d4f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48220/0 (socket says 172.21.15.31:48220) 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1850119615 learned_addr learned my addr 172.21.15.31:0/1850119615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.072+0000 7f3d4effd640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d50103160 0x7f3d5019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4e7fc640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d50105170 0x7f3d5019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1850119615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d50103160 msgr2=0x7f3d5019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4f7fe640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d50103160 0x7f3d5019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1850119615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d50105170 msgr2=0x7f3d5019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4f7fe640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d50105170 0x7f3d5019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1850119615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d501a0350 con 0x7f3d50105ac0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d4f7fe640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5019fce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009a20 tx=0x7f3d48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4803f030 con 0x7f3d50105ac0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d501a05e0 con 0x7f3d50105ac0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d48003aa0 con 0x7f3d50105ac0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48035070 con 0x7f3d50105ac0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.073+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d501a0b40 con 0x7f3d50105ac0 2024-09-21T02:50:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.074+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d50103570 con 0x7f3d50105ac0 2024-09-21T02:50:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.076+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3d48042030 con 0x7f3d50105ac0 2024-09-21T02:50:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.076+0000 7f3d37fff640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d380b6590 0x7f3d380b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.077+0000 7f3d4effd640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d380b6590 0x7f3d380b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.077+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f3d48107930 con 0x7f3d50105ac0 2024-09-21T02:50:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.077+0000 7f3d4effd640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d380b6590 0x7f3d380b8aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000a400 tx=0x7f3d4000d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:56.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.081+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3d480ce1e0 con 0x7f3d50105ac0 2024-09-21T02:50:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.311+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3d5002e640 con 0x7f3d50105ac0 2024-09-21T02:50:56.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.369+0000 7f3d37fff640 1 -- 172.21.15.31:0/1850119615 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f3d480d25c0 con 0x7f3d50105ac0 2024-09-21T02:50:56.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d380b6590 msgr2=0x7f3d380b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:56.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d380b6590 0x7f3d380b8aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000a400 tx=0x7f3d4000d040 comp rx=0 tx=0).stop 2024-09-21T02:50:56.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 msgr2=0x7f3d5019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:56.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5019fce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009a20 tx=0x7f3d48004270 comp rx=0 tx=0).stop 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 shutdown_connections 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d380b6590 0x7f3d380b8aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d50105ac0 0x7f3d5019fce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d50105170 0x7f3d5019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.372+0000 7f3d5569b640 1 --2- 172.21.15.31:0/1850119615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d50103160 0x7f3d5019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.373+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 >> 172.21.15.31:0/1850119615 conn(0x7f3d50077c60 msgr2=0x7f3d501011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.373+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 shutdown_connections 2024-09-21T02:50:56.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:56.373+0000 7f3d5569b640 1 -- 172.21.15.31:0/1850119615 wait complete. 2024-09-21T02:50:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:56 smithi031 ceph-mon[31209]: pgmap v264: 97 pgs: 97 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-21T02:50:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:56 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1850119615' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:50:56.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:56 smithi178 ceph-mon[34759]: pgmap v264: 97 pgs: 97 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-21T02:50:56.628 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:56 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1850119615' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:50:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:56 smithi165 ceph-mon[34818]: pgmap v264: 97 pgs: 97 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-09-21T02:50:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:56 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1850119615' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:50:57.020 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T02:50:57.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T02:50:57.279 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:57.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:57 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1850119615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:50:57.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:57 smithi165 ceph-mon[34818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:57.693 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:57 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1850119615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:50:57.694 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:57 smithi031 ceph-mon[31209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:57.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:57 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1850119615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:50:57.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:57 smithi178 ceph-mon[34759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 -- 172.21.15.31:0/4144906034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 msgr2=0x7fcbc4103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 --2- 172.21.15.31:0/4144906034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc4103fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009900 tx=0x7fcbb402a410 comp rx=0 tx=0).stop 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 -- 172.21.15.31:0/4144906034 shutdown_connections 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 --2- 172.21.15.31:0/4144906034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc41094b0 0x7fcbc41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 --2- 172.21.15.31:0/4144906034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc4109e00 0x7fcbc4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 --2- 172.21.15.31:0/4144906034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc4103fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 -- 172.21.15.31:0/4144906034 >> 172.21.15.31:0/4144906034 conn(0x7fcbc407b910 msgr2=0x7fcbc407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 -- 172.21.15.31:0/4144906034 shutdown_connections 2024-09-21T02:50:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.788+0000 7fcbc9206640 1 -- 172.21.15.31:0/4144906034 wait complete. 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 Processor -- start 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 -- start start 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc41094b0 0x7fcbc419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc4109e00 0x7fcbc419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4113ac0 con 0x7fcbc4103bc0 2024-09-21T02:50:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4113940 con 0x7fcbc41094b0 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc9206640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4113c40 con 0x7fcbc4109e00 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48228/0 (socket says 172.21.15.31:48228) 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc2d9d640 1 -- 172.21.15.31:0/2356105263 learned_addr learned my addr 172.21.15.31:0/2356105263 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc259c640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc41094b0 0x7fcbc419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.789+0000 7fcbc359e640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc4109e00 0x7fcbc419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc2d9d640 1 -- 172.21.15.31:0/2356105263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc4109e00 msgr2=0x7fcbc419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc2d9d640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc4109e00 0x7fcbc419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc2d9d640 1 -- 172.21.15.31:0/2356105263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc41094b0 msgr2=0x7fcbc419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc2d9d640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc41094b0 0x7fcbc419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc2d9d640 1 -- 172.21.15.31:0/2356105263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc41a0480 con 0x7fcbc4103bc0 2024-09-21T02:50:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc259c640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc41094b0 0x7fcbc419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:50:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc2d9d640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc419a510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4005940 tx=0x7fcbb40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb403f030 con 0x7fcbc4103bc0 2024-09-21T02:50:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc41a0710 con 0x7fcbc4103bc0 2024-09-21T02:50:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb40044c0 con 0x7fcbc4103bc0 2024-09-21T02:50:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4035070 con 0x7fcbc4103bc0 2024-09-21T02:50:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.790+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc41a6ed0 con 0x7fcbc4103bc0 2024-09-21T02:50:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.792+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc4104df0 con 0x7fcbc4103bc0 2024-09-21T02:50:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.793+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fcbb4042030 con 0x7fcbc4103bc0 2024-09-21T02:50:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.794+0000 7fcbbbfff640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcbb00b6560 0x7fcbb00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.794+0000 7fcbc259c640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcbb00b6560 0x7fcbb00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.794+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7fcbb41076e0 con 0x7fcbc4103bc0 2024-09-21T02:50:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.794+0000 7fcbc259c640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcbb00b6560 0x7fcbb00b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbc407beb0 tx=0x7fcbac005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:57.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:57.798+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcbb40d2310 con 0x7fcbc4103bc0 2024-09-21T02:50:58.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.025+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fcbc410bc30 con 0x7fcbc4103bc0 2024-09-21T02:50:58.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.383+0000 7fcbbbfff640 1 -- 172.21.15.31:0/2356105263 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fcbb40d7250 con 0x7fcbc4103bc0 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.385+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcbb00b6560 msgr2=0x7fcbb00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.385+0000 7fcbc9206640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcbb00b6560 0x7fcbb00b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbc407beb0 tx=0x7fcbac005f90 comp rx=0 tx=0).stop 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.385+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 msgr2=0x7fcbc419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.385+0000 7fcbc9206640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc419a510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4005940 tx=0x7fcbb40040c0 comp rx=0 tx=0).stop 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 shutdown_connections 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcbb00b6560 0x7fcbb00b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbc4109e00 0x7fcbc419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbc41094b0 0x7fcbc419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:58.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 --2- 172.21.15.31:0/2356105263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbc4103bc0 0x7fcbc419a510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 >> 172.21.15.31:0/2356105263 conn(0x7fcbc407b910 msgr2=0x7fcbc41075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 shutdown_connections 2024-09-21T02:50:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:58.386+0000 7fcbc9206640 1 -- 172.21.15.31:0/2356105263 wait complete. 2024-09-21T02:50:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:58 smithi165 ceph-mon[34818]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T02:50:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2356105263' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T02:50:58.697 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:58 smithi031 ceph-mon[31209]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T02:50:58.697 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2356105263' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T02:50:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:58 smithi178 ceph-mon[34759]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T02:50:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2356105263' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T02:50:59.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T02:50:59.297 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:50:59.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:59 smithi165 ceph-mon[34818]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T02:50:59.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2356105263' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T02:50:59.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:59 smithi165 ceph-mon[34818]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T02:50:59.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:59 smithi165 ceph-mon[34818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:59 smithi165 ceph-mon[34818]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T02:50:59.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:50:59 smithi165 ceph-mon[34818]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T02:50:59.703 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:59 smithi031 ceph-mon[31209]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T02:50:59.703 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:59 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2356105263' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T02:50:59.703 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:59 smithi031 ceph-mon[31209]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T02:50:59.704 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:59 smithi031 ceph-mon[31209]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:59.704 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:59 smithi031 ceph-mon[31209]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T02:50:59.704 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:50:59 smithi031 ceph-mon[31209]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T02:50:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:59 smithi178 ceph-mon[34759]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T02:50:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:59 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2356105263' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T02:50:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:59 smithi178 ceph-mon[34759]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T02:50:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:59 smithi178 ceph-mon[34759]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:50:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:59 smithi178 ceph-mon[34759]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T02:50:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:50:59 smithi178 ceph-mon[34759]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/4055579146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4107be0 msgr2=0x7f6df4107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/4055579146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4107be0 0x7f6df4107ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009900 tx=0x7f6de402a410 comp rx=0 tx=0).stop 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/4055579146 shutdown_connections 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/4055579146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df4108530 0x7f6df410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/4055579146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4107be0 0x7f6df4107ff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/4055579146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6df41022a0 0x7f6df41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/4055579146 >> 172.21.15.31:0/4055579146 conn(0x7f6df4079ec0 msgr2=0x7f6df407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/4055579146 shutdown_connections 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.797+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/4055579146 wait complete. 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 Processor -- start 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 -- start start 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df41022a0 0x7f6df419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6df4107be0 0x7f6df419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 0x7f6df419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111970 con 0x7f6df4108530 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6df41117f0 con 0x7f6df41022a0 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfbf2b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111af0 con 0x7f6df4107be0 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 0x7f6df419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 0x7f6df419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53852/0 (socket says 172.21.15.31:53852) 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 -- 172.21.15.31:0/2220437402 learned_addr learned my addr 172.21.15.31:0/2220437402 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:50:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6df9cc7640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df41022a0 0x7f6df419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6df94c6640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6df4107be0 0x7f6df419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 -- 172.21.15.31:0/2220437402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6df4107be0 msgr2=0x7f6df419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6df4107be0 0x7f6df419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 -- 172.21.15.31:0/2220437402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df41022a0 msgr2=0x7f6df419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df41022a0 0x7f6df419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6dfa4c8640 1 -- 172.21.15.31:0/2220437402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41a0520 con 0x7f6df4108530 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.798+0000 7f6df9cc7640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df41022a0 0x7f6df419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.799+0000 7f6dfa4c8640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 0x7f6df419fdf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00ec60 tx=0x7f6dec00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.799+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec004d20 con 0x7f6df4108530 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.799+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df41a0810 con 0x7f6df4108530 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.799+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dec00ba20 con 0x7f6df4108530 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.799+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0056c0 con 0x7f6df4108530 2024-09-21T02:50:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.799+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41a6f70 con 0x7f6df4108530 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.800+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df41026b0 con 0x7f6df4108530 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.801+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f6dec005860 con 0x7f6df4108530 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.802+0000 7f6deaffd640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6de00b6560 0x7f6de00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.802+0000 7f6df9cc7640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6de00b6560 0x7f6de00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.802+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f6dec0e7ce0 con 0x7f6df4108530 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.803+0000 7f6df9cc7640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6de00b6560 0x7f6de00b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009b50 tx=0x7f6df0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:50:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:50:59.806+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6dec0ae530 con 0x7f6df4108530 2024-09-21T02:51:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.033+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6df419b6e0 con 0x7f6df4108530 2024-09-21T02:51:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.403+0000 7f6deaffd640 1 -- 172.21.15.31:0/2220437402 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f6dec0b2910 con 0x7f6df4108530 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.405+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6de00b6560 msgr2=0x7f6de00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.405+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6de00b6560 0x7f6de00b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009b50 tx=0x7f6df0005f90 comp rx=0 tx=0).stop 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.405+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 msgr2=0x7f6df419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.405+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 0x7f6df419fdf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00ec60 tx=0x7f6dec00c7a0 comp rx=0 tx=0).stop 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 shutdown_connections 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6de00b6560 0x7f6de00b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6df4107be0 0x7f6df419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6df4108530 0x7f6df419fdf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 --2- 172.21.15.31:0/2220437402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6df41022a0 0x7f6df419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 >> 172.21.15.31:0/2220437402 conn(0x7f6df4079ec0 msgr2=0x7f6df4103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 shutdown_connections 2024-09-21T02:51:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:00.406+0000 7f6dfbf2b640 1 -- 172.21.15.31:0/2220437402 wait complete. 2024-09-21T02:51:00.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: pgmap v266: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T02:51:00.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] up:active 2024-09-21T02:51:00.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T02:51:00.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T02:51:00.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T02:51:00.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T02:51:00.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2220437402' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:51:00.693 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: 2024-09-21T02:51:00.618+0000 7f821a441640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T02:51:00.693 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:51:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: 2024-09-21T02:51:00.618+0000 7f821a441640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: pgmap v266: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] up:active 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T02:51:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:00 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2220437402' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: pgmap v266: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] up:active 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T02:51:00.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:00 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2220437402' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:51:01.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T02:51:01.339 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:01.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:01 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2220437402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:51:01.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:01 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] up:active 2024-09-21T02:51:01.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:01 smithi165 ceph-mon[34818]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T02:51:01.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:01 smithi165 ceph-mon[34818]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T02:51:01.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:01 smithi165 ceph-mon[34818]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T02:51:01.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:01 smithi165 ceph-mon[34818]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T02:51:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:01 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2220437402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:51:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:01 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] up:active 2024-09-21T02:51:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:01 smithi178 ceph-mon[34759]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T02:51:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:01 smithi178 ceph-mon[34759]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T02:51:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:01 smithi178 ceph-mon[34759]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T02:51:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:01 smithi178 ceph-mon[34759]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T02:51:01.738 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:01 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2220437402' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:51:01.738 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:01 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] up:active 2024-09-21T02:51:01.738 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:01 smithi031 ceph-mon[31209]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T02:51:01.738 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:01 smithi031 ceph-mon[31209]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T02:51:01.738 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:01 smithi031 ceph-mon[31209]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T02:51:01.738 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:01 smithi031 ceph-mon[31209]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T02:51:01.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 -- 172.21.15.31:0/1960355085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278107b80 msgr2=0x7f0278107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:01.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 --2- 172.21.15.31:0/1960355085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278107b80 0x7f0278107f90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f026c009900 tx=0x7f026c02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 -- 172.21.15.31:0/1960355085 shutdown_connections 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 --2- 172.21.15.31:0/1960355085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02781084d0 0x7f027810ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 --2- 172.21.15.31:0/1960355085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278107b80 0x7f0278107f90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 --2- 172.21.15.31:0/1960355085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0278102240 0x7f0278102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.831+0000 7f027e524640 1 -- 172.21.15.31:0/1960355085 >> 172.21.15.31:0/1960355085 conn(0x7f0278079f90 msgr2=0x7f027807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.832+0000 7f027e524640 1 -- 172.21.15.31:0/1960355085 shutdown_connections 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.832+0000 7f027e524640 1 -- 172.21.15.31:0/1960355085 wait complete. 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.832+0000 7f027e524640 1 Processor -- start 2024-09-21T02:51:01.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.832+0000 7f027e524640 1 -- start start 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027e524640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 0x7f027819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027e524640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0278107b80 0x7f027819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027e524640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02781084d0 0x7f027819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027e524640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0278112140 con 0x7f0278102240 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027e524640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0278111fc0 con 0x7f0278107b80 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027e524640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02781122c0 con 0x7f02781084d0 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 0x7f027819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 0x7f027819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53876/0 (socket says 172.21.15.31:53876) 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 -- 172.21.15.31:0/2972217154 learned_addr learned my addr 172.21.15.31:0/2972217154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f02777fe640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0278107b80 0x7f027819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f027cac1640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02781084d0 0x7f027819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:01.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 -- 172.21.15.31:0/2972217154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02781084d0 msgr2=0x7f027819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02781084d0 0x7f027819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 -- 172.21.15.31:0/2972217154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0278107b80 msgr2=0x7f027819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0278107b80 0x7f027819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.833+0000 7f0277fff640 1 -- 172.21.15.31:0/2972217154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a65d0 con 0x7f0278102240 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.834+0000 7f0277fff640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 0x7f027819b340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f026800d950 tx=0x7f026800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.834+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268017070 con 0x7f0278102240 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.834+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0268009c30 con 0x7f0278102240 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.834+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a68c0 con 0x7f0278102240 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.834+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026800f620 con 0x7f0278102240 2024-09-21T02:51:01.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.834+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a6dd0 con 0x7f0278102240 2024-09-21T02:51:01.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.835+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0278102650 con 0x7f0278102240 2024-09-21T02:51:01.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.836+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0268010460 con 0x7f0278102240 2024-09-21T02:51:01.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.837+0000 7f02757fa640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f02600b6590 0x7f02600b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:01.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.837+0000 7f02777fe640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f02600b6590 0x7f02600b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:01.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.837+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f02680e6e70 con 0x7f0278102240 2024-09-21T02:51:01.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.838+0000 7f02777fe640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f02600b6590 0x7f02600b8aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f026c009230 tx=0x7f026c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:01.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:01.841+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0268009de0 con 0x7f0278102240 2024-09-21T02:51:02.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.067+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f0278105da0 con 0x7f0278102240 2024-09-21T02:51:02.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.520+0000 7f02757fa640 1 -- 172.21.15.31:0/2972217154 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f02680b1b00 con 0x7f0278102240 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.523+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f02600b6590 msgr2=0x7f02600b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.523+0000 7f027e524640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f02600b6590 0x7f02600b8aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f026c009230 tx=0x7f026c031400 comp rx=0 tx=0).stop 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.523+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 msgr2=0x7f027819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.523+0000 7f027e524640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 0x7f027819b340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f026800d950 tx=0x7f026800de10 comp rx=0 tx=0).stop 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 shutdown_connections 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f02600b6590 0x7f02600b8aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02781084d0 0x7f027819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0278107b80 0x7f027819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 --2- 172.21.15.31:0/2972217154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0278102240 0x7f027819b340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 >> 172.21.15.31:0/2972217154 conn(0x7f0278079f90 msgr2=0x7f0278100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 shutdown_connections 2024-09-21T02:51:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:02.524+0000 7f027e524640 1 -- 172.21.15.31:0/2972217154 wait complete. 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: pgmap v267: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 8 op/s 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] up:active 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: Cluster is now healthy 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T02:51:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T02:51:02.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T02:51:02.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] up:active 2024-09-21T02:51:02.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:51:02.692 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:02 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2972217154' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:51:02.729 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: pgmap v267: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 8 op/s 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] up:active 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: Cluster is now healthy 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] up:active 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:51:02.730 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:02 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2972217154' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:51:02.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: pgmap v267: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 8 op/s 2024-09-21T02:51:02.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] up:active 2024-09-21T02:51:02.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T02:51:02.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T02:51:02.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: Cluster is now healthy 2024-09-21T02:51:02.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T02:51:02.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T02:51:02.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-21T02:51:02.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] up:active 2024-09-21T02:51:02.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:51:02.815 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:02 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2972217154' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:51:03.413 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-21T02:51:03.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T02:51:03.685 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:03.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:03 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2972217154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:51:03.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:03 smithi031 ceph-mon[31209]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:51:03.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:03 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2972217154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:51:03.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:03 smithi165 ceph-mon[34818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:51:03.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:03 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2972217154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:51:03.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:03 smithi178 ceph-mon[34759]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:51:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 -- 172.21.15.31:0/1586111989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 msgr2=0x7f50fc106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 --2- 172.21.15.31:0/1586111989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc106720 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009900 tx=0x7f50ec02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 -- 172.21.15.31:0/1586111989 shutdown_connections 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 --2- 172.21.15.31:0/1586111989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 0x7f50fc1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 --2- 172.21.15.31:0/1586111989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc106720 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 --2- 172.21.15.31:0/1586111989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50fc102120 0x7f50fc102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 -- 172.21.15.31:0/1586111989 >> 172.21.15.31:0/1586111989 conn(0x7f50fc079ea0 msgr2=0x7f50fc07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 -- 172.21.15.31:0/1586111989 shutdown_connections 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.168+0000 7f5103ecf640 1 -- 172.21.15.31:0/1586111989 wait complete. 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 Processor -- start 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 -- start start 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50fc102120 0x7f50fc19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 0x7f50fc1a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113b60 con 0x7f50fc106310 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1139e0 con 0x7f50fc106c60 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5103ecf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113ce0 con 0x7f50fc102120 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53904/0 (socket says 172.21.15.31:53904) 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 -- 172.21.15.31:0/4192141624 learned_addr learned my addr 172.21.15.31:0/4192141624 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510246c640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 0x7f50fc1a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f5101c6b640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50fc102120 0x7f50fc19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 -- 172.21.15.31:0/4192141624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50fc102120 msgr2=0x7f50fc19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50fc102120 0x7f50fc19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 -- 172.21.15.31:0/4192141624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 msgr2=0x7f50fc1a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 0x7f50fc1a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510146a640 1 -- 172.21.15.31:0/4192141624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc1a26a0 con 0x7f50fc106310 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.169+0000 7f510246c640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 0x7f50fc1a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.170+0000 7f510146a640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc19ccd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006010 tx=0x7f50ec004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.170+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec03f030 con 0x7f50fc106310 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.170+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1a2930 con 0x7f50fc106310 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.170+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ec003aa0 con 0x7f50fc106310 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.170+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec035070 con 0x7f50fc106310 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.170+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1a90f0 con 0x7f50fc106310 2024-09-21T02:51:04.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.171+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc06b8a0 con 0x7f50fc106310 2024-09-21T02:51:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.173+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f50ec042030 con 0x7f50fc106310 2024-09-21T02:51:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.173+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50e80b6560 0x7f50e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.174+0000 7f5101c6b640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50e80b6560 0x7f50e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.174+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f50ec107ca0 con 0x7f50fc106310 2024-09-21T02:51:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.174+0000 7f5101c6b640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50e80b6560 0x7f50e80b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009b60 tx=0x7f50f8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.177+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50ec0ce550 con 0x7f50fc106310 2024-09-21T02:51:04.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.391+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f50fc107bb0 con 0x7f50e80b6560 2024-09-21T02:51:04.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.425+0000 7f50f2ffd640 1 -- 172.21.15.31:0/4192141624 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50fc107bb0 con 0x7f50e80b6560 2024-09-21T02:51:04.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50e80b6560 msgr2=0x7f50e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:04.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50e80b6560 0x7f50e80b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009b60 tx=0x7f50f8005f90 comp rx=0 tx=0).stop 2024-09-21T02:51:04.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 msgr2=0x7f50fc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc19ccd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006010 tx=0x7f50ec004060 comp rx=0 tx=0).stop 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 shutdown_connections 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50e80b6560 0x7f50e80b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50fc106c60 0x7f50fc1a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50fc106310 0x7f50fc19ccd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 --2- 172.21.15.31:0/4192141624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50fc102120 0x7f50fc19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 >> 172.21.15.31:0/4192141624 conn(0x7f50fc079ea0 msgr2=0x7f50fc1074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 shutdown_connections 2024-09-21T02:51:04.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:04.427+0000 7f5103ecf640 1 -- 172.21.15.31:0/4192141624 wait complete. 2024-09-21T02:51:04.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:04 smithi031 ceph-mon[31209]: pgmap v268: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 434 B/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-21T02:51:04.814 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:51:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:51:04.396+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:51:04.814 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:51:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:51:04.396+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:51:04.814 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:51:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:51:04.396+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:51:04.815 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:51:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:51:04.396+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:51:04.815 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:51:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:51:04.396+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:51:04.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:04 smithi165 ceph-mon[34818]: pgmap v268: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 434 B/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-21T02:51:04.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:04 smithi178 ceph-mon[34759]: pgmap v268: 97 pgs: 97 active+clean; 580 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 434 B/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-21T02:51:05.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T02:51:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:05.824 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:05 smithi031 ceph-mon[31209]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 -- 172.21.15.31:0/2351208813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d410de10 msgr2=0x7f67d4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2351208813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d410de10 0x7f67d4110300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009ca0 tx=0x7f67cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 -- 172.21.15.31:0/2351208813 shutdown_connections 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2351208813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d410de10 0x7f67d4110300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2351208813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d4105300 0x7f67d410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2351208813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67d41049b0 0x7f67d4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 -- 172.21.15.31:0/2351208813 >> 172.21.15.31:0/2351208813 conn(0x7f67d4079730 msgr2=0x7f67d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 -- 172.21.15.31:0/2351208813 shutdown_connections 2024-09-21T02:51:05.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.921+0000 7f67da8a3640 1 -- 172.21.15.31:0/2351208813 wait complete. 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 Processor -- start 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 -- start start 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67d41049b0 0x7f67d419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 0x7f67d419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d410de10 0x7f67d419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4113a90 con 0x7f67d4105300 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4113910 con 0x7f67d410de10 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.922+0000 7f67da8a3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4113c10 con 0x7f67d41049b0 2024-09-21T02:51:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 0x7f67d419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 0x7f67d419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53936/0 (socket says 172.21.15.31:53936) 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 -- 172.21.15.31:0/2969987248 learned_addr learned my addr 172.21.15.31:0/2969987248 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67d3fff640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67d41049b0 0x7f67d419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67d8e40640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d410de10 0x7f67d419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 -- 172.21.15.31:0/2969987248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67d41049b0 msgr2=0x7f67d419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67d41049b0 0x7f67d419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 -- 172.21.15.31:0/2969987248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d410de10 msgr2=0x7f67d419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d410de10 0x7f67d419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 -- 172.21.15.31:0/2969987248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d41a03d0 con 0x7f67d4105300 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67cbfff640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 0x7f67d419aa00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00d740 tx=0x7f67bc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc017070 con 0x7f67d4105300 2024-09-21T02:51:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67bc00de50 con 0x7f67d4105300 2024-09-21T02:51:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.923+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d41a06c0 con 0x7f67d4105300 2024-09-21T02:51:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.924+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00f620 con 0x7f67d4105300 2024-09-21T02:51:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.924+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d41a6e20 con 0x7f67d4105300 2024-09-21T02:51:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.924+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d4104dc0 con 0x7f67d4105300 2024-09-21T02:51:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.926+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f67bc01a070 con 0x7f67d4105300 2024-09-21T02:51:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.927+0000 7f67d1ffb640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c00b65f0 0x7f67c00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.927+0000 7f67d3fff640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c00b65f0 0x7f67c00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.927+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f67bc0e6f30 con 0x7f67d4105300 2024-09-21T02:51:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.927+0000 7f67d3fff640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c00b65f0 0x7f67c00b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f67d406bc60 tx=0x7f67c4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:05.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:05.931+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f67bc0ad620 con 0x7f67d4105300 2024-09-21T02:51:05.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:05 smithi165 ceph-mon[34818]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:05.941 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 02:51:05 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: 2024-09-21T02:51:05.618+0000 7f821a441640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T02:51:05.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:05 smithi178 ceph-mon[34759]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:06.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.149+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f67d4100c70 con 0x7f67c00b65f0 2024-09-21T02:51:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.178+0000 7f67d1ffb640 1 -- 172.21.15.31:0/2969987248 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67d4100c70 con 0x7f67c00b65f0 2024-09-21T02:51:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c00b65f0 msgr2=0x7f67c00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c00b65f0 0x7f67c00b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f67d406bc60 tx=0x7f67c4005f90 comp rx=0 tx=0).stop 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 msgr2=0x7f67d419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 0x7f67d419aa00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00d740 tx=0x7f67bc00dc00 comp rx=0 tx=0).stop 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 shutdown_connections 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c00b65f0 0x7f67c00b8b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d410de10 0x7f67d419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d4105300 0x7f67d419aa00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 --2- 172.21.15.31:0/2969987248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67d41049b0 0x7f67d419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 >> 172.21.15.31:0/2969987248 conn(0x7f67d4079730 msgr2=0x7f67d40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 shutdown_connections 2024-09-21T02:51:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:06.180+0000 7f67da8a3640 1 -- 172.21.15.31:0/2969987248 wait complete. 2024-09-21T02:51:06.796 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:06 smithi031 ceph-mon[31209]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-09-21T02:51:06.796 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:06 smithi031 ceph-mon[31209]: mgrmap e22: x(active, since 6m), standbys: z, y 2024-09-21T02:51:06.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T02:51:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:06 smithi165 ceph-mon[34818]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-09-21T02:51:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:06 smithi165 ceph-mon[34818]: mgrmap e22: x(active, since 6m), standbys: z, y 2024-09-21T02:51:06.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:06 smithi178 ceph-mon[34759]: pgmap v269: 97 pgs: 97 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-09-21T02:51:06.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:06 smithi178 ceph-mon[34759]: mgrmap e22: x(active, since 6m), standbys: z, y 2024-09-21T02:51:07.081 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 -- 172.21.15.31:0/2499766552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad181094a0 msgr2=0x7fad181098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/2499766552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad181094a0 0x7fad181098b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad08009900 tx=0x7fad0802a410 comp rx=0 tx=0).stop 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 -- 172.21.15.31:0/2499766552 shutdown_connections 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/2499766552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad18109df0 0x7fad18110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/2499766552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad181094a0 0x7fad181098b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/2499766552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18103b60 0x7fad18103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 -- 172.21.15.31:0/2499766552 >> 172.21.15.31:0/2499766552 conn(0x7fad1807b850 msgr2=0x7fad1807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 -- 172.21.15.31:0/2499766552 shutdown_connections 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 -- 172.21.15.31:0/2499766552 wait complete. 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.606+0000 7fad1f5fe640 1 Processor -- start 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 -- start start 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18103b60 0x7fad18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad181094a0 0x7fad18075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 0x7fad18076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fad18113ab0 con 0x7fad18109df0 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad18113930 con 0x7fad181094a0 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1f5fe640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad18113c30 con 0x7fad18103b60 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 0x7fad18076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 0x7fad18076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53952/0 (socket says 172.21.15.31:53952) 2024-09-21T02:51:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 -- 172.21.15.31:0/1437764604 learned_addr learned my addr 172.21.15.31:0/1437764604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1cb99640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad181094a0 0x7fad18075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1d39a640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18103b60 0x7fad18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 -- 172.21.15.31:0/1437764604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18103b60 msgr2=0x7fad18075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18103b60 0x7fad18075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 -- 172.21.15.31:0/1437764604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad181094a0 msgr2=0x7fad18075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad181094a0 0x7fad18075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.607+0000 7fad1db9b640 1 -- 172.21.15.31:0/1437764604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad181a8760 con 0x7fad18109df0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.608+0000 7fad1db9b640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 0x7fad18076190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad1000eb10 tx=0x7fad1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.608+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10004b20 con 0x7fad18109df0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.608+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad10004cc0 con 0x7fad18109df0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.608+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad181a8a50 con 0x7fad18109df0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.608+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10010670 con 0x7fad18109df0 2024-09-21T02:51:07.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.608+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181a8f60 con 0x7fad18109df0 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.609+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18103f70 con 0x7fad18109df0 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.610+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fad10005000 con 0x7fad18109df0 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.611+0000 7fad0e7fc640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fad040b6670 0x7fad040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.611+0000 7fad1d39a640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fad040b6670 0x7fad040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.611+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7fad100e7d10 con 0x7fad18109df0 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.612+0000 7fad1d39a640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fad040b6670 0x7fad040b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad14005fd0 tx=0x7fad14005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:07.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.614+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad10056710 con 0x7fad18109df0 2024-09-21T02:51:07.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:07 smithi165 ceph-mon[34818]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:07.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:07 smithi178 ceph-mon[34759]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.832+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fad18063800 con 0x7fad040b6670 2024-09-21T02:51:07.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:07 smithi031 ceph-mon[31209]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.838+0000 7fad0e7fc640 1 -- 172.21.15.31:0/1437764604 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fad18063800 con 0x7fad040b6670 2024-09-21T02:51:07.839 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/0a009cb1-cd1f-49e7-8b6d-be671db908e4 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fad040b6670 msgr2=0x7fad040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fad040b6670 0x7fad040b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad14005fd0 tx=0x7fad14005850 comp rx=0 tx=0).stop 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 msgr2=0x7fad18076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 0x7fad18076190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad1000eb10 tx=0x7fad1000c2d0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 shutdown_connections 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fad040b6670 0x7fad040b8b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad18109df0 0x7fad18076190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad181094a0 0x7fad18075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 --2- 172.21.15.31:0/1437764604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad18103b60 0x7fad18075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.840+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 >> 172.21.15.31:0/1437764604 conn(0x7fad1807b850 msgr2=0x7fad18107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.841+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 shutdown_connections 2024-09-21T02:51:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:07.841+0000 7fad1f5fe640 1 -- 172.21.15.31:0/1437764604 wait complete. 2024-09-21T02:51:08.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T02:51:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:08 smithi031 ceph-mon[31209]: pgmap v270: 97 pgs: 97 active+clean; 607 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-21T02:51:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:08 smithi031 ceph-mon[31209]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:08 smithi165 ceph-mon[34818]: pgmap v270: 97 pgs: 97 active+clean; 607 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-21T02:51:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:08 smithi165 ceph-mon[34818]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:08 smithi178 ceph-mon[34759]: pgmap v270: 97 pgs: 97 active+clean; 607 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s 2024-09-21T02:51:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:08 smithi178 ceph-mon[34759]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:09.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 -- 172.21.15.31:0/341647350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 msgr2=0x7f1e30108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:09.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 --2- 172.21.15.31:0/341647350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e30108830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009900 tx=0x7f1e2402a410 comp rx=0 tx=0).stop 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 -- 172.21.15.31:0/341647350 shutdown_connections 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 --2- 172.21.15.31:0/341647350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e30108d70 0x7f1e3010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 --2- 172.21.15.31:0/341647350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e30108830 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 --2- 172.21.15.31:0/341647350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30102ae0 0x7f1e30102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 -- 172.21.15.31:0/341647350 >> 172.21.15.31:0/341647350 conn(0x7f1e30079770 msgr2=0x7f1e30079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 -- 172.21.15.31:0/341647350 shutdown_connections 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.408+0000 7f1e35237640 1 -- 172.21.15.31:0/341647350 wait complete. 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 Processor -- start 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 -- start start 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e30102ae0 0x7f1e3019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e3019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30108d70 0x7f1e3019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30112a30 con 0x7f1e30108420 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301128b0 con 0x7f1e30102ae0 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e35237640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30112bb0 con 0x7f1e30108d70 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.409+0000 7f1e2e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e3019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e3019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42394/0 (socket says 172.21.15.31:42394) 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 -- 172.21.15.31:0/468632402 learned_addr learned my addr 172.21.15.31:0/468632402 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2f59e640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30108d70 0x7f1e3019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2ed9d640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e30102ae0 0x7f1e3019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 -- 172.21.15.31:0/468632402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30108d70 msgr2=0x7f1e3019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30108d70 0x7f1e3019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 -- 172.21.15.31:0/468632402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e30102ae0 msgr2=0x7f1e3019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e30102ae0 0x7f1e3019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 -- 172.21.15.31:0/468632402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e301a6640 con 0x7f1e30108420 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2f59e640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30108d70 0x7f1e3019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:09.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e2e59c640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e3019b8c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006010 tx=0x7f1e24004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2403f030 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e24003aa0 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e301a68d0 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24035070 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.410+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e301a6db0 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.411+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e30103d60 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.413+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1e24042030 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.414+0000 7f1e17fff640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1e180b65f0 0x7f1e180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.414+0000 7f1e2ed9d640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1e180b65f0 0x7f1e180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.414+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f1e24107ed0 con 0x7f1e30108420 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.414+0000 7f1e2ed9d640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1e180b65f0 0x7f1e180b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1e3006c6e0 tx=0x7f1e20005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.417+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e240ce5c0 con 0x7f1e30108420 2024-09-21T02:51:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.634+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1e30063800 con 0x7f1e180b65f0 2024-09-21T02:51:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.665+0000 7f1e17fff640 1 -- 172.21.15.31:0/468632402 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1e30063800 con 0x7f1e180b65f0 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1e180b65f0 msgr2=0x7f1e180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1e180b65f0 0x7f1e180b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1e3006c6e0 tx=0x7f1e20005f90 comp rx=0 tx=0).stop 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 msgr2=0x7f1e3019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e3019b8c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006010 tx=0x7f1e24004060 comp rx=0 tx=0).stop 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 shutdown_connections 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1e180b65f0 0x7f1e180b8b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e30108d70 0x7f1e3019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e30108420 0x7f1e3019b8c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 --2- 172.21.15.31:0/468632402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e30102ae0 0x7f1e3019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 >> 172.21.15.31:0/468632402 conn(0x7f1e30079770 msgr2=0x7f1e301044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.667+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 shutdown_connections 2024-09-21T02:51:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:09.668+0000 7f1e35237640 1 -- 172.21.15.31:0/468632402 wait complete. 2024-09-21T02:51:10.442 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T02:51:10.708 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:10.829 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:10 smithi031 ceph-mon[31209]: pgmap v271: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2024-09-21T02:51:10.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:10 smithi031 ceph-mon[31209]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:10 smithi165 ceph-mon[34818]: pgmap v271: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2024-09-21T02:51:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:10 smithi165 ceph-mon[34818]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:10 smithi178 ceph-mon[34759]: pgmap v271: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2024-09-21T02:51:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:10 smithi178 ceph-mon[34759]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 -- 172.21.15.31:0/684367887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 msgr2=0x7f3420103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 --2- 172.21.15.31:0/684367887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f3420103f70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3410009ca0 tx=0x7f341002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 -- 172.21.15.31:0/684367887 shutdown_connections 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 --2- 172.21.15.31:0/684367887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3420109df0 0x7f3420110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 --2- 172.21.15.31:0/684367887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 0x7f34201098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 --2- 172.21.15.31:0/684367887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f3420103f70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 -- 172.21.15.31:0/684367887 >> 172.21.15.31:0/684367887 conn(0x7f342007b850 msgr2=0x7f342007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 -- 172.21.15.31:0/684367887 shutdown_connections 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.232+0000 7f34285e7640 1 -- 172.21.15.31:0/684367887 wait complete. 2024-09-21T02:51:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 Processor -- start 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 -- start start 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f342019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 0x7f342019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3420109df0 0x7f342019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113ab0 con 0x7f3420103b60 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113930 con 0x7f34201094a0 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.233+0000 7f34285e7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3420113c30 con 0x7f3420109df0 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f342019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f342019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42406/0 (socket says 172.21.15.31:42406) 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 -- 172.21.15.31:0/2005105288 learned_addr learned my addr 172.21.15.31:0/2005105288 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3425b82640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 0x7f342019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426b84640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3420109df0 0x7f342019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:11.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 -- 172.21.15.31:0/2005105288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3420109df0 msgr2=0x7f342019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3420109df0 0x7f342019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 -- 172.21.15.31:0/2005105288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 msgr2=0x7f342019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 0x7f342019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 -- 172.21.15.31:0/2005105288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34201a0410 con 0x7f3420103b60 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3425b82640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 0x7f342019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f3426383640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f342019a4a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3410006010 tx=0x7f34100319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410003bf0 con 0x7f3420103b60 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3410003d90 con 0x7f3420103b60 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201a06a0 con 0x7f3420103b60 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341002fc20 con 0x7f3420103b60 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.234+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a6e60 con 0x7f3420103b60 2024-09-21T02:51:11.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.235+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3420103f70 con 0x7f3420103b60 2024-09-21T02:51:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.237+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f34100046a0 con 0x7f3420103b60 2024-09-21T02:51:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.238+0000 7f341f7fe640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34080b65f0 0x7f34080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.238+0000 7f3425b82640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34080b65f0 0x7f34080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.238+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f3410108ae0 con 0x7f3420103b60 2024-09-21T02:51:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.238+0000 7f3425b82640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34080b65f0 0x7f34080b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34140058c0 tx=0x7f3414005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:11.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.241+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34100cf230 con 0x7f3420103b60 2024-09-21T02:51:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.457+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f3420063800 con 0x7f34080b65f0 2024-09-21T02:51:11.463 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/75fbed08-2606-481b-8769-d706cef27b6b 2024-09-21T02:51:11.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.462+0000 7f341f7fe640 1 -- 172.21.15.31:0/2005105288 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3420063800 con 0x7f34080b65f0 2024-09-21T02:51:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34080b65f0 msgr2=0x7f34080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34080b65f0 0x7f34080b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34140058c0 tx=0x7f3414005850 comp rx=0 tx=0).stop 2024-09-21T02:51:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 msgr2=0x7f342019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f342019a4a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3410006010 tx=0x7f34100319f0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 shutdown_connections 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34080b65f0 0x7f34080b8b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3420109df0 0x7f342019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34201094a0 0x7f342019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 --2- 172.21.15.31:0/2005105288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3420103b60 0x7f342019a4a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 >> 172.21.15.31:0/2005105288 conn(0x7f342007b850 msgr2=0x7f3420107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 shutdown_connections 2024-09-21T02:51:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:11.465+0000 7f34285e7640 1 -- 172.21.15.31:0/2005105288 wait complete. 2024-09-21T02:51:12.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:51:12.428 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:12 smithi031 ceph-mon[31209]: pgmap v272: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-21T02:51:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:12 smithi031 ceph-mon[31209]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 -- 172.21.15.31:0/4168432143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b41083a0 msgr2=0x7fc3b41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 --2- 172.21.15.31:0/4168432143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b41083a0 0x7fc3b41087b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009900 tx=0x7fc3a802a410 comp rx=0 tx=0).stop 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 -- 172.21.15.31:0/4168432143 shutdown_connections 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 --2- 172.21.15.31:0/4168432143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3b4108cf0 0x7fc3b410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 --2- 172.21.15.31:0/4168432143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b41083a0 0x7fc3b41087b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 --2- 172.21.15.31:0/4168432143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b4102a60 0x7fc3b4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 -- 172.21.15.31:0/4168432143 >> 172.21.15.31:0/4168432143 conn(0x7fc3b4079730 msgr2=0x7fc3b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 -- 172.21.15.31:0/4168432143 shutdown_connections 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.926+0000 7fc3b8972640 1 -- 172.21.15.31:0/4168432143 wait complete. 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 Processor -- start 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 -- start start 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 0x7fc3b40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3b41083a0 0x7fc3b40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b4108cf0 0x7fc3b41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4112960 con 0x7fc3b4102a60 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b41127e0 con 0x7fc3b4108cf0 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b8972640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4112ae0 con 0x7fc3b41083a0 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 0x7fc3b40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 0x7fc3b40ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42422/0 (socket says 172.21.15.31:42422) 2024-09-21T02:51:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b259c640 1 -- 172.21.15.31:0/794488782 learned_addr learned my addr 172.21.15.31:0/794488782 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.927+0000 7fc3b2d9d640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b4108cf0 0x7fc3b41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b1d9b640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3b41083a0 0x7fc3b40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b259c640 1 -- 172.21.15.31:0/794488782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3b41083a0 msgr2=0x7fc3b40ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b259c640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3b41083a0 0x7fc3b40ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b259c640 1 -- 172.21.15.31:0/794488782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b4108cf0 msgr2=0x7fc3b41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b259c640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b4108cf0 0x7fc3b41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b259c640 1 -- 172.21.15.31:0/794488782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b41a8700 con 0x7fc3b4102a60 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b259c640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 0x7fc3b40ff670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400d950 tx=0x7fc3a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4017070 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b41a89f0 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b41a8f50 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a4009c30 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.928+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a400f620 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.929+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b4102e70 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.931+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc3a4005030 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.932+0000 7fc3a37fe640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc39c0b6670 0x7fc39c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.932+0000 7fc3b1d9b640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc39c0b6670 0x7fc39c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.932+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7fc3a40e7090 con 0x7fc3b4102a60 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.933+0000 7fc3b1d9b640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc39c0b6670 0x7fc39c0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000c00 tx=0x7fc3a8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:12.936+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3a40ad860 con 0x7fc3b4102a60 2024-09-21T02:51:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:12 smithi165 ceph-mon[34818]: pgmap v272: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-21T02:51:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:12 smithi165 ceph-mon[34818]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:12 smithi178 ceph-mon[34759]: pgmap v272: 97 pgs: 97 active+clean; 608 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-21T02:51:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:12 smithi178 ceph-mon[34759]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.161+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3b4100970 con 0x7fc3b4102a60 2024-09-21T02:51:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.162+0000 7fc3a37fe640 1 -- 172.21.15.31:0/794488782 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7fc3a4009de0 con 0x7fc3b4102a60 2024-09-21T02:51:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:13.166 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:51:02.068424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24659,24655,24637,24632]},"id":1}]} 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc39c0b6670 msgr2=0x7fc39c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc39c0b6670 0x7fc39c0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000c00 tx=0x7fc3a8031400 comp rx=0 tx=0).stop 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 msgr2=0x7fc3b40ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 0x7fc3b40ff670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400d950 tx=0x7fc3a400de10 comp rx=0 tx=0).stop 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 shutdown_connections 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc39c0b6670 0x7fc39c0b8b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3b41083a0 0x7fc3b40ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b4108cf0 0x7fc3b41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 --2- 172.21.15.31:0/794488782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b4102a60 0x7fc3b40ff670 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 >> 172.21.15.31:0/794488782 conn(0x7fc3b4079730 msgr2=0x7fc3b406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:13.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 shutdown_connections 2024-09-21T02:51:13.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:13.165+0000 7fc3b8972640 1 -- 172.21.15.31:0/794488782 wait complete. 2024-09-21T02:51:13.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:13 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/794488782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:51:13.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:13 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/794488782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:51:13.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:13 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/794488782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:51:14.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:51:14.387 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:14.787 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:14 smithi031 ceph-mon[31209]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2024-09-21T02:51:14.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 -- 172.21.15.31:0/1716595322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc1092d0 msgr2=0x7f55dc1096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 --2- 172.21.15.31:0/1716595322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc1092d0 0x7f55dc1096e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009900 tx=0x7f55d002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 -- 172.21.15.31:0/1716595322 shutdown_connections 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 --2- 172.21.15.31:0/1716595322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55dc109c20 0x7f55dc110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 --2- 172.21.15.31:0/1716595322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc1092d0 0x7f55dc1096e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 --2- 172.21.15.31:0/1716595322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55dc103990 0x7f55dc103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 -- 172.21.15.31:0/1716595322 >> 172.21.15.31:0/1716595322 conn(0x7f55dc0ff3e0 msgr2=0x7f55dc101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 -- 172.21.15.31:0/1716595322 shutdown_connections 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.877+0000 7f55db59e640 1 -- 172.21.15.31:0/1716595322 wait complete. 2024-09-21T02:51:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 Processor -- start 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 -- start start 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55dc103990 0x7f55dc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55dc1092d0 0x7f55dc19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 0x7f55dc19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1138e0 con 0x7f55dc109c20 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc113760 con 0x7f55dc103990 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55db59e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc113a60 con 0x7f55dc1092d0 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 0x7f55dc19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 0x7f55dc19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42434/0 (socket says 172.21.15.31:42434) 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.878+0000 7f55dad9d640 1 -- 172.21.15.31:0/1674655643 learned_addr learned my addr 172.21.15.31:0/1674655643 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55da59c640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55dc103990 0x7f55dc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55d9d9b640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55dc1092d0 0x7f55dc19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55dad9d640 1 -- 172.21.15.31:0/1674655643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55dc1092d0 msgr2=0x7f55dc19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55dad9d640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55dc1092d0 0x7f55dc19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55dad9d640 1 -- 172.21.15.31:0/1674655643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55dc103990 msgr2=0x7f55dc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55dad9d640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55dc103990 0x7f55dc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55dad9d640 1 -- 172.21.15.31:0/1674655643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1a0260 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55dad9d640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 0x7f55dc19fb30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55d400eaa0 tx=0x7f55d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d400cdb0 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1a0550 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d4005020 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40116d0 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.879+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a6cb0 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.880+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc103da0 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.882+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f55d40051d0 con 0x7f55dc109c20 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.883+0000 7f55cb7fe640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f55c40b65f0 0x7f55c40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:14.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.883+0000 7f55da59c640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f55c40b65f0 0x7f55c40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.883+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8194+0+0 (secure 0 0 0) 0x7f55d40e7a20 con 0x7f55dc109c20 2024-09-21T02:51:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.883+0000 7f55da59c640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f55c40b65f0 0x7f55c40b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55cc005fd0 tx=0x7f55cc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:14.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:14.886+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55d4016030 con 0x7f55dc109c20 2024-09-21T02:51:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:14 smithi165 ceph-mon[34818]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2024-09-21T02:51:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:14 smithi178 ceph-mon[34759]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2024-09-21T02:51:15.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.073+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55dc19b420 con 0x7f55dc109c20 2024-09-21T02:51:15.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.075+0000 7f55cb7fe640 1 -- 172.21.15.31:0/1674655643 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22617 (secure 0 0 0) 0x7f55dc19b420 con 0x7f55dc109c20 2024-09-21T02:51:15.076 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:15.076 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:50:50.973567+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:51:15.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f55c40b65f0 msgr2=0x7f55c40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f55c40b65f0 0x7f55c40b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55cc005fd0 tx=0x7f55cc00b390 comp rx=0 tx=0).stop 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 msgr2=0x7f55dc19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 0x7f55dc19fb30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55d400eaa0 tx=0x7f55d400ef60 comp rx=0 tx=0).stop 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 shutdown_connections 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f55c40b65f0 0x7f55c40b8b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55dc1092d0 0x7f55dc19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55dc109c20 0x7f55dc19fb30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 --2- 172.21.15.31:0/1674655643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55dc103990 0x7f55dc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 >> 172.21.15.31:0/1674655643 conn(0x7f55dc0ff3e0 msgr2=0x7f55dc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 shutdown_connections 2024-09-21T02:51:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:15.077+0000 7f55db59e640 1 -- 172.21.15.31:0/1674655643 wait complete. 2024-09-21T02:51:15.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1674655643' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:15.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T02:51:15.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T02:51:15.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T02:51:15.745 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T02:51:15.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T02:51:15.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2024-09-21T02:51:15.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-21T02:51:15.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]} : dispatch 2024-09-21T02:51:15.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T02:51:15.746 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-21T02:51:15.775 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T02:51:15.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:51:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1674655643' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T02:51:15.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T02:51:15.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-21T02:51:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1674655643' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T02:51:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T02:51:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-21T02:51:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-21T02:51:15.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T02:51:15.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2024-09-21T02:51:15.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-21T02:51:15.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]} : dispatch 2024-09-21T02:51:15.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-21T02:51:15.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-21T02:51:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:16.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 -- 172.21.15.31:0/648114051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d01094a0 msgr2=0x7ff9d01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:16.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/648114051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d01094a0 0x7ff9d01098b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009900 tx=0x7ff9c002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:16.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 -- 172.21.15.31:0/648114051 shutdown_connections 2024-09-21T02:51:16.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/648114051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 0x7ff9d0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/648114051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d01094a0 0x7ff9d01098b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/648114051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d0103b60 0x7ff9d0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 -- 172.21.15.31:0/648114051 >> 172.21.15.31:0/648114051 conn(0x7ff9d007b850 msgr2=0x7ff9d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 -- 172.21.15.31:0/648114051 shutdown_connections 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.557+0000 7ff9d80d9640 1 -- 172.21.15.31:0/648114051 wait complete. 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 Processor -- start 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 -- start start 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0113ab0 con 0x7ff9d0103b60 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0113930 con 0x7ff9d0109df0 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d80d9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0113c30 con 0x7ff9d01094a0 2024-09-21T02:51:16.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.558+0000 7ff9d5e75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42456/0 (socket says 172.21.15.31:42456) 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 -- 172.21.15.31:0/1699413909 learned_addr learned my addr 172.21.15.31:0/1699413909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5674640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d6676640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 -- 172.21.15.31:0/1699413909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d01094a0 msgr2=0x7ff9d0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 -- 172.21.15.31:0/1699413909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 msgr2=0x7ff9d0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 -- 172.21.15.31:0/1699413909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d01a8740 con 0x7ff9d0103b60 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d6676640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:16.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d5e75640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e9f0 tx=0x7ff9cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00cd80 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9cc0054d0 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc010690 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d01a8a30 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.559+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d01a8f40 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.560+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d0103f70 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.562+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff9cc004e30 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.563+0000 7ff9c6ffd640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff9bc0b6670 0x7ff9bc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.563+0000 7ff9d5674640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff9bc0b6670 0x7ff9bc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.563+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8484+0+0 (secure 0 0 0) 0x7ff9cc0e7be0 con 0x7ff9d0103b60 2024-09-21T02:51:16.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.564+0000 7ff9d5674640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff9bc0b6670 0x7ff9bc0b8b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009230 tx=0x7ff9c0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:16.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.567+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff9cc056710 con 0x7ff9d0103b60 2024-09-21T02:51:16.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.756+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9d0063800 con 0x7ff9bc0b6670 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: pgmap v274: 97 pgs: 97 active+clean; 634 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T02:51:16.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-09-21T02:51:16.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-21T02:51:16.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-21T02:51:16.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T02:51:16.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-21T02:51:16.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:16 smithi031 ceph-mon[31209]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:51:16.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.764+0000 7ff9c6ffd640 1 -- 172.21.15.31:0/1699413909 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258817 (secure 0 0 0) 0x7ff9d0063800 con 0x7ff9bc0b6670 2024-09-21T02:51:16.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:16.769 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff9bc0b6670 msgr2=0x7ff9bc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff9bc0b6670 0x7ff9bc0b8b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009230 tx=0x7ff9c0031400 comp rx=0 tx=0).stop 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 msgr2=0x7ff9d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e9f0 tx=0x7ff9cc00edf0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 shutdown_connections 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff9bc0b6670 0x7ff9bc0b8b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9d01094a0 0x7ff9d0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9d0109df0 0x7ff9d0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 --2- 172.21.15.31:0/1699413909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9d0103b60 0x7ff9d0075710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 >> 172.21.15.31:0/1699413909 conn(0x7ff9d007b850 msgr2=0x7ff9d0105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 shutdown_connections 2024-09-21T02:51:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:16.768+0000 7ff9d80d9640 1 -- 172.21.15.31:0/1699413909 wait complete. 2024-09-21T02:51:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: pgmap v274: 97 pgs: 97 active+clean; 634 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-09-21T02:51:16.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T02:51:16.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T02:51:16.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T02:51:16.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T02:51:16.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T02:51:16.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-09-21T02:51:16.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-21T02:51:16.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-21T02:51:16.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T02:51:16.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-21T02:51:16.943 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:16 smithi165 ceph-mon[34818]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:51:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: pgmap v274: 97 pgs: 97 active+clean; 634 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-09-21T02:51:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T02:51:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T02:51:16.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-21T02:51:16.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-21T02:51:16.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T02:51:16.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-09-21T02:51:16.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-21T02:51:16.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-09-21T02:51:16.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-21T02:51:16.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-21T02:51:16.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:16 smithi178 ceph-mon[34759]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:51:17.512 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-21T02:51:16.570006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649137,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":245,"num_read_kb":216,"num_write":357,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371980,"kb_used_data":10248,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569844,"statfs":{"total":1151940427776,"available":1151559520256,"internally_reserved":0,"allocated":10493952,"data_stored":6777685,"data_compressed":117042,"data_compressed_allocated":151552,"data_compressed_original":1802240,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27439,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":84,"num_read_kb":64,"num_write":18,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.427730"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.349993+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:50:51.349993+0000","last_peered":"2024-09-21T02:50:51.349993+0000","last_clean":"2024-09-21T02:50:51.349993+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:50:51.349993+0000","last_undegraded":"2024-09-21T02:50:51.349993+0000","last_fullsized":"2024-09-21T02:50:51.349993+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366951+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:50:51.366951+0000","last_peered":"2024-09-21T02:50:51.366951+0000","last_clean":"2024-09-21T02:50:51.366951+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:50:51.366951+0000","last_undegraded":"2024-09-21T02:50:51.366951+0000","last_fullsized":"2024-09-21T02:50:51.366951+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367489+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:50:51.367489+0000","last_peered":"2024-09-21T02:50:51.367489+0000","last_clean":"2024-09-21T02:50:51.367489+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:50:51.367489+0000","last_undegraded":"2024-09-21T02:50:51.367489+0000","last_fullsized":"2024-09-21T02:50:51.367489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367793+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:50:51.367793+0000","last_peered":"2024-09-21T02:50:51.367793+0000","last_clean":"2024-09-21T02:50:51.367793+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:50:51.367793+0000","last_undegraded":"2024-09-21T02:50:51.367793+0000","last_fullsized":"2024-09-21T02:50:51.367793+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364196+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:50:51.364196+0000","last_peered":"2024-09-21T02:50:51.364196+0000","last_clean":"2024-09-21T02:50:51.364196+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:50:51.364196+0000","last_undegraded":"2024-09-21T02:50:51.364196+0000","last_fullsized":"2024-09-21T02:50:51.364196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.351560+0000","last_change":"2024-09-21T02:50:50.351679+0000","last_active":"2024-09-21T02:50:51.351560+0000","last_peered":"2024-09-21T02:50:51.351560+0000","last_clean":"2024-09-21T02:50:51.351560+0000","last_became_active":"2024-09-21T02:50:50.351261+0000","last_became_peered":"2024-09-21T02:50:50.351261+0000","last_unstale":"2024-09-21T02:50:51.351560+0000","last_undegraded":"2024-09-21T02:50:51.351560+0000","last_fullsized":"2024-09-21T02:50:51.351560+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:28.366863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365493+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:50:51.365493+0000","last_peered":"2024-09-21T02:50:51.365493+0000","last_clean":"2024-09-21T02:50:51.365493+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:50:51.365493+0000","last_undegraded":"2024-09-21T02:50:51.365493+0000","last_fullsized":"2024-09-21T02:50:51.365493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366874+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:50:51.366874+0000","last_peered":"2024-09-21T02:50:51.366874+0000","last_clean":"2024-09-21T02:50:51.366874+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:50:51.366874+0000","last_undegraded":"2024-09-21T02:50:51.366874+0000","last_fullsized":"2024-09-21T02:50:51.366874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365471+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:50:51.365471+0000","last_peered":"2024-09-21T02:50:51.365471+0000","last_clean":"2024-09-21T02:50:51.365471+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:50:51.365471+0000","last_undegraded":"2024-09-21T02:50:51.365471+0000","last_fullsized":"2024-09-21T02:50:51.365471+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.352179+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:50:51.352179+0000","last_peered":"2024-09-21T02:50:51.352179+0000","last_clean":"2024-09-21T02:50:51.352179+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:50:51.352179+0000","last_undegraded":"2024-09-21T02:50:51.352179+0000","last_fullsized":"2024-09-21T02:50:51.352179+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366457+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:50:51.366457+0000","last_peered":"2024-09-21T02:50:51.366457+0000","last_clean":"2024-09-21T02:50:51.366457+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:50:51.366457+0000","last_undegraded":"2024-09-21T02:50:51.366457+0000","last_fullsized":"2024-09-21T02:50:51.366457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":2,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-21T02:50:53.795329+0000","last_change":"2024-09-21T02:51:16.189802+0000","last_active":"2024-09-21T02:51:16.189802+0000","last_peered":"2024-09-21T02:50:53.795329+0000","last_clean":"2024-09-21T02:50:53.795329+0000","last_became_active":"2024-09-21T02:50:50.355048+0000","last_became_peered":"2024-09-21T02:50:50.355048+0000","last_unstale":"2024-09-21T02:50:53.795329+0000","last_undegraded":"2024-09-21T02:50:53.795329+0000","last_fullsized":"2024-09-21T02:50:53.795329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.568940+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:15.568940+0000","last_peered":"2024-09-21T02:51:15.568940+0000","last_clean":"2024-09-21T02:51:15.568940+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:15.568940+0000","last_undegraded":"2024-09-21T02:51:15.568940+0000","last_fullsized":"2024-09-21T02:51:15.568940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366976+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:50:51.366976+0000","last_peered":"2024-09-21T02:50:51.366976+0000","last_clean":"2024-09-21T02:50:51.366976+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:50:51.366976+0000","last_undegraded":"2024-09-21T02:50:51.366976+0000","last_fullsized":"2024-09-21T02:50:51.366976+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367308+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:50:51.367308+0000","last_peered":"2024-09-21T02:50:51.367308+0000","last_clean":"2024-09-21T02:50:51.367308+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:50:51.367308+0000","last_undegraded":"2024-09-21T02:50:51.367308+0000","last_fullsized":"2024-09-21T02:50:51.367308+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364651+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:50:51.364651+0000","last_peered":"2024-09-21T02:50:51.364651+0000","last_clean":"2024-09-21T02:50:51.364651+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:50:51.364651+0000","last_undegraded":"2024-09-21T02:50:51.364651+0000","last_fullsized":"2024-09-21T02:50:51.364651+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.350076+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:50:51.350076+0000","last_peered":"2024-09-21T02:50:51.350076+0000","last_clean":"2024-09-21T02:50:51.350076+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:50:51.350076+0000","last_undegraded":"2024-09-21T02:50:51.350076+0000","last_fullsized":"2024-09-21T02:50:51.350076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571093+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:15.571093+0000","last_peered":"2024-09-21T02:51:15.571093+0000","last_clean":"2024-09-21T02:51:15.571093+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:15.571093+0000","last_undegraded":"2024-09-21T02:51:15.571093+0000","last_fullsized":"2024-09-21T02:51:15.571093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.369695+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:50:51.369695+0000","last_peered":"2024-09-21T02:50:51.369695+0000","last_clean":"2024-09-21T02:50:51.369695+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:50:51.369695+0000","last_undegraded":"2024-09-21T02:50:51.369695+0000","last_fullsized":"2024-09-21T02:50:51.369695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.349618+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:50:51.349618+0000","last_peered":"2024-09-21T02:50:51.349618+0000","last_clean":"2024-09-21T02:50:51.349618+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:50:51.349618+0000","last_undegraded":"2024-09-21T02:50:51.349618+0000","last_fullsized":"2024-09-21T02:50:51.349618+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569053+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:15.569053+0000","last_peered":"2024-09-21T02:51:15.569053+0000","last_clean":"2024-09-21T02:51:15.569053+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:15.569053+0000","last_undegraded":"2024-09-21T02:51:15.569053+0000","last_fullsized":"2024-09-21T02:51:15.569053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364133+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:50:51.364133+0000","last_peered":"2024-09-21T02:50:51.364133+0000","last_clean":"2024-09-21T02:50:51.364133+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:50:51.364133+0000","last_undegraded":"2024-09-21T02:50:51.364133+0000","last_fullsized":"2024-09-21T02:50:51.364133+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366687+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:50:51.366687+0000","last_peered":"2024-09-21T02:50:51.366687+0000","last_clean":"2024-09-21T02:50:51.366687+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:50:51.366687+0000","last_undegraded":"2024-09-21T02:50:51.366687+0000","last_fullsized":"2024-09-21T02:50:51.366687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-21T02:50:50.346201+0000","last_change":"2024-09-21T02:51:15.944950+0000","last_active":"2024-09-21T02:50:49.327862+0000","last_peered":"2024-09-21T02:50:49.327862+0000","last_clean":"2024-09-21T02:50:49.327862+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:50:50.346201+0000","last_undegraded":"2024-09-21T02:50:50.346201+0000","last_fullsized":"2024-09-21T02:50:50.346201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365463+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:50:51.365463+0000","last_peered":"2024-09-21T02:50:51.365463+0000","last_clean":"2024-09-21T02:50:51.365463+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:50:51.365463+0000","last_undegraded":"2024-09-21T02:50:51.365463+0000","last_fullsized":"2024-09-21T02:50:51.365463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.369227+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:50:51.369227+0000","last_peered":"2024-09-21T02:50:51.369227+0000","last_clean":"2024-09-21T02:50:51.369227+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:50:51.369227+0000","last_undegraded":"2024-09-21T02:50:51.369227+0000","last_fullsized":"2024-09-21T02:50:51.369227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.349935+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:50:51.349935+0000","last_peered":"2024-09-21T02:50:51.349935+0000","last_clean":"2024-09-21T02:50:51.349935+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:50:51.349935+0000","last_undegraded":"2024-09-21T02:50:51.349935+0000","last_fullsized":"2024-09-21T02:50:51.349935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569250+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:15.569250+0000","last_peered":"2024-09-21T02:51:15.569250+0000","last_clean":"2024-09-21T02:51:15.569250+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:15.569250+0000","last_undegraded":"2024-09-21T02:51:15.569250+0000","last_fullsized":"2024-09-21T02:51:15.569250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367380+0000","last_change":"2024-09-21T02:50:50.364293+0000","last_active":"2024-09-21T02:50:51.367380+0000","last_peered":"2024-09-21T02:50:51.367380+0000","last_clean":"2024-09-21T02:50:51.367380+0000","last_became_active":"2024-09-21T02:50:50.363879+0000","last_became_peered":"2024-09-21T02:50:50.363879+0000","last_unstale":"2024-09-21T02:50:51.367380+0000","last_undegraded":"2024-09-21T02:50:51.367380+0000","last_fullsized":"2024-09-21T02:50:51.367380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.466771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366966+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:50:51.366966+0000","last_peered":"2024-09-21T02:50:51.366966+0000","last_clean":"2024-09-21T02:50:51.366966+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:50:51.366966+0000","last_undegraded":"2024-09-21T02:50:51.366966+0000","last_fullsized":"2024-09-21T02:50:51.366966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367361+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:50:51.367361+0000","last_peered":"2024-09-21T02:50:51.367361+0000","last_clean":"2024-09-21T02:50:51.367361+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:50:51.367361+0000","last_undegraded":"2024-09-21T02:50:51.367361+0000","last_fullsized":"2024-09-21T02:50:51.367361+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.349547+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:50:51.349547+0000","last_peered":"2024-09-21T02:50:51.349547+0000","last_clean":"2024-09-21T02:50:51.349547+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:50:51.349547+0000","last_undegraded":"2024-09-21T02:50:51.349547+0000","last_fullsized":"2024-09-21T02:50:51.349547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.363864+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:50:51.363864+0000","last_peered":"2024-09-21T02:50:51.363864+0000","last_clean":"2024-09-21T02:50:51.363864+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:50:51.363864+0000","last_undegraded":"2024-09-21T02:50:51.363864+0000","last_fullsized":"2024-09-21T02:50:51.363864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.546741+0000","last_change":"2024-09-21T02:50:48.335956+0000","last_active":"2024-09-21T02:51:12.546741+0000","last_peered":"2024-09-21T02:51:12.546741+0000","last_clean":"2024-09-21T02:51:12.546741+0000","last_became_active":"2024-09-21T02:50:48.335545+0000","last_became_peered":"2024-09-21T02:50:48.335545+0000","last_unstale":"2024-09-21T02:51:12.546741+0000","last_undegraded":"2024-09-21T02:51:12.546741+0000","last_fullsized":"2024-09-21T02:51:12.546741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:26.486593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365426+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:50:51.365426+0000","last_peered":"2024-09-21T02:50:51.365426+0000","last_clean":"2024-09-21T02:50:51.365426+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:50:51.365426+0000","last_undegraded":"2024-09-21T02:50:51.365426+0000","last_fullsized":"2024-09-21T02:50:51.365426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.422473+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:00.422473+0000","last_peered":"2024-09-21T02:51:00.422473+0000","last_clean":"2024-09-21T02:51:00.422473+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:00.422473+0000","last_undegraded":"2024-09-21T02:51:00.422473+0000","last_fullsized":"2024-09-21T02:51:00.422473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569862+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:15.569862+0000","last_peered":"2024-09-21T02:51:15.569862+0000","last_clean":"2024-09-21T02:51:15.569862+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:15.569862+0000","last_undegraded":"2024-09-21T02:51:15.569862+0000","last_fullsized":"2024-09-21T02:51:15.569862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367125+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:50:51.367125+0000","last_peered":"2024-09-21T02:50:51.367125+0000","last_clean":"2024-09-21T02:50:51.367125+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:50:51.367125+0000","last_undegraded":"2024-09-21T02:50:51.367125+0000","last_fullsized":"2024-09-21T02:50:51.367125+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.433979+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:01.433979+0000","last_peered":"2024-09-21T02:51:01.433979+0000","last_clean":"2024-09-21T02:51:01.433979+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:01.433979+0000","last_undegraded":"2024-09-21T02:51:01.433979+0000","last_fullsized":"2024-09-21T02:51:01.433979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.570493+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:15.570493+0000","last_peered":"2024-09-21T02:51:15.570493+0000","last_clean":"2024-09-21T02:51:15.570493+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:15.570493+0000","last_undegraded":"2024-09-21T02:51:15.570493+0000","last_fullsized":"2024-09-21T02:51:15.570493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:11.544549+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:51:11.544549+0000","last_peered":"2024-09-21T02:51:11.544549+0000","last_clean":"2024-09-21T02:51:11.544549+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:51:11.544549+0000","last_undegraded":"2024-09-21T02:51:11.544549+0000","last_fullsized":"2024-09-21T02:51:11.544549+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364788+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:50:51.364788+0000","last_peered":"2024-09-21T02:50:51.364788+0000","last_clean":"2024-09-21T02:50:51.364788+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:50:51.364788+0000","last_undegraded":"2024-09-21T02:50:51.364788+0000","last_fullsized":"2024-09-21T02:50:51.364788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434034+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:01.434034+0000","last_peered":"2024-09-21T02:51:01.434034+0000","last_clean":"2024-09-21T02:51:01.434034+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:01.434034+0000","last_undegraded":"2024-09-21T02:51:01.434034+0000","last_fullsized":"2024-09-21T02:51:01.434034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.368031+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:50:51.368031+0000","last_peered":"2024-09-21T02:50:51.368031+0000","last_clean":"2024-09-21T02:50:51.368031+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:50:51.368031+0000","last_undegraded":"2024-09-21T02:50:51.368031+0000","last_fullsized":"2024-09-21T02:50:51.368031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.421603+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:00.421603+0000","last_peered":"2024-09-21T02:51:00.421603+0000","last_clean":"2024-09-21T02:51:00.421603+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:00.421603+0000","last_undegraded":"2024-09-21T02:51:00.421603+0000","last_fullsized":"2024-09-21T02:51:00.421603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364386+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:50:51.364386+0000","last_peered":"2024-09-21T02:50:51.364386+0000","last_clean":"2024-09-21T02:50:51.364386+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:50:51.364386+0000","last_undegraded":"2024-09-21T02:50:51.364386+0000","last_fullsized":"2024-09-21T02:50:51.364386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'24","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:11.548615+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:51:11.548615+0000","last_peered":"2024-09-21T02:51:11.548615+0000","last_clean":"2024-09-21T02:51:11.548615+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:51:11.548615+0000","last_undegraded":"2024-09-21T02:51:11.548615+0000","last_fullsized":"2024-09-21T02:51:11.548615+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":62,"num_write":26,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364106+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:50:51.364106+0000","last_peered":"2024-09-21T02:50:51.364106+0000","last_clean":"2024-09-21T02:50:51.364106+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:50:51.364106+0000","last_undegraded":"2024-09-21T02:50:51.364106+0000","last_fullsized":"2024-09-21T02:50:51.364106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.526223+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:01.526223+0000","last_peered":"2024-09-21T02:51:01.526223+0000","last_clean":"2024-09-21T02:51:01.526223+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:01.526223+0000","last_undegraded":"2024-09-21T02:51:01.526223+0000","last_fullsized":"2024-09-21T02:51:01.526223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"peering","last_fresh":"2024-09-21T02:50:51.364322+0000","last_change":"2024-09-21T02:51:15.568971+0000","last_active":"2024-09-21T02:51:15.568971+0000","last_peered":"2024-09-21T02:50:51.364322+0000","last_clean":"2024-09-21T02:50:51.364322+0000","last_became_active":"2024-09-21T02:50:50.358686+0000","last_became_peered":"2024-09-21T02:50:50.358686+0000","last_unstale":"2024-09-21T02:50:51.364322+0000","last_undegraded":"2024-09-21T02:50:51.364322+0000","last_fullsized":"2024-09-21T02:50:51.364322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434044+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:01.434044+0000","last_peered":"2024-09-21T02:51:01.434044+0000","last_clean":"2024-09-21T02:51:01.434044+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:01.434044+0000","last_undegraded":"2024-09-21T02:51:01.434044+0000","last_fullsized":"2024-09-21T02:51:01.434044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365911+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:50:51.365911+0000","last_peered":"2024-09-21T02:50:51.365911+0000","last_clean":"2024-09-21T02:50:51.365911+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:50:51.365911+0000","last_undegraded":"2024-09-21T02:50:51.365911+0000","last_fullsized":"2024-09-21T02:50:51.365911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.546861+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:51:12.546861+0000","last_peered":"2024-09-21T02:51:12.546861+0000","last_clean":"2024-09-21T02:51:12.546861+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:51:12.546861+0000","last_undegraded":"2024-09-21T02:51:12.546861+0000","last_fullsized":"2024-09-21T02:51:12.546861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365211+0000","last_change":"2024-09-21T02:50:50.355712+0000","last_active":"2024-09-21T02:50:51.365211+0000","last_peered":"2024-09-21T02:50:51.365211+0000","last_clean":"2024-09-21T02:50:51.365211+0000","last_became_active":"2024-09-21T02:50:50.355315+0000","last_became_peered":"2024-09-21T02:50:50.355315+0000","last_unstale":"2024-09-21T02:50:51.365211+0000","last_undegraded":"2024-09-21T02:50:51.365211+0000","last_fullsized":"2024-09-21T02:50:51.365211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:52.187455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.432474+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:01.432474+0000","last_peered":"2024-09-21T02:51:01.432474+0000","last_clean":"2024-09-21T02:51:01.432474+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:01.432474+0000","last_undegraded":"2024-09-21T02:51:01.432474+0000","last_fullsized":"2024-09-21T02:51:01.432474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.368281+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:50:51.368281+0000","last_peered":"2024-09-21T02:50:51.368281+0000","last_clean":"2024-09-21T02:50:51.368281+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:50:51.368281+0000","last_undegraded":"2024-09-21T02:50:51.368281+0000","last_fullsized":"2024-09-21T02:50:51.368281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.407391+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:00.407391+0000","last_peered":"2024-09-21T02:51:00.407391+0000","last_clean":"2024-09-21T02:51:00.407391+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:00.407391+0000","last_undegraded":"2024-09-21T02:51:00.407391+0000","last_fullsized":"2024-09-21T02:51:00.407391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.349850+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:50:51.349850+0000","last_peered":"2024-09-21T02:50:51.349850+0000","last_clean":"2024-09-21T02:50:51.349850+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:50:51.349850+0000","last_undegraded":"2024-09-21T02:50:51.349850+0000","last_fullsized":"2024-09-21T02:50:51.349850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569756+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:15.569756+0000","last_peered":"2024-09-21T02:51:15.569756+0000","last_clean":"2024-09-21T02:51:15.569756+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:15.569756+0000","last_undegraded":"2024-09-21T02:51:15.569756+0000","last_fullsized":"2024-09-21T02:51:15.569756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366578+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:50:51.366578+0000","last_peered":"2024-09-21T02:50:51.366578+0000","last_clean":"2024-09-21T02:50:51.366578+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:50:51.366578+0000","last_undegraded":"2024-09-21T02:50:51.366578+0000","last_fullsized":"2024-09-21T02:50:51.366578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.431282+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:01.431282+0000","last_peered":"2024-09-21T02:51:01.431282+0000","last_clean":"2024-09-21T02:51:01.431282+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:01.431282+0000","last_undegraded":"2024-09-21T02:51:01.431282+0000","last_fullsized":"2024-09-21T02:51:01.431282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366265+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:50:51.366265+0000","last_peered":"2024-09-21T02:50:51.366265+0000","last_clean":"2024-09-21T02:50:51.366265+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:50:51.366265+0000","last_undegraded":"2024-09-21T02:50:51.366265+0000","last_fullsized":"2024-09-21T02:50:51.366265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.546628+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:51:12.546628+0000","last_peered":"2024-09-21T02:51:12.546628+0000","last_clean":"2024-09-21T02:51:12.546628+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:51:12.546628+0000","last_undegraded":"2024-09-21T02:51:12.546628+0000","last_fullsized":"2024-09-21T02:51:12.546628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366459+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:50:51.366459+0000","last_peered":"2024-09-21T02:50:51.366459+0000","last_clean":"2024-09-21T02:50:51.366459+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:50:51.366459+0000","last_undegraded":"2024-09-21T02:50:51.366459+0000","last_fullsized":"2024-09-21T02:50:51.366459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365725+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:50:51.365725+0000","last_peered":"2024-09-21T02:50:51.365725+0000","last_clean":"2024-09-21T02:50:51.365725+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:50:51.365725+0000","last_undegraded":"2024-09-21T02:50:51.365725+0000","last_fullsized":"2024-09-21T02:50:51.365725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.526601+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:01.526601+0000","last_peered":"2024-09-21T02:51:01.526601+0000","last_clean":"2024-09-21T02:51:01.526601+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:01.526601+0000","last_undegraded":"2024-09-21T02:51:01.526601+0000","last_fullsized":"2024-09-21T02:51:01.526601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366088+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:50:51.366088+0000","last_peered":"2024-09-21T02:50:51.366088+0000","last_clean":"2024-09-21T02:50:51.366088+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:50:51.366088+0000","last_undegraded":"2024-09-21T02:50:51.366088+0000","last_fullsized":"2024-09-21T02:50:51.366088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:59.411327+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:50:59.411327+0000","last_peered":"2024-09-21T02:50:59.411327+0000","last_clean":"2024-09-21T02:50:59.411327+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:50:59.411327+0000","last_undegraded":"2024-09-21T02:50:59.411327+0000","last_fullsized":"2024-09-21T02:50:59.411327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569647+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:15.569647+0000","last_peered":"2024-09-21T02:51:15.569647+0000","last_clean":"2024-09-21T02:51:15.569647+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:15.569647+0000","last_undegraded":"2024-09-21T02:51:15.569647+0000","last_fullsized":"2024-09-21T02:51:15.569647+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.570011+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:51:15.570011+0000","last_peered":"2024-09-21T02:51:15.570011+0000","last_clean":"2024-09-21T02:51:15.570011+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:51:15.570011+0000","last_undegraded":"2024-09-21T02:51:15.570011+0000","last_fullsized":"2024-09-21T02:51:15.570011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367740+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:50:51.367740+0000","last_peered":"2024-09-21T02:50:51.367740+0000","last_clean":"2024-09-21T02:50:51.367740+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:50:51.367740+0000","last_undegraded":"2024-09-21T02:50:51.367740+0000","last_fullsized":"2024-09-21T02:50:51.367740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.545943+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:51:12.545943+0000","last_peered":"2024-09-21T02:51:12.545943+0000","last_clean":"2024-09-21T02:51:12.545943+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:51:12.545943+0000","last_undegraded":"2024-09-21T02:51:12.545943+0000","last_fullsized":"2024-09-21T02:51:12.545943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.351690+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:50:51.351690+0000","last_peered":"2024-09-21T02:50:51.351690+0000","last_clean":"2024-09-21T02:50:51.351690+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:50:51.351690+0000","last_undegraded":"2024-09-21T02:50:51.351690+0000","last_fullsized":"2024-09-21T02:50:51.351690+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569627+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:15.569627+0000","last_peered":"2024-09-21T02:51:15.569627+0000","last_clean":"2024-09-21T02:51:15.569627+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:15.569627+0000","last_undegraded":"2024-09-21T02:51:15.569627+0000","last_fullsized":"2024-09-21T02:51:15.569627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.723906+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:50:51.723906+0000","last_peered":"2024-09-21T02:50:51.723906+0000","last_clean":"2024-09-21T02:50:51.723906+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:50:51.723906+0000","last_undegraded":"2024-09-21T02:50:51.723906+0000","last_fullsized":"2024-09-21T02:50:51.723906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.433470+0000","last_change":"2024-09-21T02:50:48.627495+0000","last_active":"2024-09-21T02:51:01.433470+0000","last_peered":"2024-09-21T02:51:01.433470+0000","last_clean":"2024-09-21T02:51:01.433470+0000","last_became_active":"2024-09-21T02:50:48.627056+0000","last_became_peered":"2024-09-21T02:50:48.627056+0000","last_unstale":"2024-09-21T02:51:01.433470+0000","last_undegraded":"2024-09-21T02:51:01.433470+0000","last_fullsized":"2024-09-21T02:51:01.433470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:31.659209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569363+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:15.569363+0000","last_peered":"2024-09-21T02:51:15.569363+0000","last_clean":"2024-09-21T02:51:15.569363+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:15.569363+0000","last_undegraded":"2024-09-21T02:51:15.569363+0000","last_fullsized":"2024-09-21T02:51:15.569363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571231+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:15.571231+0000","last_peered":"2024-09-21T02:51:15.571231+0000","last_clean":"2024-09-21T02:51:15.571231+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:15.571231+0000","last_undegraded":"2024-09-21T02:51:15.571231+0000","last_fullsized":"2024-09-21T02:51:15.571231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365747+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:50:51.365747+0000","last_peered":"2024-09-21T02:50:51.365747+0000","last_clean":"2024-09-21T02:50:51.365747+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:50:51.365747+0000","last_undegraded":"2024-09-21T02:50:51.365747+0000","last_fullsized":"2024-09-21T02:50:51.365747+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434166+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:01.434166+0000","last_peered":"2024-09-21T02:51:01.434166+0000","last_clean":"2024-09-21T02:51:01.434166+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:01.434166+0000","last_undegraded":"2024-09-21T02:51:01.434166+0000","last_fullsized":"2024-09-21T02:51:01.434166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367497+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:50:51.367497+0000","last_peered":"2024-09-21T02:50:51.367497+0000","last_clean":"2024-09-21T02:50:51.367497+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:50:51.367497+0000","last_undegraded":"2024-09-21T02:50:51.367497+0000","last_fullsized":"2024-09-21T02:50:51.367497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:58.400841+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:50:58.400841+0000","last_peered":"2024-09-21T02:50:58.400841+0000","last_clean":"2024-09-21T02:50:58.400841+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:50:58.400841+0000","last_undegraded":"2024-09-21T02:50:58.400841+0000","last_fullsized":"2024-09-21T02:50:58.400841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367060+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:50:51.367060+0000","last_peered":"2024-09-21T02:50:51.367060+0000","last_clean":"2024-09-21T02:50:51.367060+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:50:51.367060+0000","last_undegraded":"2024-09-21T02:50:51.367060+0000","last_fullsized":"2024-09-21T02:50:51.367060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.431863+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:01.431863+0000","last_peered":"2024-09-21T02:51:01.431863+0000","last_clean":"2024-09-21T02:51:01.431863+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:01.431863+0000","last_undegraded":"2024-09-21T02:51:01.431863+0000","last_fullsized":"2024-09-21T02:51:01.431863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.349828+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:50:51.349828+0000","last_peered":"2024-09-21T02:50:51.349828+0000","last_clean":"2024-09-21T02:50:51.349828+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:50:51.349828+0000","last_undegraded":"2024-09-21T02:50:51.349828+0000","last_fullsized":"2024-09-21T02:50:51.349828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.421542+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:00.421542+0000","last_peered":"2024-09-21T02:51:00.421542+0000","last_clean":"2024-09-21T02:51:00.421542+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:00.421542+0000","last_undegraded":"2024-09-21T02:51:00.421542+0000","last_fullsized":"2024-09-21T02:51:00.421542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:50:51.366034+0000","last_peered":"2024-09-21T02:50:51.366034+0000","last_clean":"2024-09-21T02:50:51.366034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:50:51.366034+0000","last_undegraded":"2024-09-21T02:50:51.366034+0000","last_fullsized":"2024-09-21T02:50:51.366034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:58.400577+0000","last_change":"2024-09-21T02:50:48.340162+0000","last_active":"2024-09-21T02:50:58.400577+0000","last_peered":"2024-09-21T02:50:58.400577+0000","last_clean":"2024-09-21T02:50:58.400577+0000","last_became_active":"2024-09-21T02:50:48.339615+0000","last_became_peered":"2024-09-21T02:50:48.339615+0000","last_unstale":"2024-09-21T02:50:58.400577+0000","last_undegraded":"2024-09-21T02:50:58.400577+0000","last_fullsized":"2024-09-21T02:50:58.400577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:09.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365600+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:50:51.365600+0000","last_peered":"2024-09-21T02:50:51.365600+0000","last_clean":"2024-09-21T02:50:51.365600+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:50:51.365600+0000","last_undegraded":"2024-09-21T02:50:51.365600+0000","last_fullsized":"2024-09-21T02:50:51.365600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365542+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:50:51.365542+0000","last_peered":"2024-09-21T02:50:51.365542+0000","last_clean":"2024-09-21T02:50:51.365542+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:50:51.365542+0000","last_undegraded":"2024-09-21T02:50:51.365542+0000","last_fullsized":"2024-09-21T02:50:51.365542+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.433791+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:01.433791+0000","last_peered":"2024-09-21T02:51:01.433791+0000","last_clean":"2024-09-21T02:51:01.433791+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:01.433791+0000","last_undegraded":"2024-09-21T02:51:01.433791+0000","last_fullsized":"2024-09-21T02:51:01.433791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364104+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:50:51.364104+0000","last_peered":"2024-09-21T02:50:51.364104+0000","last_clean":"2024-09-21T02:50:51.364104+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:50:51.364104+0000","last_undegraded":"2024-09-21T02:50:51.364104+0000","last_fullsized":"2024-09-21T02:50:51.364104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.525639+0000","last_change":"2024-09-21T02:50:48.340526+0000","last_active":"2024-09-21T02:51:01.525639+0000","last_peered":"2024-09-21T02:51:01.525639+0000","last_clean":"2024-09-21T02:51:01.525639+0000","last_became_active":"2024-09-21T02:50:48.340110+0000","last_became_peered":"2024-09-21T02:50:48.340110+0000","last_unstale":"2024-09-21T02:51:01.525639+0000","last_undegraded":"2024-09-21T02:51:01.525639+0000","last_fullsized":"2024-09-21T02:51:01.525639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:03.229869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367975+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:50:51.367975+0000","last_peered":"2024-09-21T02:50:51.367975+0000","last_clean":"2024-09-21T02:50:51.367975+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:50:51.367975+0000","last_undegraded":"2024-09-21T02:50:51.367975+0000","last_fullsized":"2024-09-21T02:50:51.367975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.371481+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:50:51.371481+0000","last_peered":"2024-09-21T02:50:51.371481+0000","last_clean":"2024-09-21T02:50:51.371481+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:50:51.371481+0000","last_undegraded":"2024-09-21T02:50:51.371481+0000","last_fullsized":"2024-09-21T02:50:51.371481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367592+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:50:51.367592+0000","last_peered":"2024-09-21T02:50:51.367592+0000","last_clean":"2024-09-21T02:50:51.367592+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:50:51.367592+0000","last_undegraded":"2024-09-21T02:50:51.367592+0000","last_fullsized":"2024-09-21T02:50:51.367592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571411+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:51:15.571411+0000","last_peered":"2024-09-21T02:51:15.571411+0000","last_clean":"2024-09-21T02:51:15.571411+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:51:15.571411+0000","last_undegraded":"2024-09-21T02:51:15.571411+0000","last_fullsized":"2024-09-21T02:51:15.571411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58535,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":164,"num_read_kb":147,"num_write":232,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":175605,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710736,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":403358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874261,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1044936,"data_compressed":38047,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037788,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":864256,"data_stored":411871,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":403275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":996184,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":403005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":995471,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":402475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":402439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":455469,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":454858,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":404344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":53083,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":53694,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":52676,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:51:17.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:51:17.778 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:17.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:17 smithi031 ceph-mon[31209]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:51:17.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:17 smithi031 ceph-mon[31209]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:17.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:17 smithi165 ceph-mon[34818]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:51:17.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:17 smithi165 ceph-mon[34818]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:17.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:17 smithi178 ceph-mon[34759]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:51:17.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:17 smithi178 ceph-mon[34759]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:18.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.292+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/2841061214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108f50 msgr2=0x7f8bb410f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:18.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.292+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/2841061214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108f50 0x7f8bb410f780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009ca0 tx=0x7f8bac02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/2841061214 shutdown_connections 2024-09-21T02:51:18.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/2841061214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4108600 0x7f8bb4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/2841061214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108f50 0x7f8bb410f780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/2841061214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb41031c0 0x7f8bb41035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/2841061214 >> 172.21.15.31:0/2841061214 conn(0x7f8bb4079e30 msgr2=0x7f8bb407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/2841061214 shutdown_connections 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/2841061214 wait complete. 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.293+0000 7f8bb8a1c640 1 Processor -- start 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 -- start start 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb41031c0 0x7f8bb4110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 0x7f8bb41107a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb4108f50 0x7f8bb4110ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb4113d30 con 0x7f8bb4108600 2024-09-21T02:51:18.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb4113bb0 con 0x7f8bb4108f50 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb8a1c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb4113eb0 con 0x7f8bb41031c0 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 0x7f8bb41107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 0x7f8bb41107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42470/0 (socket says 172.21.15.31:42470) 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 -- 172.21.15.31:0/3657899076 learned_addr learned my addr 172.21.15.31:0/3657899076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb259c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb41031c0 0x7f8bb4110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb2d9d640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb4108f50 0x7f8bb4110ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 -- 172.21.15.31:0/3657899076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb41031c0 msgr2=0x7f8bb4110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb41031c0 0x7f8bb4110260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 -- 172.21.15.31:0/3657899076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb4108f50 msgr2=0x7f8bb4110ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb4108f50 0x7f8bb4110ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb1d9b640 1 -- 172.21.15.31:0/3657899076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb41a8920 con 0x7f8bb4108600 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.294+0000 7f8bb2d9d640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb4108f50 0x7f8bb4110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.295+0000 7f8bb1d9b640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 0x7f8bb41107a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800d950 tx=0x7f8ba800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:18.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.295+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8017070 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.295+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba80054d0 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.295+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb41a8c10 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.295+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004ca0 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.295+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb41a9150 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.296+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb41035d0 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.297+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8ba8004e40 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.298+0000 7f8ba37fe640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b9c0b65f0 0x7f8b9c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.298+0000 7f8bb259c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b9c0b65f0 0x7f8b9c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.298+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8ba80e6e80 con 0x7f8bb4108600 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.299+0000 7f8bb259c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b9c0b65f0 0x7f8b9c0b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4005fd0 tx=0x7f8ba4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:18.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.303+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ba8020070 con 0x7f8bb4108600 2024-09-21T02:51:18.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.489+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8bb4063800 con 0x7f8b9c0b65f0 2024-09-21T02:51:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.494+0000 7f8ba37fe640 1 -- 172.21.15.31:0/3657899076 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258846 (secure 0 0 0) 0x7f8bb4063800 con 0x7f8b9c0b65f0 2024-09-21T02:51:18.497 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:18.498 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:51:18.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.497+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b9c0b65f0 msgr2=0x7f8b9c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:18.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.497+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b9c0b65f0 0x7f8b9c0b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4005fd0 tx=0x7f8ba4005850 comp rx=0 tx=0).stop 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.497+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 msgr2=0x7f8bb41107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.497+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 0x7f8bb41107a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800d950 tx=0x7f8ba800de10 comp rx=0 tx=0).stop 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 shutdown_connections 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b9c0b65f0 0x7f8b9c0b8b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb4108f50 0x7f8bb4110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bb4108600 0x7f8bb41107a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 --2- 172.21.15.31:0/3657899076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb41031c0 0x7f8bb4110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 >> 172.21.15.31:0/3657899076 conn(0x7f8bb4079e30 msgr2=0x7f8bb41045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 shutdown_connections 2024-09-21T02:51:18.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:18.498+0000 7f8bb8a1c640 1 -- 172.21.15.31:0/3657899076 wait complete. 2024-09-21T02:51:18.824 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:18 smithi031 ceph-mon[31209]: pgmap v277: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:18.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:18 smithi165 ceph-mon[34818]: pgmap v277: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:18.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:18 smithi178 ceph-mon[34759]: pgmap v277: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:19.307 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-21T02:51:17.145127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657289,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":263,"num_read_kb":236,"num_write":360,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372020,"kb_used_data":10280,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569804,"statfs":{"total":1151940427776,"available":1151559479296,"internally_reserved":0,"allocated":10526720,"data_stored":6803785,"data_compressed":122844,"data_compressed_allocated":159744,"data_compressed_original":1818624,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34809,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":82,"num_write":19,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002357"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574159+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:51:16.574159+0000","last_peered":"2024-09-21T02:51:16.574159+0000","last_clean":"2024-09-21T02:51:16.574159+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:51:16.574159+0000","last_undegraded":"2024-09-21T02:51:16.574159+0000","last_fullsized":"2024-09-21T02:51:16.574159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366951+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:50:51.366951+0000","last_peered":"2024-09-21T02:50:51.366951+0000","last_clean":"2024-09-21T02:50:51.366951+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:50:51.366951+0000","last_undegraded":"2024-09-21T02:50:51.366951+0000","last_fullsized":"2024-09-21T02:50:51.366951+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367489+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:50:51.367489+0000","last_peered":"2024-09-21T02:50:51.367489+0000","last_clean":"2024-09-21T02:50:51.367489+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:50:51.367489+0000","last_undegraded":"2024-09-21T02:50:51.367489+0000","last_fullsized":"2024-09-21T02:50:51.367489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367793+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:50:51.367793+0000","last_peered":"2024-09-21T02:50:51.367793+0000","last_clean":"2024-09-21T02:50:51.367793+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:50:51.367793+0000","last_undegraded":"2024-09-21T02:50:51.367793+0000","last_fullsized":"2024-09-21T02:50:51.367793+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364196+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:50:51.364196+0000","last_peered":"2024-09-21T02:50:51.364196+0000","last_clean":"2024-09-21T02:50:51.364196+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:50:51.364196+0000","last_undegraded":"2024-09-21T02:50:51.364196+0000","last_fullsized":"2024-09-21T02:50:51.364196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578336+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:51:16.578336+0000","last_peered":"2024-09-21T02:51:16.578336+0000","last_clean":"2024-09-21T02:51:16.578336+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:51:16.578336+0000","last_undegraded":"2024-09-21T02:51:16.578336+0000","last_fullsized":"2024-09-21T02:51:16.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365493+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:50:51.365493+0000","last_peered":"2024-09-21T02:50:51.365493+0000","last_clean":"2024-09-21T02:50:51.365493+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:50:51.365493+0000","last_undegraded":"2024-09-21T02:50:51.365493+0000","last_fullsized":"2024-09-21T02:50:51.365493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366874+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:50:51.366874+0000","last_peered":"2024-09-21T02:50:51.366874+0000","last_clean":"2024-09-21T02:50:51.366874+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:50:51.366874+0000","last_undegraded":"2024-09-21T02:50:51.366874+0000","last_fullsized":"2024-09-21T02:50:51.366874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365471+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:50:51.365471+0000","last_peered":"2024-09-21T02:50:51.365471+0000","last_clean":"2024-09-21T02:50:51.365471+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:50:51.365471+0000","last_undegraded":"2024-09-21T02:50:51.365471+0000","last_fullsized":"2024-09-21T02:50:51.365471+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574699+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:51:16.574699+0000","last_peered":"2024-09-21T02:51:16.574699+0000","last_clean":"2024-09-21T02:51:16.574699+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:51:16.574699+0000","last_undegraded":"2024-09-21T02:51:16.574699+0000","last_fullsized":"2024-09-21T02:51:16.574699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366457+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:50:51.366457+0000","last_peered":"2024-09-21T02:50:51.366457+0000","last_clean":"2024-09-21T02:50:51.366457+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:50:51.366457+0000","last_undegraded":"2024-09-21T02:50:51.366457+0000","last_fullsized":"2024-09-21T02:50:51.366457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":2,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-21T02:50:53.795329+0000","last_change":"2024-09-21T02:51:16.189802+0000","last_active":"2024-09-21T02:51:16.189802+0000","last_peered":"2024-09-21T02:50:53.795329+0000","last_clean":"2024-09-21T02:50:53.795329+0000","last_became_active":"2024-09-21T02:50:50.355048+0000","last_became_peered":"2024-09-21T02:50:50.355048+0000","last_unstale":"2024-09-21T02:50:53.795329+0000","last_undegraded":"2024-09-21T02:50:53.795329+0000","last_fullsized":"2024-09-21T02:50:53.795329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.568940+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:15.568940+0000","last_peered":"2024-09-21T02:51:15.568940+0000","last_clean":"2024-09-21T02:51:15.568940+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:15.568940+0000","last_undegraded":"2024-09-21T02:51:15.568940+0000","last_fullsized":"2024-09-21T02:51:15.568940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366976+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:50:51.366976+0000","last_peered":"2024-09-21T02:50:51.366976+0000","last_clean":"2024-09-21T02:50:51.366976+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:50:51.366976+0000","last_undegraded":"2024-09-21T02:50:51.366976+0000","last_fullsized":"2024-09-21T02:50:51.366976+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367308+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:50:51.367308+0000","last_peered":"2024-09-21T02:50:51.367308+0000","last_clean":"2024-09-21T02:50:51.367308+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:50:51.367308+0000","last_undegraded":"2024-09-21T02:50:51.367308+0000","last_fullsized":"2024-09-21T02:50:51.367308+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364651+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:50:51.364651+0000","last_peered":"2024-09-21T02:50:51.364651+0000","last_clean":"2024-09-21T02:50:51.364651+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:50:51.364651+0000","last_undegraded":"2024-09-21T02:50:51.364651+0000","last_fullsized":"2024-09-21T02:50:51.364651+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574274+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:51:16.574274+0000","last_peered":"2024-09-21T02:51:16.574274+0000","last_clean":"2024-09-21T02:51:16.574274+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:51:16.574274+0000","last_undegraded":"2024-09-21T02:51:16.574274+0000","last_fullsized":"2024-09-21T02:51:16.574274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571093+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:15.571093+0000","last_peered":"2024-09-21T02:51:15.571093+0000","last_clean":"2024-09-21T02:51:15.571093+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:15.571093+0000","last_undegraded":"2024-09-21T02:51:15.571093+0000","last_fullsized":"2024-09-21T02:51:15.571093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.369695+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:50:51.369695+0000","last_peered":"2024-09-21T02:50:51.369695+0000","last_clean":"2024-09-21T02:50:51.369695+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:50:51.369695+0000","last_undegraded":"2024-09-21T02:50:51.369695+0000","last_fullsized":"2024-09-21T02:50:51.369695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573836+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:51:16.573836+0000","last_peered":"2024-09-21T02:51:16.573836+0000","last_clean":"2024-09-21T02:51:16.573836+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:51:16.573836+0000","last_undegraded":"2024-09-21T02:51:16.573836+0000","last_fullsized":"2024-09-21T02:51:16.573836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569053+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:15.569053+0000","last_peered":"2024-09-21T02:51:15.569053+0000","last_clean":"2024-09-21T02:51:15.569053+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:15.569053+0000","last_undegraded":"2024-09-21T02:51:15.569053+0000","last_fullsized":"2024-09-21T02:51:15.569053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364133+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:50:51.364133+0000","last_peered":"2024-09-21T02:50:51.364133+0000","last_clean":"2024-09-21T02:50:51.364133+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:50:51.364133+0000","last_undegraded":"2024-09-21T02:50:51.364133+0000","last_fullsized":"2024-09-21T02:50:51.364133+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366687+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:50:51.366687+0000","last_peered":"2024-09-21T02:50:51.366687+0000","last_clean":"2024-09-21T02:50:51.366687+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:50:51.366687+0000","last_undegraded":"2024-09-21T02:50:51.366687+0000","last_fullsized":"2024-09-21T02:50:51.366687+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-21T02:50:50.346201+0000","last_change":"2024-09-21T02:51:15.944950+0000","last_active":"2024-09-21T02:50:49.327862+0000","last_peered":"2024-09-21T02:50:49.327862+0000","last_clean":"2024-09-21T02:50:49.327862+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:50:50.346201+0000","last_undegraded":"2024-09-21T02:50:50.346201+0000","last_fullsized":"2024-09-21T02:50:50.346201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365463+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:50:51.365463+0000","last_peered":"2024-09-21T02:50:51.365463+0000","last_clean":"2024-09-21T02:50:51.365463+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:50:51.365463+0000","last_undegraded":"2024-09-21T02:50:51.365463+0000","last_fullsized":"2024-09-21T02:50:51.365463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.369227+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:50:51.369227+0000","last_peered":"2024-09-21T02:50:51.369227+0000","last_clean":"2024-09-21T02:50:51.369227+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:50:51.369227+0000","last_undegraded":"2024-09-21T02:50:51.369227+0000","last_fullsized":"2024-09-21T02:50:51.369227+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575552+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:51:16.575552+0000","last_peered":"2024-09-21T02:51:16.575552+0000","last_clean":"2024-09-21T02:51:16.575552+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:51:16.575552+0000","last_undegraded":"2024-09-21T02:51:16.575552+0000","last_fullsized":"2024-09-21T02:51:16.575552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569250+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:15.569250+0000","last_peered":"2024-09-21T02:51:15.569250+0000","last_clean":"2024-09-21T02:51:15.569250+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:15.569250+0000","last_undegraded":"2024-09-21T02:51:15.569250+0000","last_fullsized":"2024-09-21T02:51:15.569250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367380+0000","last_change":"2024-09-21T02:50:50.364293+0000","last_active":"2024-09-21T02:50:51.367380+0000","last_peered":"2024-09-21T02:50:51.367380+0000","last_clean":"2024-09-21T02:50:51.367380+0000","last_became_active":"2024-09-21T02:50:50.363879+0000","last_became_peered":"2024-09-21T02:50:50.363879+0000","last_unstale":"2024-09-21T02:50:51.367380+0000","last_undegraded":"2024-09-21T02:50:51.367380+0000","last_fullsized":"2024-09-21T02:50:51.367380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.466771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366966+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:50:51.366966+0000","last_peered":"2024-09-21T02:50:51.366966+0000","last_clean":"2024-09-21T02:50:51.366966+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:50:51.366966+0000","last_undegraded":"2024-09-21T02:50:51.366966+0000","last_fullsized":"2024-09-21T02:50:51.366966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367361+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:50:51.367361+0000","last_peered":"2024-09-21T02:50:51.367361+0000","last_clean":"2024-09-21T02:50:51.367361+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:50:51.367361+0000","last_undegraded":"2024-09-21T02:50:51.367361+0000","last_fullsized":"2024-09-21T02:50:51.367361+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574101+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:51:16.574101+0000","last_peered":"2024-09-21T02:51:16.574101+0000","last_clean":"2024-09-21T02:51:16.574101+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:51:16.574101+0000","last_undegraded":"2024-09-21T02:51:16.574101+0000","last_fullsized":"2024-09-21T02:51:16.574101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.363864+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:50:51.363864+0000","last_peered":"2024-09-21T02:50:51.363864+0000","last_clean":"2024-09-21T02:50:51.363864+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:50:51.363864+0000","last_undegraded":"2024-09-21T02:50:51.363864+0000","last_fullsized":"2024-09-21T02:50:51.363864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.546741+0000","last_change":"2024-09-21T02:50:48.335956+0000","last_active":"2024-09-21T02:51:12.546741+0000","last_peered":"2024-09-21T02:51:12.546741+0000","last_clean":"2024-09-21T02:51:12.546741+0000","last_became_active":"2024-09-21T02:50:48.335545+0000","last_became_peered":"2024-09-21T02:50:48.335545+0000","last_unstale":"2024-09-21T02:51:12.546741+0000","last_undegraded":"2024-09-21T02:51:12.546741+0000","last_fullsized":"2024-09-21T02:51:12.546741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:26.486593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365426+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:50:51.365426+0000","last_peered":"2024-09-21T02:50:51.365426+0000","last_clean":"2024-09-21T02:50:51.365426+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:50:51.365426+0000","last_undegraded":"2024-09-21T02:50:51.365426+0000","last_fullsized":"2024-09-21T02:50:51.365426+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569862+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:15.569862+0000","last_peered":"2024-09-21T02:51:15.569862+0000","last_clean":"2024-09-21T02:51:15.569862+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:15.569862+0000","last_undegraded":"2024-09-21T02:51:15.569862+0000","last_fullsized":"2024-09-21T02:51:15.569862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367125+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:50:51.367125+0000","last_peered":"2024-09-21T02:50:51.367125+0000","last_clean":"2024-09-21T02:50:51.367125+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:50:51.367125+0000","last_undegraded":"2024-09-21T02:50:51.367125+0000","last_fullsized":"2024-09-21T02:50:51.367125+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.433979+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:01.433979+0000","last_peered":"2024-09-21T02:51:01.433979+0000","last_clean":"2024-09-21T02:51:01.433979+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:01.433979+0000","last_undegraded":"2024-09-21T02:51:01.433979+0000","last_fullsized":"2024-09-21T02:51:01.433979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.570493+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:15.570493+0000","last_peered":"2024-09-21T02:51:15.570493+0000","last_clean":"2024-09-21T02:51:15.570493+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:15.570493+0000","last_undegraded":"2024-09-21T02:51:15.570493+0000","last_fullsized":"2024-09-21T02:51:15.570493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574703+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:51:16.574703+0000","last_peered":"2024-09-21T02:51:16.574703+0000","last_clean":"2024-09-21T02:51:16.574703+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:51:16.574703+0000","last_undegraded":"2024-09-21T02:51:16.574703+0000","last_fullsized":"2024-09-21T02:51:16.574703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364788+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:50:51.364788+0000","last_peered":"2024-09-21T02:50:51.364788+0000","last_clean":"2024-09-21T02:50:51.364788+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:50:51.364788+0000","last_undegraded":"2024-09-21T02:50:51.364788+0000","last_fullsized":"2024-09-21T02:50:51.364788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434034+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:01.434034+0000","last_peered":"2024-09-21T02:51:01.434034+0000","last_clean":"2024-09-21T02:51:01.434034+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:01.434034+0000","last_undegraded":"2024-09-21T02:51:01.434034+0000","last_fullsized":"2024-09-21T02:51:01.434034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.368031+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:50:51.368031+0000","last_peered":"2024-09-21T02:50:51.368031+0000","last_clean":"2024-09-21T02:50:51.368031+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:50:51.368031+0000","last_undegraded":"2024-09-21T02:50:51.368031+0000","last_fullsized":"2024-09-21T02:50:51.368031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.421603+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:00.421603+0000","last_peered":"2024-09-21T02:51:00.421603+0000","last_clean":"2024-09-21T02:51:00.421603+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:00.421603+0000","last_undegraded":"2024-09-21T02:51:00.421603+0000","last_fullsized":"2024-09-21T02:51:00.421603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364386+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:50:51.364386+0000","last_peered":"2024-09-21T02:50:51.364386+0000","last_clean":"2024-09-21T02:50:51.364386+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:50:51.364386+0000","last_undegraded":"2024-09-21T02:50:51.364386+0000","last_fullsized":"2024-09-21T02:50:51.364386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":88,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574148+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:51:16.574148+0000","last_peered":"2024-09-21T02:51:16.574148+0000","last_clean":"2024-09-21T02:51:16.574148+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:51:16.574148+0000","last_undegraded":"2024-09-21T02:51:16.574148+0000","last_fullsized":"2024-09-21T02:51:16.574148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59092,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":75,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364106+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:50:51.364106+0000","last_peered":"2024-09-21T02:50:51.364106+0000","last_clean":"2024-09-21T02:50:51.364106+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:50:51.364106+0000","last_undegraded":"2024-09-21T02:50:51.364106+0000","last_fullsized":"2024-09-21T02:50:51.364106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.526223+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:01.526223+0000","last_peered":"2024-09-21T02:51:01.526223+0000","last_clean":"2024-09-21T02:51:01.526223+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:01.526223+0000","last_undegraded":"2024-09-21T02:51:01.526223+0000","last_fullsized":"2024-09-21T02:51:01.526223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"peering","last_fresh":"2024-09-21T02:50:51.364322+0000","last_change":"2024-09-21T02:51:15.568971+0000","last_active":"2024-09-21T02:51:15.568971+0000","last_peered":"2024-09-21T02:50:51.364322+0000","last_clean":"2024-09-21T02:50:51.364322+0000","last_became_active":"2024-09-21T02:50:50.358686+0000","last_became_peered":"2024-09-21T02:50:50.358686+0000","last_unstale":"2024-09-21T02:50:51.364322+0000","last_undegraded":"2024-09-21T02:50:51.364322+0000","last_fullsized":"2024-09-21T02:50:51.364322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434044+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:01.434044+0000","last_peered":"2024-09-21T02:51:01.434044+0000","last_clean":"2024-09-21T02:51:01.434044+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:01.434044+0000","last_undegraded":"2024-09-21T02:51:01.434044+0000","last_fullsized":"2024-09-21T02:51:01.434044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365911+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:50:51.365911+0000","last_peered":"2024-09-21T02:50:51.365911+0000","last_clean":"2024-09-21T02:50:51.365911+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:50:51.365911+0000","last_undegraded":"2024-09-21T02:50:51.365911+0000","last_fullsized":"2024-09-21T02:50:51.365911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.546861+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:51:12.546861+0000","last_peered":"2024-09-21T02:51:12.546861+0000","last_clean":"2024-09-21T02:51:12.546861+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:51:12.546861+0000","last_undegraded":"2024-09-21T02:51:12.546861+0000","last_fullsized":"2024-09-21T02:51:12.546861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365211+0000","last_change":"2024-09-21T02:50:50.355712+0000","last_active":"2024-09-21T02:50:51.365211+0000","last_peered":"2024-09-21T02:50:51.365211+0000","last_clean":"2024-09-21T02:50:51.365211+0000","last_became_active":"2024-09-21T02:50:50.355315+0000","last_became_peered":"2024-09-21T02:50:50.355315+0000","last_unstale":"2024-09-21T02:50:51.365211+0000","last_undegraded":"2024-09-21T02:50:51.365211+0000","last_fullsized":"2024-09-21T02:50:51.365211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:52.187455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.432474+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:01.432474+0000","last_peered":"2024-09-21T02:51:01.432474+0000","last_clean":"2024-09-21T02:51:01.432474+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:01.432474+0000","last_undegraded":"2024-09-21T02:51:01.432474+0000","last_fullsized":"2024-09-21T02:51:01.432474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.368281+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:50:51.368281+0000","last_peered":"2024-09-21T02:50:51.368281+0000","last_clean":"2024-09-21T02:50:51.368281+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:50:51.368281+0000","last_undegraded":"2024-09-21T02:50:51.368281+0000","last_fullsized":"2024-09-21T02:50:51.368281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.407391+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:00.407391+0000","last_peered":"2024-09-21T02:51:00.407391+0000","last_clean":"2024-09-21T02:51:00.407391+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:00.407391+0000","last_undegraded":"2024-09-21T02:51:00.407391+0000","last_fullsized":"2024-09-21T02:51:00.407391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573938+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:51:16.573938+0000","last_peered":"2024-09-21T02:51:16.573938+0000","last_clean":"2024-09-21T02:51:16.573938+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:51:16.573938+0000","last_undegraded":"2024-09-21T02:51:16.573938+0000","last_fullsized":"2024-09-21T02:51:16.573938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569756+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:15.569756+0000","last_peered":"2024-09-21T02:51:15.569756+0000","last_clean":"2024-09-21T02:51:15.569756+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:15.569756+0000","last_undegraded":"2024-09-21T02:51:15.569756+0000","last_fullsized":"2024-09-21T02:51:15.569756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366578+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:50:51.366578+0000","last_peered":"2024-09-21T02:50:51.366578+0000","last_clean":"2024-09-21T02:50:51.366578+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:50:51.366578+0000","last_undegraded":"2024-09-21T02:50:51.366578+0000","last_fullsized":"2024-09-21T02:50:51.366578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.431282+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:01.431282+0000","last_peered":"2024-09-21T02:51:01.431282+0000","last_clean":"2024-09-21T02:51:01.431282+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:01.431282+0000","last_undegraded":"2024-09-21T02:51:01.431282+0000","last_fullsized":"2024-09-21T02:51:01.431282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366265+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:50:51.366265+0000","last_peered":"2024-09-21T02:50:51.366265+0000","last_clean":"2024-09-21T02:50:51.366265+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:50:51.366265+0000","last_undegraded":"2024-09-21T02:50:51.366265+0000","last_fullsized":"2024-09-21T02:50:51.366265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.546628+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:51:12.546628+0000","last_peered":"2024-09-21T02:51:12.546628+0000","last_clean":"2024-09-21T02:51:12.546628+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:51:12.546628+0000","last_undegraded":"2024-09-21T02:51:12.546628+0000","last_fullsized":"2024-09-21T02:51:12.546628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366459+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:50:51.366459+0000","last_peered":"2024-09-21T02:50:51.366459+0000","last_clean":"2024-09-21T02:50:51.366459+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:50:51.366459+0000","last_undegraded":"2024-09-21T02:50:51.366459+0000","last_fullsized":"2024-09-21T02:50:51.366459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365725+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:50:51.365725+0000","last_peered":"2024-09-21T02:50:51.365725+0000","last_clean":"2024-09-21T02:50:51.365725+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:50:51.365725+0000","last_undegraded":"2024-09-21T02:50:51.365725+0000","last_fullsized":"2024-09-21T02:50:51.365725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.526601+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:01.526601+0000","last_peered":"2024-09-21T02:51:01.526601+0000","last_clean":"2024-09-21T02:51:01.526601+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:01.526601+0000","last_undegraded":"2024-09-21T02:51:01.526601+0000","last_fullsized":"2024-09-21T02:51:01.526601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366088+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:50:51.366088+0000","last_peered":"2024-09-21T02:50:51.366088+0000","last_clean":"2024-09-21T02:50:51.366088+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:50:51.366088+0000","last_undegraded":"2024-09-21T02:50:51.366088+0000","last_fullsized":"2024-09-21T02:50:51.366088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:59.411327+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:50:59.411327+0000","last_peered":"2024-09-21T02:50:59.411327+0000","last_clean":"2024-09-21T02:50:59.411327+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:50:59.411327+0000","last_undegraded":"2024-09-21T02:50:59.411327+0000","last_fullsized":"2024-09-21T02:50:59.411327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569647+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:15.569647+0000","last_peered":"2024-09-21T02:51:15.569647+0000","last_clean":"2024-09-21T02:51:15.569647+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:15.569647+0000","last_undegraded":"2024-09-21T02:51:15.569647+0000","last_fullsized":"2024-09-21T02:51:15.569647+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.570011+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:51:15.570011+0000","last_peered":"2024-09-21T02:51:15.570011+0000","last_clean":"2024-09-21T02:51:15.570011+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:51:15.570011+0000","last_undegraded":"2024-09-21T02:51:15.570011+0000","last_fullsized":"2024-09-21T02:51:15.570011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367740+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:50:51.367740+0000","last_peered":"2024-09-21T02:50:51.367740+0000","last_clean":"2024-09-21T02:50:51.367740+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:50:51.367740+0000","last_undegraded":"2024-09-21T02:50:51.367740+0000","last_fullsized":"2024-09-21T02:50:51.367740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.545943+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:51:12.545943+0000","last_peered":"2024-09-21T02:51:12.545943+0000","last_clean":"2024-09-21T02:51:12.545943+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:51:12.545943+0000","last_undegraded":"2024-09-21T02:51:12.545943+0000","last_fullsized":"2024-09-21T02:51:12.545943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574384+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:51:16.574384+0000","last_peered":"2024-09-21T02:51:16.574384+0000","last_clean":"2024-09-21T02:51:16.574384+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:51:16.574384+0000","last_undegraded":"2024-09-21T02:51:16.574384+0000","last_fullsized":"2024-09-21T02:51:16.574384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569627+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:15.569627+0000","last_peered":"2024-09-21T02:51:15.569627+0000","last_clean":"2024-09-21T02:51:15.569627+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:15.569627+0000","last_undegraded":"2024-09-21T02:51:15.569627+0000","last_fullsized":"2024-09-21T02:51:15.569627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574339+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:51:16.574339+0000","last_peered":"2024-09-21T02:51:16.574339+0000","last_clean":"2024-09-21T02:51:16.574339+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:51:16.574339+0000","last_undegraded":"2024-09-21T02:51:16.574339+0000","last_fullsized":"2024-09-21T02:51:16.574339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569363+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:15.569363+0000","last_peered":"2024-09-21T02:51:15.569363+0000","last_clean":"2024-09-21T02:51:15.569363+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:15.569363+0000","last_undegraded":"2024-09-21T02:51:15.569363+0000","last_fullsized":"2024-09-21T02:51:15.569363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571231+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:15.571231+0000","last_peered":"2024-09-21T02:51:15.571231+0000","last_clean":"2024-09-21T02:51:15.571231+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:15.571231+0000","last_undegraded":"2024-09-21T02:51:15.571231+0000","last_fullsized":"2024-09-21T02:51:15.571231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365747+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:50:51.365747+0000","last_peered":"2024-09-21T02:50:51.365747+0000","last_clean":"2024-09-21T02:50:51.365747+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:50:51.365747+0000","last_undegraded":"2024-09-21T02:50:51.365747+0000","last_fullsized":"2024-09-21T02:50:51.365747+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434166+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:01.434166+0000","last_peered":"2024-09-21T02:51:01.434166+0000","last_clean":"2024-09-21T02:51:01.434166+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:01.434166+0000","last_undegraded":"2024-09-21T02:51:01.434166+0000","last_fullsized":"2024-09-21T02:51:01.434166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367497+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:50:51.367497+0000","last_peered":"2024-09-21T02:50:51.367497+0000","last_clean":"2024-09-21T02:50:51.367497+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:50:51.367497+0000","last_undegraded":"2024-09-21T02:50:51.367497+0000","last_fullsized":"2024-09-21T02:50:51.367497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:58.400841+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:50:58.400841+0000","last_peered":"2024-09-21T02:50:58.400841+0000","last_clean":"2024-09-21T02:50:58.400841+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:50:58.400841+0000","last_undegraded":"2024-09-21T02:50:58.400841+0000","last_fullsized":"2024-09-21T02:50:58.400841+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367060+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:50:51.367060+0000","last_peered":"2024-09-21T02:50:51.367060+0000","last_clean":"2024-09-21T02:50:51.367060+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:50:51.367060+0000","last_undegraded":"2024-09-21T02:50:51.367060+0000","last_fullsized":"2024-09-21T02:50:51.367060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.431863+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:01.431863+0000","last_peered":"2024-09-21T02:51:01.431863+0000","last_clean":"2024-09-21T02:51:01.431863+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:01.431863+0000","last_undegraded":"2024-09-21T02:51:01.431863+0000","last_fullsized":"2024-09-21T02:51:01.431863+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573705+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:51:16.573705+0000","last_peered":"2024-09-21T02:51:16.573705+0000","last_clean":"2024-09-21T02:51:16.573705+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:51:16.573705+0000","last_undegraded":"2024-09-21T02:51:16.573705+0000","last_fullsized":"2024-09-21T02:51:16.573705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.421542+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:00.421542+0000","last_peered":"2024-09-21T02:51:00.421542+0000","last_clean":"2024-09-21T02:51:00.421542+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:00.421542+0000","last_undegraded":"2024-09-21T02:51:00.421542+0000","last_fullsized":"2024-09-21T02:51:00.421542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:50:51.366034+0000","last_peered":"2024-09-21T02:50:51.366034+0000","last_clean":"2024-09-21T02:50:51.366034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:50:51.366034+0000","last_undegraded":"2024-09-21T02:50:51.366034+0000","last_fullsized":"2024-09-21T02:50:51.366034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:58.400577+0000","last_change":"2024-09-21T02:50:48.340162+0000","last_active":"2024-09-21T02:50:58.400577+0000","last_peered":"2024-09-21T02:50:58.400577+0000","last_clean":"2024-09-21T02:50:58.400577+0000","last_became_active":"2024-09-21T02:50:48.339615+0000","last_became_peered":"2024-09-21T02:50:48.339615+0000","last_unstale":"2024-09-21T02:50:58.400577+0000","last_undegraded":"2024-09-21T02:50:58.400577+0000","last_fullsized":"2024-09-21T02:50:58.400577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:09.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365600+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:50:51.365600+0000","last_peered":"2024-09-21T02:50:51.365600+0000","last_clean":"2024-09-21T02:50:51.365600+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:50:51.365600+0000","last_undegraded":"2024-09-21T02:50:51.365600+0000","last_fullsized":"2024-09-21T02:50:51.365600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365542+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:50:51.365542+0000","last_peered":"2024-09-21T02:50:51.365542+0000","last_clean":"2024-09-21T02:50:51.365542+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:50:51.365542+0000","last_undegraded":"2024-09-21T02:50:51.365542+0000","last_fullsized":"2024-09-21T02:50:51.365542+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.433791+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:01.433791+0000","last_peered":"2024-09-21T02:51:01.433791+0000","last_clean":"2024-09-21T02:51:01.433791+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:01.433791+0000","last_undegraded":"2024-09-21T02:51:01.433791+0000","last_fullsized":"2024-09-21T02:51:01.433791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364104+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:50:51.364104+0000","last_peered":"2024-09-21T02:50:51.364104+0000","last_clean":"2024-09-21T02:50:51.364104+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:50:51.364104+0000","last_undegraded":"2024-09-21T02:50:51.364104+0000","last_fullsized":"2024-09-21T02:50:51.364104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367975+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:50:51.367975+0000","last_peered":"2024-09-21T02:50:51.367975+0000","last_clean":"2024-09-21T02:50:51.367975+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:50:51.367975+0000","last_undegraded":"2024-09-21T02:50:51.367975+0000","last_fullsized":"2024-09-21T02:50:51.367975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.371481+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:50:51.371481+0000","last_peered":"2024-09-21T02:50:51.371481+0000","last_clean":"2024-09-21T02:50:51.371481+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:50:51.371481+0000","last_undegraded":"2024-09-21T02:50:51.371481+0000","last_fullsized":"2024-09-21T02:50:51.371481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367592+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:50:51.367592+0000","last_peered":"2024-09-21T02:50:51.367592+0000","last_clean":"2024-09-21T02:50:51.367592+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:50:51.367592+0000","last_undegraded":"2024-09-21T02:50:51.367592+0000","last_fullsized":"2024-09-21T02:50:51.367592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571411+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:51:15.571411+0000","last_peered":"2024-09-21T02:51:15.571411+0000","last_clean":"2024-09-21T02:51:15.571411+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:51:15.571411+0000","last_undegraded":"2024-09-21T02:51:15.571411+0000","last_fullsized":"2024-09-21T02:51:15.571411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66687,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":182,"num_read_kb":167,"num_write":235,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":183869,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710737,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":412332,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":1062062,"data_compressed":40948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037788,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":864256,"data_stored":411871,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":403275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":996184,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":403005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":995471,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":402475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":851968,"data_stored":402439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":455469,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":454858,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":404344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":53083,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":53694,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60828,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:51:19.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:51:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:19.857 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:19 smithi031 ceph-mon[31209]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:19.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:19 smithi165 ceph-mon[34818]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:19.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:19 smithi178 ceph-mon[34759]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:20.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 -- 172.21.15.31:0/3954438856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680102a80 msgr2=0x7fe680102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 --2- 172.21.15.31:0/3954438856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680102a80 0x7fe680102e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe670009ca0 tx=0x7fe67002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 -- 172.21.15.31:0/3954438856 shutdown_connections 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 --2- 172.21.15.31:0/3954438856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe680108d10 0x7fe68010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 --2- 172.21.15.31:0/3954438856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6801083c0 0x7fe6801087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 --2- 172.21.15.31:0/3954438856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680102a80 0x7fe680102e90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 -- 172.21.15.31:0/3954438856 >> 172.21.15.31:0/3954438856 conn(0x7fe680079740 msgr2=0x7fe680079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 -- 172.21.15.31:0/3954438856 shutdown_connections 2024-09-21T02:51:20.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.070+0000 7fe684f88640 1 -- 172.21.15.31:0/3954438856 wait complete. 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 Processor -- start 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 -- start start 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe680102a80 0x7fe680104270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6801083c0 0x7fe6801047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 0x7fe680104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe6801129e0 con 0x7fe680108d10 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe680112860 con 0x7fe6801083c0 2024-09-21T02:51:20.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe684f88640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe680112b60 con 0x7fe680102a80 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 0x7fe680104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe67ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 0x7fe680104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52276/0 (socket says 172.21.15.31:52276) 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe67ed9d640 1 -- 172.21.15.31:0/1190476408 learned_addr learned my addr 172.21.15.31:0/1190476408 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.071+0000 7fe67dd9b640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6801083c0 0x7fe6801047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67e59c640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe680102a80 0x7fe680104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67ed9d640 1 -- 172.21.15.31:0/1190476408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe680102a80 msgr2=0x7fe680104270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67ed9d640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe680102a80 0x7fe680104270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67ed9d640 1 -- 172.21.15.31:0/1190476408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6801083c0 msgr2=0x7fe6801047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67ed9d640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6801083c0 0x7fe6801047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67ed9d640 1 -- 172.21.15.31:0/1190476408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6801a87a0 con 0x7fe680108d10 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe67ed9d640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 0x7fe680104cf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe67800c970 tx=0x7fe67800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67800ae60 con 0x7fe680108d10 2024-09-21T02:51:20.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6780054d0 con 0x7fe680108d10 2024-09-21T02:51:20.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6801a8a90 con 0x7fe680108d10 2024-09-21T02:51:20.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678004d60 con 0x7fe680108d10 2024-09-21T02:51:20.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.072+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801a9000 con 0x7fe680108d10 2024-09-21T02:51:20.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.073+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe680103d00 con 0x7fe680108d10 2024-09-21T02:51:20.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.075+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe678005020 con 0x7fe680108d10 2024-09-21T02:51:20.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.076+0000 7fe66f7fe640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe6680b65f0 0x7fe6680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:20.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.076+0000 7fe67e59c640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe6680b65f0 0x7fe6680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:20.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.076+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe6780e7240 con 0x7fe680108d10 2024-09-21T02:51:20.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.076+0000 7fe67e59c640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe6680b65f0 0x7fe6680b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098d0 tx=0x7fe6700093e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:20.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.080+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe678014030 con 0x7fe680108d10 2024-09-21T02:51:20.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.264+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe680106340 con 0x7fe6680b65f0 2024-09-21T02:51:20.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.268+0000 7fe66f7fe640 1 -- 172.21.15.31:0/1190476408 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258877 (secure 0 0 0) 0x7fe680106340 con 0x7fe6680b65f0 2024-09-21T02:51:20.272 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:20.273 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe6680b65f0 msgr2=0x7fe6680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe6680b65f0 0x7fe6680b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098d0 tx=0x7fe6700093e0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 msgr2=0x7fe680104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 0x7fe680104cf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe67800c970 tx=0x7fe67800cd70 comp rx=0 tx=0).stop 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 shutdown_connections 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe6680b65f0 0x7fe6680b8b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe680108d10 0x7fe680104cf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6801083c0 0x7fe6801047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 --2- 172.21.15.31:0/1190476408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe680102a80 0x7fe680104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 >> 172.21.15.31:0/1190476408 conn(0x7fe680079740 msgr2=0x7fe680101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 shutdown_connections 2024-09-21T02:51:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:20.272+0000 7fe684f88640 1 -- 172.21.15.31:0/1190476408 wait complete. 2024-09-21T02:51:20.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:20 smithi165 ceph-mon[34818]: pgmap v278: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.4 KiB/s wr, 12 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:20 smithi178 ceph-mon[34759]: pgmap v278: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.4 KiB/s wr, 12 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:20 smithi031 ceph-mon[31209]: pgmap v278: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.4 KiB/s wr, 12 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:21.176 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-21T02:51:19.145565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657289,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":360,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376336,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565488,"statfs":{"total":1151940427776,"available":1151555059712,"internally_reserved":0,"allocated":10665984,"data_stored":6908699,"data_compressed":151889,"data_compressed_allocated":200704,"data_compressed_original":1900544,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34809,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":76,"num_write":19,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002239"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574159+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:51:16.574159+0000","last_peered":"2024-09-21T02:51:16.574159+0000","last_clean":"2024-09-21T02:51:16.574159+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:51:16.574159+0000","last_undegraded":"2024-09-21T02:51:16.574159+0000","last_fullsized":"2024-09-21T02:51:16.574159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366951+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:50:51.366951+0000","last_peered":"2024-09-21T02:50:51.366951+0000","last_clean":"2024-09-21T02:50:51.366951+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:50:51.366951+0000","last_undegraded":"2024-09-21T02:50:51.366951+0000","last_fullsized":"2024-09-21T02:50:51.366951+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367489+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:50:51.367489+0000","last_peered":"2024-09-21T02:50:51.367489+0000","last_clean":"2024-09-21T02:50:51.367489+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:50:51.367489+0000","last_undegraded":"2024-09-21T02:50:51.367489+0000","last_fullsized":"2024-09-21T02:50:51.367489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573194+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:51:16.573194+0000","last_peered":"2024-09-21T02:51:16.573194+0000","last_clean":"2024-09-21T02:51:16.573194+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:51:16.573194+0000","last_undegraded":"2024-09-21T02:51:16.573194+0000","last_fullsized":"2024-09-21T02:51:16.573194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364196+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:50:51.364196+0000","last_peered":"2024-09-21T02:50:51.364196+0000","last_clean":"2024-09-21T02:50:51.364196+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:50:51.364196+0000","last_undegraded":"2024-09-21T02:50:51.364196+0000","last_fullsized":"2024-09-21T02:50:51.364196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578336+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:51:16.578336+0000","last_peered":"2024-09-21T02:51:16.578336+0000","last_clean":"2024-09-21T02:51:16.578336+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:51:16.578336+0000","last_undegraded":"2024-09-21T02:51:16.578336+0000","last_fullsized":"2024-09-21T02:51:16.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365493+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:50:51.365493+0000","last_peered":"2024-09-21T02:50:51.365493+0000","last_clean":"2024-09-21T02:50:51.365493+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:50:51.365493+0000","last_undegraded":"2024-09-21T02:50:51.365493+0000","last_fullsized":"2024-09-21T02:50:51.365493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976088+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:51:16.976088+0000","last_peered":"2024-09-21T02:51:16.976088+0000","last_clean":"2024-09-21T02:51:16.976088+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:51:16.976088+0000","last_undegraded":"2024-09-21T02:51:16.976088+0000","last_fullsized":"2024-09-21T02:51:16.976088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977495+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:51:16.977495+0000","last_peered":"2024-09-21T02:51:16.977495+0000","last_clean":"2024-09-21T02:51:16.977495+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:51:16.977495+0000","last_undegraded":"2024-09-21T02:51:16.977495+0000","last_fullsized":"2024-09-21T02:51:16.977495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574699+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:51:16.574699+0000","last_peered":"2024-09-21T02:51:16.574699+0000","last_clean":"2024-09-21T02:51:16.574699+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:51:16.574699+0000","last_undegraded":"2024-09-21T02:51:16.574699+0000","last_fullsized":"2024-09-21T02:51:16.574699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366457+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:50:51.366457+0000","last_peered":"2024-09-21T02:50:51.366457+0000","last_clean":"2024-09-21T02:50:51.366457+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:50:51.366457+0000","last_undegraded":"2024-09-21T02:50:51.366457+0000","last_fullsized":"2024-09-21T02:50:51.366457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":2,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-21T02:50:53.795329+0000","last_change":"2024-09-21T02:51:16.189802+0000","last_active":"2024-09-21T02:51:16.189802+0000","last_peered":"2024-09-21T02:50:53.795329+0000","last_clean":"2024-09-21T02:50:53.795329+0000","last_became_active":"2024-09-21T02:50:50.355048+0000","last_became_peered":"2024-09-21T02:50:50.355048+0000","last_unstale":"2024-09-21T02:50:53.795329+0000","last_undegraded":"2024-09-21T02:50:53.795329+0000","last_fullsized":"2024-09-21T02:50:53.795329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.568940+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:15.568940+0000","last_peered":"2024-09-21T02:51:15.568940+0000","last_clean":"2024-09-21T02:51:15.568940+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:15.568940+0000","last_undegraded":"2024-09-21T02:51:15.568940+0000","last_fullsized":"2024-09-21T02:51:15.568940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366976+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:50:51.366976+0000","last_peered":"2024-09-21T02:50:51.366976+0000","last_clean":"2024-09-21T02:50:51.366976+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:50:51.366976+0000","last_undegraded":"2024-09-21T02:50:51.366976+0000","last_fullsized":"2024-09-21T02:50:51.366976+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367308+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:50:51.367308+0000","last_peered":"2024-09-21T02:50:51.367308+0000","last_clean":"2024-09-21T02:50:51.367308+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:50:51.367308+0000","last_undegraded":"2024-09-21T02:50:51.367308+0000","last_fullsized":"2024-09-21T02:50:51.367308+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976749+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:51:16.976749+0000","last_peered":"2024-09-21T02:51:16.976749+0000","last_clean":"2024-09-21T02:51:16.976749+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:51:16.976749+0000","last_undegraded":"2024-09-21T02:51:16.976749+0000","last_fullsized":"2024-09-21T02:51:16.976749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574274+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:51:16.574274+0000","last_peered":"2024-09-21T02:51:16.574274+0000","last_clean":"2024-09-21T02:51:16.574274+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:51:16.574274+0000","last_undegraded":"2024-09-21T02:51:16.574274+0000","last_fullsized":"2024-09-21T02:51:16.574274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571093+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:15.571093+0000","last_peered":"2024-09-21T02:51:15.571093+0000","last_clean":"2024-09-21T02:51:15.571093+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:15.571093+0000","last_undegraded":"2024-09-21T02:51:15.571093+0000","last_fullsized":"2024-09-21T02:51:15.571093+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575827+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:51:16.575827+0000","last_peered":"2024-09-21T02:51:16.575827+0000","last_clean":"2024-09-21T02:51:16.575827+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:51:16.575827+0000","last_undegraded":"2024-09-21T02:51:16.575827+0000","last_fullsized":"2024-09-21T02:51:16.575827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573836+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:51:16.573836+0000","last_peered":"2024-09-21T02:51:16.573836+0000","last_clean":"2024-09-21T02:51:16.573836+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:51:16.573836+0000","last_undegraded":"2024-09-21T02:51:16.573836+0000","last_fullsized":"2024-09-21T02:51:16.573836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569053+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:15.569053+0000","last_peered":"2024-09-21T02:51:15.569053+0000","last_clean":"2024-09-21T02:51:15.569053+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:15.569053+0000","last_undegraded":"2024-09-21T02:51:15.569053+0000","last_fullsized":"2024-09-21T02:51:15.569053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976953+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:51:16.976953+0000","last_peered":"2024-09-21T02:51:16.976953+0000","last_clean":"2024-09-21T02:51:16.976953+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:51:16.976953+0000","last_undegraded":"2024-09-21T02:51:16.976953+0000","last_fullsized":"2024-09-21T02:51:16.976953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975815+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:51:16.975815+0000","last_peered":"2024-09-21T02:51:16.975815+0000","last_clean":"2024-09-21T02:51:16.975815+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:51:16.975815+0000","last_undegraded":"2024-09-21T02:51:16.975815+0000","last_fullsized":"2024-09-21T02:51:16.975815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":2,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-21T02:50:50.346201+0000","last_change":"2024-09-21T02:51:15.944950+0000","last_active":"2024-09-21T02:50:49.327862+0000","last_peered":"2024-09-21T02:50:49.327862+0000","last_clean":"2024-09-21T02:50:49.327862+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:50:50.346201+0000","last_undegraded":"2024-09-21T02:50:50.346201+0000","last_fullsized":"2024-09-21T02:50:50.346201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976482+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:51:16.976482+0000","last_peered":"2024-09-21T02:51:16.976482+0000","last_clean":"2024-09-21T02:51:16.976482+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:51:16.976482+0000","last_undegraded":"2024-09-21T02:51:16.976482+0000","last_fullsized":"2024-09-21T02:51:16.976482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576864+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:51:16.576864+0000","last_peered":"2024-09-21T02:51:16.576864+0000","last_clean":"2024-09-21T02:51:16.576864+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:51:16.576864+0000","last_undegraded":"2024-09-21T02:51:16.576864+0000","last_fullsized":"2024-09-21T02:51:16.576864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575552+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:51:16.575552+0000","last_peered":"2024-09-21T02:51:16.575552+0000","last_clean":"2024-09-21T02:51:16.575552+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:51:16.575552+0000","last_undegraded":"2024-09-21T02:51:16.575552+0000","last_fullsized":"2024-09-21T02:51:16.575552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569250+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:15.569250+0000","last_peered":"2024-09-21T02:51:15.569250+0000","last_clean":"2024-09-21T02:51:15.569250+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:15.569250+0000","last_undegraded":"2024-09-21T02:51:15.569250+0000","last_fullsized":"2024-09-21T02:51:15.569250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367380+0000","last_change":"2024-09-21T02:50:50.364293+0000","last_active":"2024-09-21T02:50:51.367380+0000","last_peered":"2024-09-21T02:50:51.367380+0000","last_clean":"2024-09-21T02:50:51.367380+0000","last_became_active":"2024-09-21T02:50:50.363879+0000","last_became_peered":"2024-09-21T02:50:50.363879+0000","last_unstale":"2024-09-21T02:50:51.367380+0000","last_undegraded":"2024-09-21T02:50:51.367380+0000","last_fullsized":"2024-09-21T02:50:51.367380+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:31.466771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575696+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:51:16.575696+0000","last_peered":"2024-09-21T02:51:16.575696+0000","last_clean":"2024-09-21T02:51:16.575696+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:51:16.575696+0000","last_undegraded":"2024-09-21T02:51:16.575696+0000","last_fullsized":"2024-09-21T02:51:16.575696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576084+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:51:16.576084+0000","last_peered":"2024-09-21T02:51:16.576084+0000","last_clean":"2024-09-21T02:51:16.576084+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:51:16.576084+0000","last_undegraded":"2024-09-21T02:51:16.576084+0000","last_fullsized":"2024-09-21T02:51:16.576084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574101+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:51:16.574101+0000","last_peered":"2024-09-21T02:51:16.574101+0000","last_clean":"2024-09-21T02:51:16.574101+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:51:16.574101+0000","last_undegraded":"2024-09-21T02:51:16.574101+0000","last_fullsized":"2024-09-21T02:51:16.574101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.363864+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:50:51.363864+0000","last_peered":"2024-09-21T02:50:51.363864+0000","last_clean":"2024-09-21T02:50:51.363864+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:50:51.363864+0000","last_undegraded":"2024-09-21T02:50:51.363864+0000","last_fullsized":"2024-09-21T02:50:51.363864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.585130+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:51:17.585130+0000","last_peered":"2024-09-21T02:51:17.585130+0000","last_clean":"2024-09-21T02:51:17.585130+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:51:17.585130+0000","last_undegraded":"2024-09-21T02:51:17.585130+0000","last_fullsized":"2024-09-21T02:51:17.585130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578401+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:51:16.578401+0000","last_peered":"2024-09-21T02:51:16.578401+0000","last_clean":"2024-09-21T02:51:16.578401+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:51:16.578401+0000","last_undegraded":"2024-09-21T02:51:16.578401+0000","last_fullsized":"2024-09-21T02:51:16.578401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569862+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:15.569862+0000","last_peered":"2024-09-21T02:51:15.569862+0000","last_clean":"2024-09-21T02:51:15.569862+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:15.569862+0000","last_undegraded":"2024-09-21T02:51:15.569862+0000","last_fullsized":"2024-09-21T02:51:15.569862+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976219+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:51:16.976219+0000","last_peered":"2024-09-21T02:51:16.976219+0000","last_clean":"2024-09-21T02:51:16.976219+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:51:16.976219+0000","last_undegraded":"2024-09-21T02:51:16.976219+0000","last_fullsized":"2024-09-21T02:51:16.976219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.570493+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:15.570493+0000","last_peered":"2024-09-21T02:51:15.570493+0000","last_clean":"2024-09-21T02:51:15.570493+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:15.570493+0000","last_undegraded":"2024-09-21T02:51:15.570493+0000","last_fullsized":"2024-09-21T02:51:15.570493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574703+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:51:16.574703+0000","last_peered":"2024-09-21T02:51:16.574703+0000","last_clean":"2024-09-21T02:51:16.574703+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:51:16.574703+0000","last_undegraded":"2024-09-21T02:51:16.574703+0000","last_fullsized":"2024-09-21T02:51:16.574703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977390+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:51:16.977390+0000","last_peered":"2024-09-21T02:51:16.977390+0000","last_clean":"2024-09-21T02:51:16.977390+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:51:16.977390+0000","last_undegraded":"2024-09-21T02:51:16.977390+0000","last_fullsized":"2024-09-21T02:51:16.977390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.434034+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:01.434034+0000","last_peered":"2024-09-21T02:51:01.434034+0000","last_clean":"2024-09-21T02:51:01.434034+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:01.434034+0000","last_undegraded":"2024-09-21T02:51:01.434034+0000","last_fullsized":"2024-09-21T02:51:01.434034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.368031+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:50:51.368031+0000","last_peered":"2024-09-21T02:50:51.368031+0000","last_clean":"2024-09-21T02:50:51.368031+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:50:51.368031+0000","last_undegraded":"2024-09-21T02:50:51.368031+0000","last_fullsized":"2024-09-21T02:50:51.368031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977117+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:51:16.977117+0000","last_peered":"2024-09-21T02:51:16.977117+0000","last_clean":"2024-09-21T02:51:16.977117+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:51:16.977117+0000","last_undegraded":"2024-09-21T02:51:16.977117+0000","last_fullsized":"2024-09-21T02:51:16.977117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":88,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574148+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:51:16.574148+0000","last_peered":"2024-09-21T02:51:16.574148+0000","last_clean":"2024-09-21T02:51:16.574148+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:51:16.574148+0000","last_undegraded":"2024-09-21T02:51:16.574148+0000","last_fullsized":"2024-09-21T02:51:16.574148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59092,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":75,"num_write":29,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.364106+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:50:51.364106+0000","last_peered":"2024-09-21T02:50:51.364106+0000","last_clean":"2024-09-21T02:50:51.364106+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:50:51.364106+0000","last_undegraded":"2024-09-21T02:50:51.364106+0000","last_fullsized":"2024-09-21T02:50:51.364106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"peering","last_fresh":"2024-09-21T02:50:51.364322+0000","last_change":"2024-09-21T02:51:15.568971+0000","last_active":"2024-09-21T02:51:15.568971+0000","last_peered":"2024-09-21T02:50:51.364322+0000","last_clean":"2024-09-21T02:50:51.364322+0000","last_became_active":"2024-09-21T02:50:50.358686+0000","last_became_peered":"2024-09-21T02:50:50.358686+0000","last_unstale":"2024-09-21T02:50:51.364322+0000","last_undegraded":"2024-09-21T02:50:51.364322+0000","last_fullsized":"2024-09-21T02:50:51.364322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7,9],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365911+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:50:51.365911+0000","last_peered":"2024-09-21T02:50:51.365911+0000","last_clean":"2024-09-21T02:50:51.365911+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:50:51.365911+0000","last_undegraded":"2024-09-21T02:50:51.365911+0000","last_fullsized":"2024-09-21T02:50:51.365911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.554605+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:51:17.554605+0000","last_peered":"2024-09-21T02:51:17.554605+0000","last_clean":"2024-09-21T02:51:17.554605+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:51:17.554605+0000","last_undegraded":"2024-09-21T02:51:17.554605+0000","last_fullsized":"2024-09-21T02:51:17.554605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365211+0000","last_change":"2024-09-21T02:50:50.355712+0000","last_active":"2024-09-21T02:50:51.365211+0000","last_peered":"2024-09-21T02:50:51.365211+0000","last_clean":"2024-09-21T02:50:51.365211+0000","last_became_active":"2024-09-21T02:50:50.355315+0000","last_became_peered":"2024-09-21T02:50:50.355315+0000","last_unstale":"2024-09-21T02:50:51.365211+0000","last_undegraded":"2024-09-21T02:50:51.365211+0000","last_fullsized":"2024-09-21T02:50:51.365211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:52.187455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.432474+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:01.432474+0000","last_peered":"2024-09-21T02:51:01.432474+0000","last_clean":"2024-09-21T02:51:01.432474+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:01.432474+0000","last_undegraded":"2024-09-21T02:51:01.432474+0000","last_fullsized":"2024-09-21T02:51:01.432474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577762+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:51:16.577762+0000","last_peered":"2024-09-21T02:51:16.577762+0000","last_clean":"2024-09-21T02:51:16.577762+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:51:16.577762+0000","last_undegraded":"2024-09-21T02:51:16.577762+0000","last_fullsized":"2024-09-21T02:51:16.577762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:00.407391+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:00.407391+0000","last_peered":"2024-09-21T02:51:00.407391+0000","last_clean":"2024-09-21T02:51:00.407391+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:00.407391+0000","last_undegraded":"2024-09-21T02:51:00.407391+0000","last_fullsized":"2024-09-21T02:51:00.407391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573938+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:51:16.573938+0000","last_peered":"2024-09-21T02:51:16.573938+0000","last_clean":"2024-09-21T02:51:16.573938+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:51:16.573938+0000","last_undegraded":"2024-09-21T02:51:16.573938+0000","last_fullsized":"2024-09-21T02:51:16.573938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569756+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:15.569756+0000","last_peered":"2024-09-21T02:51:15.569756+0000","last_clean":"2024-09-21T02:51:15.569756+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:15.569756+0000","last_undegraded":"2024-09-21T02:51:15.569756+0000","last_fullsized":"2024-09-21T02:51:15.569756+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366578+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:50:51.366578+0000","last_peered":"2024-09-21T02:50:51.366578+0000","last_clean":"2024-09-21T02:50:51.366578+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:50:51.366578+0000","last_undegraded":"2024-09-21T02:50:51.366578+0000","last_fullsized":"2024-09-21T02:50:51.366578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366265+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:50:51.366265+0000","last_peered":"2024-09-21T02:50:51.366265+0000","last_clean":"2024-09-21T02:50:51.366265+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:50:51.366265+0000","last_undegraded":"2024-09-21T02:50:51.366265+0000","last_fullsized":"2024-09-21T02:50:51.366265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.554359+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:51:17.554359+0000","last_peered":"2024-09-21T02:51:17.554359+0000","last_clean":"2024-09-21T02:51:17.554359+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:51:17.554359+0000","last_undegraded":"2024-09-21T02:51:17.554359+0000","last_fullsized":"2024-09-21T02:51:17.554359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.366459+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:50:51.366459+0000","last_peered":"2024-09-21T02:50:51.366459+0000","last_clean":"2024-09-21T02:50:51.366459+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:50:51.366459+0000","last_undegraded":"2024-09-21T02:50:51.366459+0000","last_fullsized":"2024-09-21T02:50:51.366459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365725+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:50:51.365725+0000","last_peered":"2024-09-21T02:50:51.365725+0000","last_clean":"2024-09-21T02:50:51.365725+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:50:51.365725+0000","last_undegraded":"2024-09-21T02:50:51.365725+0000","last_fullsized":"2024-09-21T02:50:51.365725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573447+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573447+0000","last_peered":"2024-09-21T02:51:16.573447+0000","last_clean":"2024-09-21T02:51:16.573447+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573447+0000","last_undegraded":"2024-09-21T02:51:16.573447+0000","last_fullsized":"2024-09-21T02:51:16.573447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976061+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:51:16.976061+0000","last_peered":"2024-09-21T02:51:16.976061+0000","last_clean":"2024-09-21T02:51:16.976061+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:51:16.976061+0000","last_undegraded":"2024-09-21T02:51:16.976061+0000","last_fullsized":"2024-09-21T02:51:16.976061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:59.411327+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:50:59.411327+0000","last_peered":"2024-09-21T02:50:59.411327+0000","last_clean":"2024-09-21T02:50:59.411327+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:50:59.411327+0000","last_undegraded":"2024-09-21T02:50:59.411327+0000","last_fullsized":"2024-09-21T02:50:59.411327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569647+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:15.569647+0000","last_peered":"2024-09-21T02:51:15.569647+0000","last_clean":"2024-09-21T02:51:15.569647+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:15.569647+0000","last_undegraded":"2024-09-21T02:51:15.569647+0000","last_fullsized":"2024-09-21T02:51:15.569647+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.570011+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:51:15.570011+0000","last_peered":"2024-09-21T02:51:15.570011+0000","last_clean":"2024-09-21T02:51:15.570011+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:51:15.570011+0000","last_undegraded":"2024-09-21T02:51:15.570011+0000","last_fullsized":"2024-09-21T02:51:15.570011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578540+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:51:16.578540+0000","last_peered":"2024-09-21T02:51:16.578540+0000","last_clean":"2024-09-21T02:51:16.578540+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:51:16.578540+0000","last_undegraded":"2024-09-21T02:51:16.578540+0000","last_fullsized":"2024-09-21T02:51:16.578540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:12.545943+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:51:12.545943+0000","last_peered":"2024-09-21T02:51:12.545943+0000","last_clean":"2024-09-21T02:51:12.545943+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:51:12.545943+0000","last_undegraded":"2024-09-21T02:51:12.545943+0000","last_fullsized":"2024-09-21T02:51:12.545943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574384+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:51:16.574384+0000","last_peered":"2024-09-21T02:51:16.574384+0000","last_clean":"2024-09-21T02:51:16.574384+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:51:16.574384+0000","last_undegraded":"2024-09-21T02:51:16.574384+0000","last_fullsized":"2024-09-21T02:51:16.574384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569627+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:15.569627+0000","last_peered":"2024-09-21T02:51:15.569627+0000","last_clean":"2024-09-21T02:51:15.569627+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:15.569627+0000","last_undegraded":"2024-09-21T02:51:15.569627+0000","last_fullsized":"2024-09-21T02:51:15.569627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574339+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:51:16.574339+0000","last_peered":"2024-09-21T02:51:16.574339+0000","last_clean":"2024-09-21T02:51:16.574339+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:51:16.574339+0000","last_undegraded":"2024-09-21T02:51:16.574339+0000","last_fullsized":"2024-09-21T02:51:16.574339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.569363+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:15.569363+0000","last_peered":"2024-09-21T02:51:15.569363+0000","last_clean":"2024-09-21T02:51:15.569363+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:15.569363+0000","last_undegraded":"2024-09-21T02:51:15.569363+0000","last_fullsized":"2024-09-21T02:51:15.569363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571231+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:15.571231+0000","last_peered":"2024-09-21T02:51:15.571231+0000","last_clean":"2024-09-21T02:51:15.571231+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:15.571231+0000","last_undegraded":"2024-09-21T02:51:15.571231+0000","last_fullsized":"2024-09-21T02:51:15.571231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976502+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:51:16.976502+0000","last_peered":"2024-09-21T02:51:16.976502+0000","last_clean":"2024-09-21T02:51:16.976502+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:51:16.976502+0000","last_undegraded":"2024-09-21T02:51:16.976502+0000","last_fullsized":"2024-09-21T02:51:16.976502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573953+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.573953+0000","last_peered":"2024-09-21T02:51:16.573953+0000","last_clean":"2024-09-21T02:51:16.573953+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.573953+0000","last_undegraded":"2024-09-21T02:51:16.573953+0000","last_fullsized":"2024-09-21T02:51:16.573953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367497+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:50:51.367497+0000","last_peered":"2024-09-21T02:50:51.367497+0000","last_clean":"2024-09-21T02:50:51.367497+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:50:51.367497+0000","last_undegraded":"2024-09-21T02:50:51.367497+0000","last_fullsized":"2024-09-21T02:50:51.367497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976881+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976881+0000","last_peered":"2024-09-21T02:51:16.976881+0000","last_clean":"2024-09-21T02:51:16.976881+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976881+0000","last_undegraded":"2024-09-21T02:51:16.976881+0000","last_fullsized":"2024-09-21T02:51:16.976881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367060+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:50:51.367060+0000","last_peered":"2024-09-21T02:50:51.367060+0000","last_clean":"2024-09-21T02:50:51.367060+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:50:51.367060+0000","last_undegraded":"2024-09-21T02:50:51.367060+0000","last_fullsized":"2024-09-21T02:50:51.367060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975953+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.975953+0000","last_peered":"2024-09-21T02:51:16.975953+0000","last_clean":"2024-09-21T02:51:16.975953+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.975953+0000","last_undegraded":"2024-09-21T02:51:16.975953+0000","last_fullsized":"2024-09-21T02:51:16.975953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573705+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:51:16.573705+0000","last_peered":"2024-09-21T02:51:16.573705+0000","last_clean":"2024-09-21T02:51:16.573705+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:51:16.573705+0000","last_undegraded":"2024-09-21T02:51:16.573705+0000","last_fullsized":"2024-09-21T02:51:16.573705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575838+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:16.575838+0000","last_peered":"2024-09-21T02:51:16.575838+0000","last_clean":"2024-09-21T02:51:16.575838+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:16.575838+0000","last_undegraded":"2024-09-21T02:51:16.575838+0000","last_fullsized":"2024-09-21T02:51:16.575838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:51:16.976034+0000","last_peered":"2024-09-21T02:51:16.976034+0000","last_clean":"2024-09-21T02:51:16.976034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:51:16.976034+0000","last_undegraded":"2024-09-21T02:51:16.976034+0000","last_fullsized":"2024-09-21T02:51:16.976034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:58.400577+0000","last_change":"2024-09-21T02:50:48.340162+0000","last_active":"2024-09-21T02:50:58.400577+0000","last_peered":"2024-09-21T02:50:58.400577+0000","last_clean":"2024-09-21T02:50:58.400577+0000","last_became_active":"2024-09-21T02:50:48.339615+0000","last_became_peered":"2024-09-21T02:50:48.339615+0000","last_unstale":"2024-09-21T02:50:58.400577+0000","last_undegraded":"2024-09-21T02:50:58.400577+0000","last_fullsized":"2024-09-21T02:50:58.400577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:09.326695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.365600+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:50:51.365600+0000","last_peered":"2024-09-21T02:50:51.365600+0000","last_clean":"2024-09-21T02:50:51.365600+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:50:51.365600+0000","last_undegraded":"2024-09-21T02:50:51.365600+0000","last_fullsized":"2024-09-21T02:50:51.365600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578106+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:51:16.578106+0000","last_peered":"2024-09-21T02:51:16.578106+0000","last_clean":"2024-09-21T02:51:16.578106+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:51:16.578106+0000","last_undegraded":"2024-09-21T02:51:16.578106+0000","last_fullsized":"2024-09-21T02:51:16.578106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:51:01.433791+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:01.433791+0000","last_peered":"2024-09-21T02:51:01.433791+0000","last_clean":"2024-09-21T02:51:01.433791+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:01.433791+0000","last_undegraded":"2024-09-21T02:51:01.433791+0000","last_fullsized":"2024-09-21T02:51:01.433791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977018+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:51:16.977018+0000","last_peered":"2024-09-21T02:51:16.977018+0000","last_clean":"2024-09-21T02:51:16.977018+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:51:16.977018+0000","last_undegraded":"2024-09-21T02:51:16.977018+0000","last_fullsized":"2024-09-21T02:51:16.977018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T02:50:51.367975+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:50:51.367975+0000","last_peered":"2024-09-21T02:50:51.367975+0000","last_clean":"2024-09-21T02:50:51.367975+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:50:51.367975+0000","last_undegraded":"2024-09-21T02:50:51.367975+0000","last_fullsized":"2024-09-21T02:50:51.367975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976572+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976572+0000","last_peered":"2024-09-21T02:51:16.976572+0000","last_clean":"2024-09-21T02:51:16.976572+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976572+0000","last_undegraded":"2024-09-21T02:51:16.976572+0000","last_fullsized":"2024-09-21T02:51:16.976572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976425+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:51:16.976425+0000","last_peered":"2024-09-21T02:51:16.976425+0000","last_clean":"2024-09-21T02:51:16.976425+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:51:16.976425+0000","last_undegraded":"2024-09-21T02:51:16.976425+0000","last_fullsized":"2024-09-21T02:51:16.976425+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:51:15.571411+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:51:15.571411+0000","last_peered":"2024-09-21T02:51:15.571411+0000","last_clean":"2024-09-21T02:51:15.571411+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:51:15.571411+0000","last_undegraded":"2024-09-21T02:51:15.571411+0000","last_fullsized":"2024-09-21T02:51:15.571411+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66687,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":193,"num_read_kb":167,"num_write":235,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":200173,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710737,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":412332,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":991232,"data_stored":1062062,"data_compressed":40948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037788,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":864256,"data_stored":411871,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":403275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":996184,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":403005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013193,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":421210,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":480330,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":917504,"data_stored":480732,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":404344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61235,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60833,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60828,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:51:21.180 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-21T02:51:21.180 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 77, 'state': 'peering', 'last_fresh': '2024-09-21T02:50:53.795329+0000', 'last_change': '2024-09-21T02:51:16.189802+0000', 'last_active': '2024-09-21T02:51:16.189802+0000', 'last_peered': '2024-09-21T02:50:53.795329+0000', 'last_clean': '2024-09-21T02:50:53.795329+0000', 'last_became_active': '2024-09-21T02:50:50.355048+0000', 'last_became_peered': '2024-09-21T02:50:50.355048+0000', 'last_unstale': '2024-09-21T02:50:53.795329+0000', 'last_undegraded': '2024-09-21T02:50:53.795329+0000', 'last_fullsized': '2024-09-21T02:50:53.795329+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 75, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'last_clean_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 10, 8], 'acting': [9, 10, 8], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2024-09-21T02:51:21.180 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-21T02:51:21.180 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 77, 'state': 'peering', 'last_fresh': '2024-09-21T02:50:50.346201+0000', 'last_change': '2024-09-21T02:51:15.944950+0000', 'last_active': '2024-09-21T02:50:49.327862+0000', 'last_peered': '2024-09-21T02:50:49.327862+0000', 'last_clean': '2024-09-21T02:50:49.327862+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:50:50.346201+0000', 'last_undegraded': '2024-09-21T02:50:50.346201+0000', 'last_fullsized': '2024-09-21T02:50:50.346201+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 75, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'last_clean_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 9, 3], 'acting': [6, 9, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:51:21.181 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-21T02:51:21.181 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 15, 'reported_epoch': 76, 'state': 'peering', 'last_fresh': '2024-09-21T02:50:51.364322+0000', 'last_change': '2024-09-21T02:51:15.568971+0000', 'last_active': '2024-09-21T02:51:15.568971+0000', 'last_peered': '2024-09-21T02:50:51.364322+0000', 'last_clean': '2024-09-21T02:50:51.364322+0000', 'last_became_active': '2024-09-21T02:50:50.358686+0000', 'last_became_peered': '2024-09-21T02:50:50.358686+0000', 'last_unstale': '2024-09-21T02:50:51.364322+0000', 'last_undegraded': '2024-09-21T02:50:51.364322+0000', 'last_fullsized': '2024-09-21T02:50:51.364322+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 75, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'last_clean_scrub_stamp': '2024-09-21T02:50:49.327862+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7, 9], 'acting': [6, 7, 9], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 7, 9], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:51:21.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-21T02:51:21.444 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:21.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:21 smithi031 ceph-mon[31209]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:21.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:21 smithi165 ceph-mon[34818]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:21.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.946+0000 7f895ca95640 1 -- 172.21.15.31:0/2315555312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958102a80 msgr2=0x7f8958102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:21.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.946+0000 7f895ca95640 1 --2- 172.21.15.31:0/2315555312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958102a80 0x7f8958102e90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f894802a3f0 comp rx=0 tx=0).stop 2024-09-21T02:51:21.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 -- 172.21.15.31:0/2315555312 shutdown_connections 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 --2- 172.21.15.31:0/2315555312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89581083c0 0x7f89581087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 --2- 172.21.15.31:0/2315555312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958108d10 0x7f895810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 --2- 172.21.15.31:0/2315555312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958102a80 0x7f8958102e90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 -- 172.21.15.31:0/2315555312 >> 172.21.15.31:0/2315555312 conn(0x7f8958079740 msgr2=0x7f8958079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 -- 172.21.15.31:0/2315555312 shutdown_connections 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 -- 172.21.15.31:0/2315555312 wait complete. 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.947+0000 7f895ca95640 1 Processor -- start 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 -- start start 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958102a80 0x7f895819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89581083c0 0x7f895819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 0x7f895819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:21.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89581129e0 con 0x7f8958108d10 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8958112860 con 0x7f8958102a80 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895ca95640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8958112b60 con 0x7f89581083c0 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 0x7f895819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 0x7f895819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52296/0 (socket says 172.21.15.31:52296) 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 -- 172.21.15.31:0/2304185627 learned_addr learned my addr 172.21.15.31:0/2304185627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8955d9b640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89581083c0 0x7f895819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895659c640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958102a80 0x7f895819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 -- 172.21.15.31:0/2304185627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89581083c0 msgr2=0x7f895819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89581083c0 0x7f895819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 -- 172.21.15.31:0/2304185627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958102a80 msgr2=0x7f895819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958102a80 0x7f895819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:21.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f8956d9d640 1 -- 172.21.15.31:0/2304185627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89581a6550 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.948+0000 7f895659c640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958102a80 0x7f895819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.949+0000 7f8956d9d640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 0x7f895819bdd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f895000cce0 tx=0x7f895000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.949+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895000f450 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.949+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89581a6840 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.949+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f895000faa0 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.949+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950018b70 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.949+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581a6d30 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.950+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8958103d00 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.951+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8950018d10 con 0x7f8958108d10 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.952+0000 7f89477fe640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f89400b6620 0x7f89400b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.952+0000 7f895659c640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f89400b6620 0x7f89400b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:21.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.952+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f89500e7b50 con 0x7f8958108d10 2024-09-21T02:51:21.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.953+0000 7f895659c640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f89400b6620 0x7f89400b8b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8948000c00 tx=0x7f8948031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:21.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:21.955+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8950014030 con 0x7f8958108d10 2024-09-21T02:51:21.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:21 smithi178 ceph-mon[34759]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.217+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8958102e90 con 0x7f8958108d10 2024-09-21T02:51:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.217+0000 7f89477fe640 1 -- 172.21.15.31:0/2304185627 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3428 (secure 0 0 0) 0x7f89500ae210 con 0x7f8958108d10 2024-09-21T02:51:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:22.219 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":435,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726887000,"num_in_osds":12,"osd_in_since":1726886978,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":96},{"state_name":"peering","count":1}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":657289,"bytes_used":385482752,"bytes_avail":1151554945024,"bytes_total":1151940427776,"inactive_pgs_ratio":0.010309278033673763,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":140,"recovering_keys_per_sec":1,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"read_bytes_sec":4478,"write_bytes_sec":1407,"read_op_per_sec":8,"write_op_per_sec":0},"fsmap":{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14838},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14850},{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24583},{"filesystem_id":1,"rank":1,"name":"k","status":"up:standby-replay","gid":24602},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24625},{"filesystem_id":1,"rank":4,"name":"e","status":"up:active","gid":24632},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24637},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24655},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":24659},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24695}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"services":{"dashboard":"https://172.21.15.31:8443/"}},"servicemap":{"epoch":18,"modified":"2024-09-21T02:51:19.145747+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f89400b6620 msgr2=0x7f89400b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f89400b6620 0x7f89400b8b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8948000c00 tx=0x7f8948031400 comp rx=0 tx=0).stop 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 msgr2=0x7f895819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 0x7f895819bdd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f895000cce0 tx=0x7f895000a620 comp rx=0 tx=0).stop 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 shutdown_connections 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f89400b6620 0x7f89400b8b30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89581083c0 0x7f895819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8958108d10 0x7f895819bdd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 --2- 172.21.15.31:0/2304185627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8958102a80 0x7f895819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.220+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 >> 172.21.15.31:0/2304185627 conn(0x7f8958079740 msgr2=0x7f89581014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.221+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 shutdown_connections 2024-09-21T02:51:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:22.221+0000 7f895ca95640 1 -- 172.21.15.31:0/2304185627 wait complete. 2024-09-21T02:51:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:22 smithi165 ceph-mon[34818]: pgmap v279: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:22 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2304185627' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T02:51:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:22 smithi178 ceph-mon[34759]: pgmap v279: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:22 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2304185627' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T02:51:23.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:22 smithi031 ceph-mon[31209]: pgmap v279: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:23.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:22 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2304185627' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T02:51:23.041 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T02:51:23.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:51:23.307 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:23.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 -- 172.21.15.31:0/1394726727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109500 msgr2=0x7fda30109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:23.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 --2- 172.21.15.31:0/1394726727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109500 0x7fda30109910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda24009900 tx=0x7fda2402a410 comp rx=0 tx=0).stop 2024-09-21T02:51:23.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 -- 172.21.15.31:0/1394726727 shutdown_connections 2024-09-21T02:51:23.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 --2- 172.21.15.31:0/1394726727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109e50 0x7fda30110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:23.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 --2- 172.21.15.31:0/1394726727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109500 0x7fda30109910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 --2- 172.21.15.31:0/1394726727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 0x7fda301023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 -- 172.21.15.31:0/1394726727 >> 172.21.15.31:0/1394726727 conn(0x7fda30079f90 msgr2=0x7fda3007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 -- 172.21.15.31:0/1394726727 shutdown_connections 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.801+0000 7fda3577e640 1 -- 172.21.15.31:0/1394726727 wait complete. 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 Processor -- start 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 -- start start 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 0x7fda3019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109500 0x7fda3019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 0x7fda3019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113b10 con 0x7fda30109e50 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113990 con 0x7fda30101fc0 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda3577e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda30113c90 con 0x7fda30109500 2024-09-21T02:51:23.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 0x7fda3019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 0x7fda3019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52304/0 (socket says 172.21.15.31:52304) 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2f7fe640 1 -- 172.21.15.31:0/2948078293 learned_addr learned my addr 172.21.15.31:0/2948078293 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2e7fc640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109500 0x7fda3019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2effd640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 0x7fda3019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2f7fe640 1 -- 172.21.15.31:0/2948078293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109500 msgr2=0x7fda3019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2f7fe640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109500 0x7fda3019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.802+0000 7fda2f7fe640 1 -- 172.21.15.31:0/2948078293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 msgr2=0x7fda3019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda2f7fe640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 0x7fda3019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda2f7fe640 1 -- 172.21.15.31:0/2948078293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301a0370 con 0x7fda30109e50 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda2effd640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 0x7fda3019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda2e7fc640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109500 0x7fda3019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda2f7fe640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 0x7fda3019fd00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda2800cad0 tx=0x7fda2800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:23.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2800ae20 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda301a0660 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda28005020 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2800f6b0 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.803+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a6e80 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.804+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda301023d0 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.806+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fda280051d0 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.806+0000 7fda17fff640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fda180b6620 0x7fda180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.807+0000 7fda2effd640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fda180b6620 0x7fda180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.807+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fda280e7b40 con 0x7fda30109e50 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.807+0000 7fda2effd640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fda180b6620 0x7fda180b8b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda2000abe0 tx=0x7fda2000a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.810+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda28014030 con 0x7fda30109e50 2024-09-21T02:51:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.994+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda30063800 con 0x7fda180b6620 2024-09-21T02:51:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:23.999+0000 7fda17fff640 1 -- 172.21.15.31:0/2948078293 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259059 (secure 0 0 0) 0x7fda30063800 con 0x7fda180b6620 2024-09-21T02:51:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:51:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fda180b6620 msgr2=0x7fda180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fda180b6620 0x7fda180b8b30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda2000abe0 tx=0x7fda2000a5c0 comp rx=0 tx=0).stop 2024-09-21T02:51:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 msgr2=0x7fda3019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 0x7fda3019fd00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda2800cad0 tx=0x7fda2800cf90 comp rx=0 tx=0).stop 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 shutdown_connections 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fda180b6620 0x7fda180b8b30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda30109500 0x7fda3019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda30109e50 0x7fda3019fd00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.001+0000 7fda3577e640 1 --2- 172.21.15.31:0/2948078293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda30101fc0 0x7fda3019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.002+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 >> 172.21.15.31:0/2948078293 conn(0x7fda30079f90 msgr2=0x7fda301075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.002+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 shutdown_connections 2024-09-21T02:51:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:24.002+0000 7fda3577e640 1 -- 172.21.15.31:0/2948078293 wait complete. 2024-09-21T02:51:24.854 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-21T02:51:23.146627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661546,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":310,"num_read_kb":266,"num_write":365,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":296,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376492,"kb_used_data":10552,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565332,"statfs":{"total":1151940427776,"available":1151554899968,"internally_reserved":0,"allocated":10805248,"data_stored":7010088,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12409,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":12,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002231"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574159+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:51:16.574159+0000","last_peered":"2024-09-21T02:51:16.574159+0000","last_clean":"2024-09-21T02:51:16.574159+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:51:16.574159+0000","last_undegraded":"2024-09-21T02:51:16.574159+0000","last_fullsized":"2024-09-21T02:51:16.574159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576740+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:51:16.576740+0000","last_peered":"2024-09-21T02:51:16.576740+0000","last_clean":"2024-09-21T02:51:16.576740+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:51:16.576740+0000","last_undegraded":"2024-09-21T02:51:16.576740+0000","last_fullsized":"2024-09-21T02:51:16.576740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576913+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:51:16.576913+0000","last_peered":"2024-09-21T02:51:16.576913+0000","last_clean":"2024-09-21T02:51:16.576913+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:51:16.576913+0000","last_undegraded":"2024-09-21T02:51:16.576913+0000","last_fullsized":"2024-09-21T02:51:16.576913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573194+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:51:16.573194+0000","last_peered":"2024-09-21T02:51:16.573194+0000","last_clean":"2024-09-21T02:51:16.573194+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:51:16.573194+0000","last_undegraded":"2024-09-21T02:51:16.573194+0000","last_fullsized":"2024-09-21T02:51:16.573194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572183+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:51:16.572183+0000","last_peered":"2024-09-21T02:51:16.572183+0000","last_clean":"2024-09-21T02:51:16.572183+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:51:16.572183+0000","last_undegraded":"2024-09-21T02:51:16.572183+0000","last_fullsized":"2024-09-21T02:51:16.572183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578336+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:51:16.578336+0000","last_peered":"2024-09-21T02:51:16.578336+0000","last_clean":"2024-09-21T02:51:16.578336+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:51:16.578336+0000","last_undegraded":"2024-09-21T02:51:16.578336+0000","last_fullsized":"2024-09-21T02:51:16.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576570+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:51:16.576570+0000","last_peered":"2024-09-21T02:51:16.576570+0000","last_clean":"2024-09-21T02:51:16.576570+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:51:16.576570+0000","last_undegraded":"2024-09-21T02:51:16.576570+0000","last_fullsized":"2024-09-21T02:51:16.576570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976088+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:51:16.976088+0000","last_peered":"2024-09-21T02:51:16.976088+0000","last_clean":"2024-09-21T02:51:16.976088+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:51:16.976088+0000","last_undegraded":"2024-09-21T02:51:16.976088+0000","last_fullsized":"2024-09-21T02:51:16.976088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977495+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:51:16.977495+0000","last_peered":"2024-09-21T02:51:16.977495+0000","last_clean":"2024-09-21T02:51:16.977495+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:51:16.977495+0000","last_undegraded":"2024-09-21T02:51:16.977495+0000","last_fullsized":"2024-09-21T02:51:16.977495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574699+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:51:16.574699+0000","last_peered":"2024-09-21T02:51:16.574699+0000","last_clean":"2024-09-21T02:51:16.574699+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:51:16.574699+0000","last_undegraded":"2024-09-21T02:51:16.574699+0000","last_fullsized":"2024-09-21T02:51:16.574699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576637+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:51:16.576637+0000","last_peered":"2024-09-21T02:51:16.576637+0000","last_clean":"2024-09-21T02:51:16.576637+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:51:16.576637+0000","last_undegraded":"2024-09-21T02:51:16.576637+0000","last_fullsized":"2024-09-21T02:51:16.576637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576458+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:51:16.576458+0000","last_peered":"2024-09-21T02:51:16.576458+0000","last_clean":"2024-09-21T02:51:16.576458+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:51:16.576458+0000","last_undegraded":"2024-09-21T02:51:16.576458+0000","last_fullsized":"2024-09-21T02:51:16.576458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573699+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:16.573699+0000","last_peered":"2024-09-21T02:51:16.573699+0000","last_clean":"2024-09-21T02:51:16.573699+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:16.573699+0000","last_undegraded":"2024-09-21T02:51:16.573699+0000","last_fullsized":"2024-09-21T02:51:16.573699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576623+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:51:16.576623+0000","last_peered":"2024-09-21T02:51:16.576623+0000","last_clean":"2024-09-21T02:51:16.576623+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:51:16.576623+0000","last_undegraded":"2024-09-21T02:51:16.576623+0000","last_fullsized":"2024-09-21T02:51:16.576623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576412+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:51:16.576412+0000","last_peered":"2024-09-21T02:51:16.576412+0000","last_clean":"2024-09-21T02:51:16.576412+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:51:16.576412+0000","last_undegraded":"2024-09-21T02:51:16.576412+0000","last_fullsized":"2024-09-21T02:51:16.576412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976749+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:51:16.976749+0000","last_peered":"2024-09-21T02:51:16.976749+0000","last_clean":"2024-09-21T02:51:16.976749+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:51:16.976749+0000","last_undegraded":"2024-09-21T02:51:16.976749+0000","last_fullsized":"2024-09-21T02:51:16.976749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574274+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:51:16.574274+0000","last_peered":"2024-09-21T02:51:16.574274+0000","last_clean":"2024-09-21T02:51:16.574274+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:51:16.574274+0000","last_undegraded":"2024-09-21T02:51:16.574274+0000","last_fullsized":"2024-09-21T02:51:16.574274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571557+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:16.571557+0000","last_peered":"2024-09-21T02:51:16.571557+0000","last_clean":"2024-09-21T02:51:16.571557+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:16.571557+0000","last_undegraded":"2024-09-21T02:51:16.571557+0000","last_fullsized":"2024-09-21T02:51:16.571557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575827+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:51:16.575827+0000","last_peered":"2024-09-21T02:51:16.575827+0000","last_clean":"2024-09-21T02:51:16.575827+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:51:16.575827+0000","last_undegraded":"2024-09-21T02:51:16.575827+0000","last_fullsized":"2024-09-21T02:51:16.575827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573836+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:51:16.573836+0000","last_peered":"2024-09-21T02:51:16.573836+0000","last_clean":"2024-09-21T02:51:16.573836+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:51:16.573836+0000","last_undegraded":"2024-09-21T02:51:16.573836+0000","last_fullsized":"2024-09-21T02:51:16.573836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573770+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:16.573770+0000","last_peered":"2024-09-21T02:51:16.573770+0000","last_clean":"2024-09-21T02:51:16.573770+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:16.573770+0000","last_undegraded":"2024-09-21T02:51:16.573770+0000","last_fullsized":"2024-09-21T02:51:16.573770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976953+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:51:16.976953+0000","last_peered":"2024-09-21T02:51:16.976953+0000","last_clean":"2024-09-21T02:51:16.976953+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:51:16.976953+0000","last_undegraded":"2024-09-21T02:51:16.976953+0000","last_fullsized":"2024-09-21T02:51:16.976953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975815+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:51:16.975815+0000","last_peered":"2024-09-21T02:51:16.975815+0000","last_clean":"2024-09-21T02:51:16.975815+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:51:16.975815+0000","last_undegraded":"2024-09-21T02:51:16.975815+0000","last_fullsized":"2024-09-21T02:51:16.975815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977399+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:51:16.977399+0000","last_peered":"2024-09-21T02:51:16.977399+0000","last_clean":"2024-09-21T02:51:16.977399+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:51:16.977399+0000","last_undegraded":"2024-09-21T02:51:16.977399+0000","last_fullsized":"2024-09-21T02:51:16.977399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976482+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:51:16.976482+0000","last_peered":"2024-09-21T02:51:16.976482+0000","last_clean":"2024-09-21T02:51:16.976482+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:51:16.976482+0000","last_undegraded":"2024-09-21T02:51:16.976482+0000","last_fullsized":"2024-09-21T02:51:16.976482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576864+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:51:16.576864+0000","last_peered":"2024-09-21T02:51:16.576864+0000","last_clean":"2024-09-21T02:51:16.576864+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:51:16.576864+0000","last_undegraded":"2024-09-21T02:51:16.576864+0000","last_fullsized":"2024-09-21T02:51:16.576864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575552+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:51:16.575552+0000","last_peered":"2024-09-21T02:51:16.575552+0000","last_clean":"2024-09-21T02:51:16.575552+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:51:16.575552+0000","last_undegraded":"2024-09-21T02:51:16.575552+0000","last_fullsized":"2024-09-21T02:51:16.575552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574009+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:16.574009+0000","last_peered":"2024-09-21T02:51:16.574009+0000","last_clean":"2024-09-21T02:51:16.574009+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:16.574009+0000","last_undegraded":"2024-09-21T02:51:16.574009+0000","last_fullsized":"2024-09-21T02:51:16.574009+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580839+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:51:16.580839+0000","last_peered":"2024-09-21T02:51:16.580839+0000","last_clean":"2024-09-21T02:51:16.580839+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:51:16.580839+0000","last_undegraded":"2024-09-21T02:51:16.580839+0000","last_fullsized":"2024-09-21T02:51:16.580839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575696+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:51:16.575696+0000","last_peered":"2024-09-21T02:51:16.575696+0000","last_clean":"2024-09-21T02:51:16.575696+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:51:16.575696+0000","last_undegraded":"2024-09-21T02:51:16.575696+0000","last_fullsized":"2024-09-21T02:51:16.575696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576084+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:51:16.576084+0000","last_peered":"2024-09-21T02:51:16.576084+0000","last_clean":"2024-09-21T02:51:16.576084+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:51:16.576084+0000","last_undegraded":"2024-09-21T02:51:16.576084+0000","last_fullsized":"2024-09-21T02:51:16.576084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574101+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:51:16.574101+0000","last_peered":"2024-09-21T02:51:16.574101+0000","last_clean":"2024-09-21T02:51:16.574101+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:51:16.574101+0000","last_undegraded":"2024-09-21T02:51:16.574101+0000","last_fullsized":"2024-09-21T02:51:16.574101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572143+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:51:16.572143+0000","last_peered":"2024-09-21T02:51:16.572143+0000","last_clean":"2024-09-21T02:51:16.572143+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:51:16.572143+0000","last_undegraded":"2024-09-21T02:51:16.572143+0000","last_fullsized":"2024-09-21T02:51:16.572143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.585130+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:51:17.585130+0000","last_peered":"2024-09-21T02:51:17.585130+0000","last_clean":"2024-09-21T02:51:17.585130+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:51:17.585130+0000","last_undegraded":"2024-09-21T02:51:17.585130+0000","last_fullsized":"2024-09-21T02:51:17.585130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578401+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:51:16.578401+0000","last_peered":"2024-09-21T02:51:16.578401+0000","last_clean":"2024-09-21T02:51:16.578401+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:51:16.578401+0000","last_undegraded":"2024-09-21T02:51:16.578401+0000","last_fullsized":"2024-09-21T02:51:16.578401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976219+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:51:16.976219+0000","last_peered":"2024-09-21T02:51:16.976219+0000","last_clean":"2024-09-21T02:51:16.976219+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:51:16.976219+0000","last_undegraded":"2024-09-21T02:51:16.976219+0000","last_fullsized":"2024-09-21T02:51:16.976219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571256+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:16.571256+0000","last_peered":"2024-09-21T02:51:16.571256+0000","last_clean":"2024-09-21T02:51:16.571256+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:16.571256+0000","last_undegraded":"2024-09-21T02:51:16.571256+0000","last_fullsized":"2024-09-21T02:51:16.571256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:21.559789+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:51:21.559789+0000","last_peered":"2024-09-21T02:51:21.559789+0000","last_clean":"2024-09-21T02:51:21.559789+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:51:21.559789+0000","last_undegraded":"2024-09-21T02:51:21.559789+0000","last_fullsized":"2024-09-21T02:51:21.559789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977390+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:51:16.977390+0000","last_peered":"2024-09-21T02:51:16.977390+0000","last_clean":"2024-09-21T02:51:16.977390+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:51:16.977390+0000","last_undegraded":"2024-09-21T02:51:16.977390+0000","last_fullsized":"2024-09-21T02:51:16.977390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577020+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:51:16.577020+0000","last_peered":"2024-09-21T02:51:16.577020+0000","last_clean":"2024-09-21T02:51:16.577020+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:51:16.577020+0000","last_undegraded":"2024-09-21T02:51:16.577020+0000","last_fullsized":"2024-09-21T02:51:16.577020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977117+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:51:16.977117+0000","last_peered":"2024-09-21T02:51:16.977117+0000","last_clean":"2024-09-21T02:51:16.977117+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:51:16.977117+0000","last_undegraded":"2024-09-21T02:51:16.977117+0000","last_fullsized":"2024-09-21T02:51:16.977117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":100,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:21.590566+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:51:21.590566+0000","last_peered":"2024-09-21T02:51:21.590566+0000","last_clean":"2024-09-21T02:51:21.590566+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:51:21.590566+0000","last_undegraded":"2024-09-21T02:51:21.590566+0000","last_fullsized":"2024-09-21T02:51:21.590566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63349,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":85,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572844+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:51:16.572844+0000","last_peered":"2024-09-21T02:51:16.572844+0000","last_clean":"2024-09-21T02:51:16.572844+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:51:16.572844+0000","last_undegraded":"2024-09-21T02:51:16.572844+0000","last_fullsized":"2024-09-21T02:51:16.572844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578577+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:51:16.578577+0000","last_peered":"2024-09-21T02:51:16.578577+0000","last_clean":"2024-09-21T02:51:16.578577+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:51:16.578577+0000","last_undegraded":"2024-09-21T02:51:16.578577+0000","last_fullsized":"2024-09-21T02:51:16.578577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576983+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:51:16.576983+0000","last_peered":"2024-09-21T02:51:16.576983+0000","last_clean":"2024-09-21T02:51:16.576983+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:51:16.576983+0000","last_undegraded":"2024-09-21T02:51:16.576983+0000","last_fullsized":"2024-09-21T02:51:16.576983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.554605+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:51:17.554605+0000","last_peered":"2024-09-21T02:51:17.554605+0000","last_clean":"2024-09-21T02:51:17.554605+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:51:17.554605+0000","last_undegraded":"2024-09-21T02:51:17.554605+0000","last_fullsized":"2024-09-21T02:51:17.554605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580840+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:51:16.580840+0000","last_peered":"2024-09-21T02:51:16.580840+0000","last_clean":"2024-09-21T02:51:16.580840+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:51:16.580840+0000","last_undegraded":"2024-09-21T02:51:16.580840+0000","last_fullsized":"2024-09-21T02:51:16.580840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577762+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:51:16.577762+0000","last_peered":"2024-09-21T02:51:16.577762+0000","last_clean":"2024-09-21T02:51:16.577762+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:51:16.577762+0000","last_undegraded":"2024-09-21T02:51:16.577762+0000","last_fullsized":"2024-09-21T02:51:16.577762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573938+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:51:16.573938+0000","last_peered":"2024-09-21T02:51:16.573938+0000","last_clean":"2024-09-21T02:51:16.573938+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:51:16.573938+0000","last_undegraded":"2024-09-21T02:51:16.573938+0000","last_fullsized":"2024-09-21T02:51:16.573938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576883+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:51:16.576883+0000","last_peered":"2024-09-21T02:51:16.576883+0000","last_clean":"2024-09-21T02:51:16.576883+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:51:16.576883+0000","last_undegraded":"2024-09-21T02:51:16.576883+0000","last_fullsized":"2024-09-21T02:51:16.576883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576759+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:51:16.576759+0000","last_peered":"2024-09-21T02:51:16.576759+0000","last_clean":"2024-09-21T02:51:16.576759+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:51:16.576759+0000","last_undegraded":"2024-09-21T02:51:16.576759+0000","last_fullsized":"2024-09-21T02:51:16.576759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.554359+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:51:17.554359+0000","last_peered":"2024-09-21T02:51:17.554359+0000","last_clean":"2024-09-21T02:51:17.554359+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:51:17.554359+0000","last_undegraded":"2024-09-21T02:51:17.554359+0000","last_fullsized":"2024-09-21T02:51:17.554359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576701+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:51:16.576701+0000","last_peered":"2024-09-21T02:51:16.576701+0000","last_clean":"2024-09-21T02:51:16.576701+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:51:16.576701+0000","last_undegraded":"2024-09-21T02:51:16.576701+0000","last_fullsized":"2024-09-21T02:51:16.576701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573447+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573447+0000","last_peered":"2024-09-21T02:51:16.573447+0000","last_clean":"2024-09-21T02:51:16.573447+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573447+0000","last_undegraded":"2024-09-21T02:51:16.573447+0000","last_fullsized":"2024-09-21T02:51:16.573447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976061+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:51:16.976061+0000","last_peered":"2024-09-21T02:51:16.976061+0000","last_clean":"2024-09-21T02:51:16.976061+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:51:16.976061+0000","last_undegraded":"2024-09-21T02:51:16.976061+0000","last_fullsized":"2024-09-21T02:51:16.976061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576775+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576775+0000","last_peered":"2024-09-21T02:51:16.576775+0000","last_clean":"2024-09-21T02:51:16.576775+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576775+0000","last_undegraded":"2024-09-21T02:51:16.576775+0000","last_fullsized":"2024-09-21T02:51:16.576775+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573136+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:16.573136+0000","last_peered":"2024-09-21T02:51:16.573136+0000","last_clean":"2024-09-21T02:51:16.573136+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:16.573136+0000","last_undegraded":"2024-09-21T02:51:16.573136+0000","last_fullsized":"2024-09-21T02:51:16.573136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:19.556940+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:51:19.556940+0000","last_peered":"2024-09-21T02:51:19.556940+0000","last_clean":"2024-09-21T02:51:19.556940+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:51:19.556940+0000","last_undegraded":"2024-09-21T02:51:19.556940+0000","last_fullsized":"2024-09-21T02:51:19.556940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578540+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:51:16.578540+0000","last_peered":"2024-09-21T02:51:16.578540+0000","last_clean":"2024-09-21T02:51:16.578540+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:51:16.578540+0000","last_undegraded":"2024-09-21T02:51:16.578540+0000","last_fullsized":"2024-09-21T02:51:16.578540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:17.553696+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:51:17.553696+0000","last_peered":"2024-09-21T02:51:17.553696+0000","last_clean":"2024-09-21T02:51:17.553696+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:51:17.553696+0000","last_undegraded":"2024-09-21T02:51:17.553696+0000","last_fullsized":"2024-09-21T02:51:17.553696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574384+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:51:16.574384+0000","last_peered":"2024-09-21T02:51:16.574384+0000","last_clean":"2024-09-21T02:51:16.574384+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:51:16.574384+0000","last_undegraded":"2024-09-21T02:51:16.574384+0000","last_fullsized":"2024-09-21T02:51:16.574384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571552+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571552+0000","last_peered":"2024-09-21T02:51:16.571552+0000","last_clean":"2024-09-21T02:51:16.571552+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571552+0000","last_undegraded":"2024-09-21T02:51:16.571552+0000","last_fullsized":"2024-09-21T02:51:16.571552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574339+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:51:16.574339+0000","last_peered":"2024-09-21T02:51:16.574339+0000","last_clean":"2024-09-21T02:51:16.574339+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:51:16.574339+0000","last_undegraded":"2024-09-21T02:51:16.574339+0000","last_fullsized":"2024-09-21T02:51:16.574339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574116+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:16.574116+0000","last_peered":"2024-09-21T02:51:16.574116+0000","last_clean":"2024-09-21T02:51:16.574116+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:16.574116+0000","last_undegraded":"2024-09-21T02:51:16.574116+0000","last_fullsized":"2024-09-21T02:51:16.574116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571740+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:16.571740+0000","last_peered":"2024-09-21T02:51:16.571740+0000","last_clean":"2024-09-21T02:51:16.571740+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:16.571740+0000","last_undegraded":"2024-09-21T02:51:16.571740+0000","last_fullsized":"2024-09-21T02:51:16.571740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976502+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:51:16.976502+0000","last_peered":"2024-09-21T02:51:16.976502+0000","last_clean":"2024-09-21T02:51:16.976502+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:51:16.976502+0000","last_undegraded":"2024-09-21T02:51:16.976502+0000","last_fullsized":"2024-09-21T02:51:16.976502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573953+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.573953+0000","last_peered":"2024-09-21T02:51:16.573953+0000","last_clean":"2024-09-21T02:51:16.573953+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.573953+0000","last_undegraded":"2024-09-21T02:51:16.573953+0000","last_fullsized":"2024-09-21T02:51:16.573953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976881+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976881+0000","last_peered":"2024-09-21T02:51:16.976881+0000","last_clean":"2024-09-21T02:51:16.976881+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976881+0000","last_undegraded":"2024-09-21T02:51:16.976881+0000","last_fullsized":"2024-09-21T02:51:16.976881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571934+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:51:16.571934+0000","last_peered":"2024-09-21T02:51:16.571934+0000","last_clean":"2024-09-21T02:51:16.571934+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:51:16.571934+0000","last_undegraded":"2024-09-21T02:51:16.571934+0000","last_fullsized":"2024-09-21T02:51:16.571934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975953+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.975953+0000","last_peered":"2024-09-21T02:51:16.975953+0000","last_clean":"2024-09-21T02:51:16.975953+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.975953+0000","last_undegraded":"2024-09-21T02:51:16.975953+0000","last_fullsized":"2024-09-21T02:51:16.975953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573705+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:51:16.573705+0000","last_peered":"2024-09-21T02:51:16.573705+0000","last_clean":"2024-09-21T02:51:16.573705+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:51:16.573705+0000","last_undegraded":"2024-09-21T02:51:16.573705+0000","last_fullsized":"2024-09-21T02:51:16.573705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575838+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:16.575838+0000","last_peered":"2024-09-21T02:51:16.575838+0000","last_clean":"2024-09-21T02:51:16.575838+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:16.575838+0000","last_undegraded":"2024-09-21T02:51:16.575838+0000","last_fullsized":"2024-09-21T02:51:16.575838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:51:16.976034+0000","last_peered":"2024-09-21T02:51:16.976034+0000","last_clean":"2024-09-21T02:51:16.976034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:51:16.976034+0000","last_undegraded":"2024-09-21T02:51:16.976034+0000","last_fullsized":"2024-09-21T02:51:16.976034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576863+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:51:16.576863+0000","last_peered":"2024-09-21T02:51:16.576863+0000","last_clean":"2024-09-21T02:51:16.576863+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:51:16.576863+0000","last_undegraded":"2024-09-21T02:51:16.576863+0000","last_fullsized":"2024-09-21T02:51:16.576863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578106+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:51:16.578106+0000","last_peered":"2024-09-21T02:51:16.578106+0000","last_clean":"2024-09-21T02:51:16.578106+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:51:16.578106+0000","last_undegraded":"2024-09-21T02:51:16.578106+0000","last_fullsized":"2024-09-21T02:51:16.578106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977018+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:51:16.977018+0000","last_peered":"2024-09-21T02:51:16.977018+0000","last_clean":"2024-09-21T02:51:16.977018+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:51:16.977018+0000","last_undegraded":"2024-09-21T02:51:16.977018+0000","last_fullsized":"2024-09-21T02:51:16.977018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577070+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:51:16.577070+0000","last_peered":"2024-09-21T02:51:16.577070+0000","last_clean":"2024-09-21T02:51:16.577070+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:51:16.577070+0000","last_undegraded":"2024-09-21T02:51:16.577070+0000","last_fullsized":"2024-09-21T02:51:16.577070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976572+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976572+0000","last_peered":"2024-09-21T02:51:16.976572+0000","last_clean":"2024-09-21T02:51:16.976572+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976572+0000","last_undegraded":"2024-09-21T02:51:16.976572+0000","last_fullsized":"2024-09-21T02:51:16.976572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976425+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:51:16.976425+0000","last_peered":"2024-09-21T02:51:16.976425+0000","last_clean":"2024-09-21T02:51:16.976425+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:51:16.976425+0000","last_undegraded":"2024-09-21T02:51:16.976425+0000","last_fullsized":"2024-09-21T02:51:16.976425+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:19.556065+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:51:19.556065+0000","last_peered":"2024-09-21T02:51:19.556065+0000","last_clean":"2024-09-21T02:51:19.556065+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:51:19.556065+0000","last_undegraded":"2024-09-21T02:51:19.556065+0000","last_fullsized":"2024-09-21T02:51:19.556065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70944,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":229,"num_read_kb":197,"num_write":240,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":204318,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710738,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":860160,"data_stored":421192,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874263,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075067,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234017,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":1013906,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":860160,"data_stored":420727,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013193,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":421210,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":480330,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":917504,"data_stored":480732,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":422066,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61235,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60833,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65085,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:51:24.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:24 smithi165 ceph-mon[34818]: pgmap v280: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:24 smithi178 ceph-mon[34759]: pgmap v280: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:24 smithi031 ceph-mon[31209]: pgmap v280: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 10 op/s; 140 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:25.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:25 smithi165 ceph-mon[34818]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:25.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:25 smithi178 ceph-mon[34759]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:26.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:25 smithi031 ceph-mon[31209]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:26.879 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:26 smithi178 ceph-mon[34759]: pgmap v281: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 11 op/s; 117 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:26 smithi165 ceph-mon[34818]: pgmap v281: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 11 op/s; 117 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:26 smithi031 ceph-mon[31209]: pgmap v281: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 11 op/s; 117 B/s, 1 keys/s, 1 objects/s recovering 2024-09-21T02:51:27.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:51:28.133 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 -- 172.21.15.31:0/2809612615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 msgr2=0x7f17d8110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 --2- 172.21.15.31:0/2809612615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d8110680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009ca0 tx=0x7f17d002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 -- 172.21.15.31:0/2809612615 shutdown_connections 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 --2- 172.21.15.31:0/2809612615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d8110680 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 --2- 172.21.15.31:0/2809612615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d81062e0 0x7f17d81066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 --2- 172.21.15.31:0/2809612615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17d8102000 0x7f17d8102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 -- 172.21.15.31:0/2809612615 >> 172.21.15.31:0/2809612615 conn(0x7f17d8079e40 msgr2=0x7f17d807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 -- 172.21.15.31:0/2809612615 shutdown_connections 2024-09-21T02:51:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.603+0000 7f17deefe640 1 -- 172.21.15.31:0/2809612615 wait complete. 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 Processor -- start 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 -- start start 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d8102000 0x7f17d819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17d81062e0 0x7f17d819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8113ad0 con 0x7f17d8106c30 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8113950 con 0x7f17d8102000 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17deefe640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8113c50 con 0x7f17d81062e0 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52312/0 (socket says 172.21.15.31:52312) 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 -- 172.21.15.31:0/1908353410 learned_addr learned my addr 172.21.15.31:0/1908353410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17d7fff640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17d81062e0 0x7f17d819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dcc9a640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d8102000 0x7f17d819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 -- 172.21.15.31:0/1908353410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17d81062e0 msgr2=0x7f17d819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17d81062e0 0x7f17d819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 -- 172.21.15.31:0/1908353410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d8102000 msgr2=0x7f17d819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.604+0000 7f17dd49b640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d8102000 0x7f17d819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17dd49b640 1 -- 172.21.15.31:0/1908353410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d81a0350 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17dd49b640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d819fc20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005cc0 tx=0x7f17d0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0003b70 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d00318f0 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d81a05e0 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d002f950 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.605+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81a6da0 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.606+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d8102410 con 0x7f17d8106c30 2024-09-21T02:51:28.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.608+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f17d0004b40 con 0x7f17d8106c30 2024-09-21T02:51:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.609+0000 7f17d5ffb640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f17c80b6670 0x7f17c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.609+0000 7f17dcc9a640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f17c80b6670 0x7f17c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.609+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f17d01088c0 con 0x7f17d8106c30 2024-09-21T02:51:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.609+0000 7f17dcc9a640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f17c80b6670 0x7f17c80b8b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009b60 tx=0x7f17cc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:28.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.612+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17d00ceef0 con 0x7f17d8106c30 2024-09-21T02:51:28.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.799+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17d8063800 con 0x7f17c80b6670 2024-09-21T02:51:28.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:28 smithi031 ceph-mon[31209]: pgmap v282: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 106 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T02:51:28.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.804+0000 7f17d5ffb640 1 -- 172.21.15.31:0/1908353410 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259060 (secure 0 0 0) 0x7f17d8063800 con 0x7f17c80b6670 2024-09-21T02:51:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:28.808 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f17c80b6670 msgr2=0x7f17c80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f17c80b6670 0x7f17c80b8b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009b60 tx=0x7f17cc005f90 comp rx=0 tx=0).stop 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 msgr2=0x7f17d819fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d819fc20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005cc0 tx=0x7f17d0031700 comp rx=0 tx=0).stop 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 shutdown_connections 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f17c80b6670 0x7f17c80b8b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17d81062e0 0x7f17d819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17d8106c30 0x7f17d819fc20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 --2- 172.21.15.31:0/1908353410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17d8102000 0x7f17d819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:28.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 >> 172.21.15.31:0/1908353410 conn(0x7f17d8079e40 msgr2=0x7f17d81074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:28.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 shutdown_connections 2024-09-21T02:51:28.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:28.807+0000 7f17deefe640 1 -- 172.21.15.31:0/1908353410 wait complete. 2024-09-21T02:51:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:28 smithi165 ceph-mon[34818]: pgmap v282: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 106 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:28 smithi178 ceph-mon[34759]: pgmap v282: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s; 106 B/s, 0 keys/s, 1 objects/s recovering 2024-09-21T02:51:29.629 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-21T02:51:27.147803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661546,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":360,"num_read_kb":291,"num_write":365,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376492,"kb_used_data":10560,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565332,"statfs":{"total":1151940427776,"available":1151554899968,"internally_reserved":0,"allocated":10813440,"data_stored":7018602,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12409,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":75,"num_write":8,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.577797"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574159+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:51:16.574159+0000","last_peered":"2024-09-21T02:51:16.574159+0000","last_clean":"2024-09-21T02:51:16.574159+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:51:16.574159+0000","last_undegraded":"2024-09-21T02:51:16.574159+0000","last_fullsized":"2024-09-21T02:51:16.574159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576740+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:51:16.576740+0000","last_peered":"2024-09-21T02:51:16.576740+0000","last_clean":"2024-09-21T02:51:16.576740+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:51:16.576740+0000","last_undegraded":"2024-09-21T02:51:16.576740+0000","last_fullsized":"2024-09-21T02:51:16.576740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576913+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:51:16.576913+0000","last_peered":"2024-09-21T02:51:16.576913+0000","last_clean":"2024-09-21T02:51:16.576913+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:51:16.576913+0000","last_undegraded":"2024-09-21T02:51:16.576913+0000","last_fullsized":"2024-09-21T02:51:16.576913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573194+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:51:16.573194+0000","last_peered":"2024-09-21T02:51:16.573194+0000","last_clean":"2024-09-21T02:51:16.573194+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:51:16.573194+0000","last_undegraded":"2024-09-21T02:51:16.573194+0000","last_fullsized":"2024-09-21T02:51:16.573194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572183+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:51:16.572183+0000","last_peered":"2024-09-21T02:51:16.572183+0000","last_clean":"2024-09-21T02:51:16.572183+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:51:16.572183+0000","last_undegraded":"2024-09-21T02:51:16.572183+0000","last_fullsized":"2024-09-21T02:51:16.572183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578336+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:51:16.578336+0000","last_peered":"2024-09-21T02:51:16.578336+0000","last_clean":"2024-09-21T02:51:16.578336+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:51:16.578336+0000","last_undegraded":"2024-09-21T02:51:16.578336+0000","last_fullsized":"2024-09-21T02:51:16.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576570+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:51:16.576570+0000","last_peered":"2024-09-21T02:51:16.576570+0000","last_clean":"2024-09-21T02:51:16.576570+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:51:16.576570+0000","last_undegraded":"2024-09-21T02:51:16.576570+0000","last_fullsized":"2024-09-21T02:51:16.576570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976088+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:51:16.976088+0000","last_peered":"2024-09-21T02:51:16.976088+0000","last_clean":"2024-09-21T02:51:16.976088+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:51:16.976088+0000","last_undegraded":"2024-09-21T02:51:16.976088+0000","last_fullsized":"2024-09-21T02:51:16.976088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977495+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:51:16.977495+0000","last_peered":"2024-09-21T02:51:16.977495+0000","last_clean":"2024-09-21T02:51:16.977495+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:51:16.977495+0000","last_undegraded":"2024-09-21T02:51:16.977495+0000","last_fullsized":"2024-09-21T02:51:16.977495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574699+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:51:16.574699+0000","last_peered":"2024-09-21T02:51:16.574699+0000","last_clean":"2024-09-21T02:51:16.574699+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:51:16.574699+0000","last_undegraded":"2024-09-21T02:51:16.574699+0000","last_fullsized":"2024-09-21T02:51:16.574699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576637+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:51:16.576637+0000","last_peered":"2024-09-21T02:51:16.576637+0000","last_clean":"2024-09-21T02:51:16.576637+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:51:16.576637+0000","last_undegraded":"2024-09-21T02:51:16.576637+0000","last_fullsized":"2024-09-21T02:51:16.576637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576458+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:51:16.576458+0000","last_peered":"2024-09-21T02:51:16.576458+0000","last_clean":"2024-09-21T02:51:16.576458+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:51:16.576458+0000","last_undegraded":"2024-09-21T02:51:16.576458+0000","last_fullsized":"2024-09-21T02:51:16.576458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573699+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:16.573699+0000","last_peered":"2024-09-21T02:51:16.573699+0000","last_clean":"2024-09-21T02:51:16.573699+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:16.573699+0000","last_undegraded":"2024-09-21T02:51:16.573699+0000","last_fullsized":"2024-09-21T02:51:16.573699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576623+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:51:16.576623+0000","last_peered":"2024-09-21T02:51:16.576623+0000","last_clean":"2024-09-21T02:51:16.576623+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:51:16.576623+0000","last_undegraded":"2024-09-21T02:51:16.576623+0000","last_fullsized":"2024-09-21T02:51:16.576623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576412+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:51:16.576412+0000","last_peered":"2024-09-21T02:51:16.576412+0000","last_clean":"2024-09-21T02:51:16.576412+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:51:16.576412+0000","last_undegraded":"2024-09-21T02:51:16.576412+0000","last_fullsized":"2024-09-21T02:51:16.576412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976749+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:51:16.976749+0000","last_peered":"2024-09-21T02:51:16.976749+0000","last_clean":"2024-09-21T02:51:16.976749+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:51:16.976749+0000","last_undegraded":"2024-09-21T02:51:16.976749+0000","last_fullsized":"2024-09-21T02:51:16.976749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574274+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:51:16.574274+0000","last_peered":"2024-09-21T02:51:16.574274+0000","last_clean":"2024-09-21T02:51:16.574274+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:51:16.574274+0000","last_undegraded":"2024-09-21T02:51:16.574274+0000","last_fullsized":"2024-09-21T02:51:16.574274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571557+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:16.571557+0000","last_peered":"2024-09-21T02:51:16.571557+0000","last_clean":"2024-09-21T02:51:16.571557+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:16.571557+0000","last_undegraded":"2024-09-21T02:51:16.571557+0000","last_fullsized":"2024-09-21T02:51:16.571557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575827+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:51:16.575827+0000","last_peered":"2024-09-21T02:51:16.575827+0000","last_clean":"2024-09-21T02:51:16.575827+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:51:16.575827+0000","last_undegraded":"2024-09-21T02:51:16.575827+0000","last_fullsized":"2024-09-21T02:51:16.575827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573836+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:51:16.573836+0000","last_peered":"2024-09-21T02:51:16.573836+0000","last_clean":"2024-09-21T02:51:16.573836+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:51:16.573836+0000","last_undegraded":"2024-09-21T02:51:16.573836+0000","last_fullsized":"2024-09-21T02:51:16.573836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573770+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:16.573770+0000","last_peered":"2024-09-21T02:51:16.573770+0000","last_clean":"2024-09-21T02:51:16.573770+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:16.573770+0000","last_undegraded":"2024-09-21T02:51:16.573770+0000","last_fullsized":"2024-09-21T02:51:16.573770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976953+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:51:16.976953+0000","last_peered":"2024-09-21T02:51:16.976953+0000","last_clean":"2024-09-21T02:51:16.976953+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:51:16.976953+0000","last_undegraded":"2024-09-21T02:51:16.976953+0000","last_fullsized":"2024-09-21T02:51:16.976953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975815+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:51:16.975815+0000","last_peered":"2024-09-21T02:51:16.975815+0000","last_clean":"2024-09-21T02:51:16.975815+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:51:16.975815+0000","last_undegraded":"2024-09-21T02:51:16.975815+0000","last_fullsized":"2024-09-21T02:51:16.975815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977399+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:51:16.977399+0000","last_peered":"2024-09-21T02:51:16.977399+0000","last_clean":"2024-09-21T02:51:16.977399+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:51:16.977399+0000","last_undegraded":"2024-09-21T02:51:16.977399+0000","last_fullsized":"2024-09-21T02:51:16.977399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976482+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:51:16.976482+0000","last_peered":"2024-09-21T02:51:16.976482+0000","last_clean":"2024-09-21T02:51:16.976482+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:51:16.976482+0000","last_undegraded":"2024-09-21T02:51:16.976482+0000","last_fullsized":"2024-09-21T02:51:16.976482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576864+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:51:16.576864+0000","last_peered":"2024-09-21T02:51:16.576864+0000","last_clean":"2024-09-21T02:51:16.576864+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:51:16.576864+0000","last_undegraded":"2024-09-21T02:51:16.576864+0000","last_fullsized":"2024-09-21T02:51:16.576864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575552+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:51:16.575552+0000","last_peered":"2024-09-21T02:51:16.575552+0000","last_clean":"2024-09-21T02:51:16.575552+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:51:16.575552+0000","last_undegraded":"2024-09-21T02:51:16.575552+0000","last_fullsized":"2024-09-21T02:51:16.575552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574009+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:16.574009+0000","last_peered":"2024-09-21T02:51:16.574009+0000","last_clean":"2024-09-21T02:51:16.574009+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:16.574009+0000","last_undegraded":"2024-09-21T02:51:16.574009+0000","last_fullsized":"2024-09-21T02:51:16.574009+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580839+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:51:16.580839+0000","last_peered":"2024-09-21T02:51:16.580839+0000","last_clean":"2024-09-21T02:51:16.580839+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:51:16.580839+0000","last_undegraded":"2024-09-21T02:51:16.580839+0000","last_fullsized":"2024-09-21T02:51:16.580839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575696+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:51:16.575696+0000","last_peered":"2024-09-21T02:51:16.575696+0000","last_clean":"2024-09-21T02:51:16.575696+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:51:16.575696+0000","last_undegraded":"2024-09-21T02:51:16.575696+0000","last_fullsized":"2024-09-21T02:51:16.575696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576084+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:51:16.576084+0000","last_peered":"2024-09-21T02:51:16.576084+0000","last_clean":"2024-09-21T02:51:16.576084+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:51:16.576084+0000","last_undegraded":"2024-09-21T02:51:16.576084+0000","last_fullsized":"2024-09-21T02:51:16.576084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574101+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:51:16.574101+0000","last_peered":"2024-09-21T02:51:16.574101+0000","last_clean":"2024-09-21T02:51:16.574101+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:51:16.574101+0000","last_undegraded":"2024-09-21T02:51:16.574101+0000","last_fullsized":"2024-09-21T02:51:16.574101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572143+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:51:16.572143+0000","last_peered":"2024-09-21T02:51:16.572143+0000","last_clean":"2024-09-21T02:51:16.572143+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:51:16.572143+0000","last_undegraded":"2024-09-21T02:51:16.572143+0000","last_fullsized":"2024-09-21T02:51:16.572143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":68,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:22.591716+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:51:22.591716+0000","last_peered":"2024-09-21T02:51:22.591716+0000","last_clean":"2024-09-21T02:51:22.591716+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:51:22.591716+0000","last_undegraded":"2024-09-21T02:51:22.591716+0000","last_fullsized":"2024-09-21T02:51:22.591716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578401+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:51:16.578401+0000","last_peered":"2024-09-21T02:51:16.578401+0000","last_clean":"2024-09-21T02:51:16.578401+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:51:16.578401+0000","last_undegraded":"2024-09-21T02:51:16.578401+0000","last_fullsized":"2024-09-21T02:51:16.578401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976219+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:51:16.976219+0000","last_peered":"2024-09-21T02:51:16.976219+0000","last_clean":"2024-09-21T02:51:16.976219+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:51:16.976219+0000","last_undegraded":"2024-09-21T02:51:16.976219+0000","last_fullsized":"2024-09-21T02:51:16.976219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571256+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:16.571256+0000","last_peered":"2024-09-21T02:51:16.571256+0000","last_clean":"2024-09-21T02:51:16.571256+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:16.571256+0000","last_undegraded":"2024-09-21T02:51:16.571256+0000","last_fullsized":"2024-09-21T02:51:16.571256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:26.567233+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:51:26.567233+0000","last_peered":"2024-09-21T02:51:26.567233+0000","last_clean":"2024-09-21T02:51:26.567233+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:51:26.567233+0000","last_undegraded":"2024-09-21T02:51:26.567233+0000","last_fullsized":"2024-09-21T02:51:26.567233+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977390+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:51:16.977390+0000","last_peered":"2024-09-21T02:51:16.977390+0000","last_clean":"2024-09-21T02:51:16.977390+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:51:16.977390+0000","last_undegraded":"2024-09-21T02:51:16.977390+0000","last_fullsized":"2024-09-21T02:51:16.977390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577020+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:51:16.577020+0000","last_peered":"2024-09-21T02:51:16.577020+0000","last_clean":"2024-09-21T02:51:16.577020+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:51:16.577020+0000","last_undegraded":"2024-09-21T02:51:16.577020+0000","last_fullsized":"2024-09-21T02:51:16.577020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977117+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:51:16.977117+0000","last_peered":"2024-09-21T02:51:16.977117+0000","last_clean":"2024-09-21T02:51:16.977117+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:51:16.977117+0000","last_undegraded":"2024-09-21T02:51:16.977117+0000","last_fullsized":"2024-09-21T02:51:16.977117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:26.597213+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:51:26.597213+0000","last_peered":"2024-09-21T02:51:26.597213+0000","last_clean":"2024-09-21T02:51:26.597213+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:51:26.597213+0000","last_undegraded":"2024-09-21T02:51:26.597213+0000","last_fullsized":"2024-09-21T02:51:26.597213+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63349,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":90,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572844+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:51:16.572844+0000","last_peered":"2024-09-21T02:51:16.572844+0000","last_clean":"2024-09-21T02:51:16.572844+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:51:16.572844+0000","last_undegraded":"2024-09-21T02:51:16.572844+0000","last_fullsized":"2024-09-21T02:51:16.572844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578577+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:51:16.578577+0000","last_peered":"2024-09-21T02:51:16.578577+0000","last_clean":"2024-09-21T02:51:16.578577+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:51:16.578577+0000","last_undegraded":"2024-09-21T02:51:16.578577+0000","last_fullsized":"2024-09-21T02:51:16.578577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576983+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:51:16.576983+0000","last_peered":"2024-09-21T02:51:16.576983+0000","last_clean":"2024-09-21T02:51:16.576983+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:51:16.576983+0000","last_undegraded":"2024-09-21T02:51:16.576983+0000","last_fullsized":"2024-09-21T02:51:16.576983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":53,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:22.562164+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:51:22.562164+0000","last_peered":"2024-09-21T02:51:22.562164+0000","last_clean":"2024-09-21T02:51:22.562164+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:51:22.562164+0000","last_undegraded":"2024-09-21T02:51:22.562164+0000","last_fullsized":"2024-09-21T02:51:22.562164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580840+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:51:16.580840+0000","last_peered":"2024-09-21T02:51:16.580840+0000","last_clean":"2024-09-21T02:51:16.580840+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:51:16.580840+0000","last_undegraded":"2024-09-21T02:51:16.580840+0000","last_fullsized":"2024-09-21T02:51:16.580840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577762+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:51:16.577762+0000","last_peered":"2024-09-21T02:51:16.577762+0000","last_clean":"2024-09-21T02:51:16.577762+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:51:16.577762+0000","last_undegraded":"2024-09-21T02:51:16.577762+0000","last_fullsized":"2024-09-21T02:51:16.577762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573938+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:51:16.573938+0000","last_peered":"2024-09-21T02:51:16.573938+0000","last_clean":"2024-09-21T02:51:16.573938+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:51:16.573938+0000","last_undegraded":"2024-09-21T02:51:16.573938+0000","last_fullsized":"2024-09-21T02:51:16.573938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576883+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:51:16.576883+0000","last_peered":"2024-09-21T02:51:16.576883+0000","last_clean":"2024-09-21T02:51:16.576883+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:51:16.576883+0000","last_undegraded":"2024-09-21T02:51:16.576883+0000","last_fullsized":"2024-09-21T02:51:16.576883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576759+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:51:16.576759+0000","last_peered":"2024-09-21T02:51:16.576759+0000","last_clean":"2024-09-21T02:51:16.576759+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:51:16.576759+0000","last_undegraded":"2024-09-21T02:51:16.576759+0000","last_fullsized":"2024-09-21T02:51:16.576759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:22.561801+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:51:22.561801+0000","last_peered":"2024-09-21T02:51:22.561801+0000","last_clean":"2024-09-21T02:51:22.561801+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:51:22.561801+0000","last_undegraded":"2024-09-21T02:51:22.561801+0000","last_fullsized":"2024-09-21T02:51:22.561801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576701+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:51:16.576701+0000","last_peered":"2024-09-21T02:51:16.576701+0000","last_clean":"2024-09-21T02:51:16.576701+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:51:16.576701+0000","last_undegraded":"2024-09-21T02:51:16.576701+0000","last_fullsized":"2024-09-21T02:51:16.576701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573447+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573447+0000","last_peered":"2024-09-21T02:51:16.573447+0000","last_clean":"2024-09-21T02:51:16.573447+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573447+0000","last_undegraded":"2024-09-21T02:51:16.573447+0000","last_fullsized":"2024-09-21T02:51:16.573447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976061+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:51:16.976061+0000","last_peered":"2024-09-21T02:51:16.976061+0000","last_clean":"2024-09-21T02:51:16.976061+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:51:16.976061+0000","last_undegraded":"2024-09-21T02:51:16.976061+0000","last_fullsized":"2024-09-21T02:51:16.976061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576775+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576775+0000","last_peered":"2024-09-21T02:51:16.576775+0000","last_clean":"2024-09-21T02:51:16.576775+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576775+0000","last_undegraded":"2024-09-21T02:51:16.576775+0000","last_fullsized":"2024-09-21T02:51:16.576775+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573136+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:16.573136+0000","last_peered":"2024-09-21T02:51:16.573136+0000","last_clean":"2024-09-21T02:51:16.573136+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:16.573136+0000","last_undegraded":"2024-09-21T02:51:16.573136+0000","last_fullsized":"2024-09-21T02:51:16.573136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:24.564700+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:51:24.564700+0000","last_peered":"2024-09-21T02:51:24.564700+0000","last_clean":"2024-09-21T02:51:24.564700+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:51:24.564700+0000","last_undegraded":"2024-09-21T02:51:24.564700+0000","last_fullsized":"2024-09-21T02:51:24.564700+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578540+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:51:16.578540+0000","last_peered":"2024-09-21T02:51:16.578540+0000","last_clean":"2024-09-21T02:51:16.578540+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:51:16.578540+0000","last_undegraded":"2024-09-21T02:51:16.578540+0000","last_fullsized":"2024-09-21T02:51:16.578540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:22.561182+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:51:22.561182+0000","last_peered":"2024-09-21T02:51:22.561182+0000","last_clean":"2024-09-21T02:51:22.561182+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:51:22.561182+0000","last_undegraded":"2024-09-21T02:51:22.561182+0000","last_fullsized":"2024-09-21T02:51:22.561182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574384+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:51:16.574384+0000","last_peered":"2024-09-21T02:51:16.574384+0000","last_clean":"2024-09-21T02:51:16.574384+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:51:16.574384+0000","last_undegraded":"2024-09-21T02:51:16.574384+0000","last_fullsized":"2024-09-21T02:51:16.574384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571552+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571552+0000","last_peered":"2024-09-21T02:51:16.571552+0000","last_clean":"2024-09-21T02:51:16.571552+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571552+0000","last_undegraded":"2024-09-21T02:51:16.571552+0000","last_fullsized":"2024-09-21T02:51:16.571552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574339+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:51:16.574339+0000","last_peered":"2024-09-21T02:51:16.574339+0000","last_clean":"2024-09-21T02:51:16.574339+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:51:16.574339+0000","last_undegraded":"2024-09-21T02:51:16.574339+0000","last_fullsized":"2024-09-21T02:51:16.574339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574116+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:16.574116+0000","last_peered":"2024-09-21T02:51:16.574116+0000","last_clean":"2024-09-21T02:51:16.574116+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:16.574116+0000","last_undegraded":"2024-09-21T02:51:16.574116+0000","last_fullsized":"2024-09-21T02:51:16.574116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571740+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:16.571740+0000","last_peered":"2024-09-21T02:51:16.571740+0000","last_clean":"2024-09-21T02:51:16.571740+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:16.571740+0000","last_undegraded":"2024-09-21T02:51:16.571740+0000","last_fullsized":"2024-09-21T02:51:16.571740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976502+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:51:16.976502+0000","last_peered":"2024-09-21T02:51:16.976502+0000","last_clean":"2024-09-21T02:51:16.976502+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:51:16.976502+0000","last_undegraded":"2024-09-21T02:51:16.976502+0000","last_fullsized":"2024-09-21T02:51:16.976502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573953+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.573953+0000","last_peered":"2024-09-21T02:51:16.573953+0000","last_clean":"2024-09-21T02:51:16.573953+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.573953+0000","last_undegraded":"2024-09-21T02:51:16.573953+0000","last_fullsized":"2024-09-21T02:51:16.573953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976881+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976881+0000","last_peered":"2024-09-21T02:51:16.976881+0000","last_clean":"2024-09-21T02:51:16.976881+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976881+0000","last_undegraded":"2024-09-21T02:51:16.976881+0000","last_fullsized":"2024-09-21T02:51:16.976881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571934+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:51:16.571934+0000","last_peered":"2024-09-21T02:51:16.571934+0000","last_clean":"2024-09-21T02:51:16.571934+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:51:16.571934+0000","last_undegraded":"2024-09-21T02:51:16.571934+0000","last_fullsized":"2024-09-21T02:51:16.571934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975953+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.975953+0000","last_peered":"2024-09-21T02:51:16.975953+0000","last_clean":"2024-09-21T02:51:16.975953+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.975953+0000","last_undegraded":"2024-09-21T02:51:16.975953+0000","last_fullsized":"2024-09-21T02:51:16.975953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573705+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:51:16.573705+0000","last_peered":"2024-09-21T02:51:16.573705+0000","last_clean":"2024-09-21T02:51:16.573705+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:51:16.573705+0000","last_undegraded":"2024-09-21T02:51:16.573705+0000","last_fullsized":"2024-09-21T02:51:16.573705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575838+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:16.575838+0000","last_peered":"2024-09-21T02:51:16.575838+0000","last_clean":"2024-09-21T02:51:16.575838+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:16.575838+0000","last_undegraded":"2024-09-21T02:51:16.575838+0000","last_fullsized":"2024-09-21T02:51:16.575838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:51:16.976034+0000","last_peered":"2024-09-21T02:51:16.976034+0000","last_clean":"2024-09-21T02:51:16.976034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:51:16.976034+0000","last_undegraded":"2024-09-21T02:51:16.976034+0000","last_fullsized":"2024-09-21T02:51:16.976034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576863+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:51:16.576863+0000","last_peered":"2024-09-21T02:51:16.576863+0000","last_clean":"2024-09-21T02:51:16.576863+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:51:16.576863+0000","last_undegraded":"2024-09-21T02:51:16.576863+0000","last_fullsized":"2024-09-21T02:51:16.576863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578106+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:51:16.578106+0000","last_peered":"2024-09-21T02:51:16.578106+0000","last_clean":"2024-09-21T02:51:16.578106+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:51:16.578106+0000","last_undegraded":"2024-09-21T02:51:16.578106+0000","last_fullsized":"2024-09-21T02:51:16.578106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977018+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:51:16.977018+0000","last_peered":"2024-09-21T02:51:16.977018+0000","last_clean":"2024-09-21T02:51:16.977018+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:51:16.977018+0000","last_undegraded":"2024-09-21T02:51:16.977018+0000","last_fullsized":"2024-09-21T02:51:16.977018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577070+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:51:16.577070+0000","last_peered":"2024-09-21T02:51:16.577070+0000","last_clean":"2024-09-21T02:51:16.577070+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:51:16.577070+0000","last_undegraded":"2024-09-21T02:51:16.577070+0000","last_fullsized":"2024-09-21T02:51:16.577070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976572+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976572+0000","last_peered":"2024-09-21T02:51:16.976572+0000","last_clean":"2024-09-21T02:51:16.976572+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976572+0000","last_undegraded":"2024-09-21T02:51:16.976572+0000","last_fullsized":"2024-09-21T02:51:16.976572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976425+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:51:16.976425+0000","last_peered":"2024-09-21T02:51:16.976425+0000","last_clean":"2024-09-21T02:51:16.976425+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:51:16.976425+0000","last_undegraded":"2024-09-21T02:51:16.976425+0000","last_fullsized":"2024-09-21T02:51:16.976425+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:24.563360+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:51:24.563360+0000","last_peered":"2024-09-21T02:51:24.563360+0000","last_clean":"2024-09-21T02:51:24.563360+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:51:24.563360+0000","last_undegraded":"2024-09-21T02:51:24.563360+0000","last_fullsized":"2024-09-21T02:51:24.563360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70944,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":279,"num_read_kb":222,"num_write":240,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":212832,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":860160,"data_stored":421192,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075067,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037790,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234018,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":1013906,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593773,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":860160,"data_stored":420727,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013193,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":421210,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":484587,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542214,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":484989,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":422066,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65492,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65090,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65085,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:51:29.634 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T02:51:29.634 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T02:51:29.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T02:51:29.898 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:29.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:29 smithi165 ceph-mon[34818]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:29 smithi178 ceph-mon[34759]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:30.020 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:29 smithi031 ceph-mon[31209]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:30.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 -- 172.21.15.31:0/2037747093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d0102a80 msgr2=0x7fd5d0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:30.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/2037747093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d0102a80 0x7fd5d0102e90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc009900 tx=0x7fd5cc02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 -- 172.21.15.31:0/2037747093 shutdown_connections 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/2037747093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d01084d0 0x7fd5d01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/2037747093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d0108e20 0x7fd5d010f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/2037747093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d0102a80 0x7fd5d0102e90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 -- 172.21.15.31:0/2037747093 >> 172.21.15.31:0/2037747093 conn(0x7fd5d0079720 msgr2=0x7fd5d0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 -- 172.21.15.31:0/2037747093 shutdown_connections 2024-09-21T02:51:30.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 -- 172.21.15.31:0/2037747093 wait complete. 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.408+0000 7fd5d75d4640 1 Processor -- start 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 -- start start 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d0102a80 0x7fd5d019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 0x7fd5d019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d0108e20 0x7fd5d019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0112ae0 con 0x7fd5d01084d0 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0112960 con 0x7fd5d0102a80 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d75d4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0112c60 con 0x7fd5d0108e20 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 0x7fd5d019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 0x7fd5d019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45934/0 (socket says 172.21.15.31:45934) 2024-09-21T02:51:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 -- 172.21.15.31:0/3106547268 learned_addr learned my addr 172.21.15.31:0/3106547268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d5b71640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d0108e20 0x7fd5d019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d5370640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d0102a80 0x7fd5d019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 -- 172.21.15.31:0/3106547268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d0108e20 msgr2=0x7fd5d019beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d0108e20 0x7fd5d019beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 -- 172.21.15.31:0/3106547268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d0102a80 msgr2=0x7fd5d019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d0102a80 0x7fd5d019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.409+0000 7fd5d4b6f640 1 -- 172.21.15.31:0/3106547268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d01a6740 con 0x7fd5d01084d0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.410+0000 7fd5d4b6f640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 0x7fd5d019b970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000d750 tx=0x7fd5c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.410+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0017070 con 0x7fd5d01084d0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.410+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01a6a30 con 0x7fd5d01084d0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.410+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c00054d0 con 0x7fd5d01084d0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.410+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0004c30 con 0x7fd5d01084d0 2024-09-21T02:51:30.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.410+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d01a6f70 con 0x7fd5d01084d0 2024-09-21T02:51:30.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.411+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d0103d00 con 0x7fd5d01084d0 2024-09-21T02:51:30.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.412+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd5c0004dd0 con 0x7fd5d01084d0 2024-09-21T02:51:30.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.413+0000 7fd5c67fc640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd5bc0b65f0 0x7fd5bc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:30.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.413+0000 7fd5d5370640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd5bc0b65f0 0x7fd5bc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:30.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.413+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd5c00e7370 con 0x7fd5d01084d0 2024-09-21T02:51:30.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.414+0000 7fd5d5370640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd5bc0b65f0 0x7fd5bc0b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc000c00 tx=0x7fd5cc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.416+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5c00b1d80 con 0x7fd5d01084d0 2024-09-21T02:51:30.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.680+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd5d0102e90 con 0x7fd5d01084d0 2024-09-21T02:51:30.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.683+0000 7fd5c67fc640 1 -- 172.21.15.31:0/3106547268 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd5d0102e90 con 0x7fd5d01084d0 2024-09-21T02:51:30.685 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-21T02:51:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCyNO5mRzCnKBAAkwOuDZLVr4SkQv6ha3ZSqw== 2024-09-21T02:51:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd5bc0b65f0 msgr2=0x7fd5bc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd5bc0b65f0 0x7fd5bc0b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc000c00 tx=0x7fd5cc031400 comp rx=0 tx=0).stop 2024-09-21T02:51:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 msgr2=0x7fd5d019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 0x7fd5d019b970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000d750 tx=0x7fd5c000dc10 comp rx=0 tx=0).stop 2024-09-21T02:51:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 shutdown_connections 2024-09-21T02:51:30.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd5bc0b65f0 0x7fd5bc0b8b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5d0108e20 0x7fd5d019beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5d01084d0 0x7fd5d019b970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 --2- 172.21.15.31:0/3106547268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5d0102a80 0x7fd5d019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:30.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 >> 172.21.15.31:0/3106547268 conn(0x7fd5d0079720 msgr2=0x7fd5d0106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:30.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 shutdown_connections 2024-09-21T02:51:30.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:30.686+0000 7fd5d75d4640 1 -- 172.21.15.31:0/3106547268 wait complete. 2024-09-21T02:51:30.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:30 smithi165 ceph-mon[34818]: pgmap v283: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s; 84 B/s, 0 objects/s recovering 2024-09-21T02:51:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:30 smithi178 ceph-mon[34759]: pgmap v283: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s; 84 B/s, 0 objects/s recovering 2024-09-21T02:51:30.993 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:30 smithi031 ceph-mon[31209]: pgmap v283: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s; 84 B/s, 0 objects/s recovering 2024-09-21T02:51:31.345 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:51:31.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T02:51:31.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T02:51:31.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T02:51:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:31.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3106547268' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:51:31.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3106547268' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:51:31.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3106547268' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:51:31.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3106547268' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:51:31.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3106547268' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:51:31.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3106547268' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:51:32.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 -- 172.21.15.31:0/340153051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74401083a0 msgr2=0x7f74401087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 --2- 172.21.15.31:0/340153051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74401083a0 0x7f74401087b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7434009900 tx=0x7f743402a410 comp rx=0 tx=0).stop 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 -- 172.21.15.31:0/340153051 shutdown_connections 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 --2- 172.21.15.31:0/340153051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440108cf0 0x7f744010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 --2- 172.21.15.31:0/340153051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74401083a0 0x7f74401087b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 --2- 172.21.15.31:0/340153051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7440102a60 0x7f7440102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 -- 172.21.15.31:0/340153051 >> 172.21.15.31:0/340153051 conn(0x7f7440079730 msgr2=0x7f7440079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 -- 172.21.15.31:0/340153051 shutdown_connections 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.169+0000 7f7445f0a640 1 -- 172.21.15.31:0/340153051 wait complete. 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 Processor -- start 2024-09-21T02:51:32.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 -- start start 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 0x7f744019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74401083a0 0x7f744019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440108cf0 0x7f744019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7440113a80 con 0x7f7440102a60 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7440113900 con 0x7f74401083a0 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f7445f0a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7440113c00 con 0x7f7440108cf0 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.170+0000 7f743f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 0x7f744019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 0x7f744019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45952/0 (socket says 172.21.15.31:45952) 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 -- 172.21.15.31:0/2545343500 learned_addr learned my addr 172.21.15.31:0/2545343500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743effd640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74401083a0 0x7f744019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743ffff640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440108cf0 0x7f744019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 -- 172.21.15.31:0/2545343500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440108cf0 msgr2=0x7f744019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440108cf0 0x7f744019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 -- 172.21.15.31:0/2545343500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74401083a0 msgr2=0x7f744019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74401083a0 0x7f744019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 -- 172.21.15.31:0/2545343500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74401a0440 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743effd640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74401083a0 0x7f744019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743f7fe640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 0x7f744019a4d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f743000d950 tx=0x7f743000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430017030 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7430004020 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74401a0730 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300047a0 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.171+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74401a6e90 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.172+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7440102e70 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.174+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f7430009a30 con 0x7f7440102a60 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.175+0000 7f743cff9640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74280b65f0 0x7f74280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.175+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f74300e7780 con 0x7f7440102a60 2024-09-21T02:51:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.175+0000 7f743effd640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74280b65f0 0x7f74280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.175+0000 7f743effd640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74280b65f0 0x7f74280b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7434000c00 tx=0x7f7434032da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.178+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74300addb0 con 0x7f7440102a60 2024-09-21T02:51:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.372+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f744019b600 con 0x7f7440102a60 2024-09-21T02:51:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.373+0000 7f743cff9640 1 -- 172.21.15.31:0/2545343500 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7f74300b2190 con 0x7f7440102a60 2024-09-21T02:51:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74280b65f0 msgr2=0x7f74280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74280b65f0 0x7f74280b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7434000c00 tx=0x7f7434032da0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 msgr2=0x7f744019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 0x7f744019a4d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f743000d950 tx=0x7f743000de10 comp rx=0 tx=0).stop 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 shutdown_connections 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f74280b65f0 0x7f74280b8b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7440108cf0 0x7f744019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74401083a0 0x7f744019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 --2- 172.21.15.31:0/2545343500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7440102a60 0x7f744019a4d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 >> 172.21.15.31:0/2545343500 conn(0x7f7440079730 msgr2=0x7f7440105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 shutdown_connections 2024-09-21T02:51:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:32.377+0000 7f7445f0a640 1 -- 172.21.15.31:0/2545343500 wait complete. 2024-09-21T02:51:32.826 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:32 smithi031 ceph-mon[31209]: pgmap v284: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:51:32.826 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:32 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2545343500' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:51:32.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:32 smithi165 ceph-mon[34818]: pgmap v284: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:51:32.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:32 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2545343500' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:51:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:32 smithi178 ceph-mon[34759]: pgmap v284: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:51:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:32 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2545343500' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:51:33.119 INFO:teuthology.orchestra.run.smithi031.stdout:[{"version":56,"timestamp":"2024-09-21T02:50:40.204344+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T02:51:33.120 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T02:51:33.120 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T02:51:33.120 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T02:51:33.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T02:51:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:33.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 -- 172.21.15.31:0/837853661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc103b80 msgr2=0x7fb4bc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 --2- 172.21.15.31:0/837853661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc103b80 0x7fb4bc103f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009900 tx=0x7fb4ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 -- 172.21.15.31:0/837853661 shutdown_connections 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 --2- 172.21.15.31:0/837853661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 --2- 172.21.15.31:0/837853661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4bc109e10 0x7fb4bc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 --2- 172.21.15.31:0/837853661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc103b80 0x7fb4bc103f90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 -- 172.21.15.31:0/837853661 >> 172.21.15.31:0/837853661 conn(0x7fb4bc0ff610 msgr2=0x7fb4bc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 -- 172.21.15.31:0/837853661 shutdown_connections 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.904+0000 7fb4c3031640 1 -- 172.21.15.31:0/837853661 wait complete. 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 Processor -- start 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 -- start start 2024-09-21T02:51:33.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4bc103b80 0x7fb4bc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4bc109e10 0x7fb4bc19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc113b50 con 0x7fb4bc1094c0 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc1139d0 con 0x7fb4bc109e10 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.905+0000 7fb4c3031640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc113cd0 con 0x7fb4bc103b80 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45980/0 (socket says 172.21.15.31:45980) 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 -- 172.21.15.31:0/2015544858 learned_addr learned my addr 172.21.15.31:0/2015544858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4c0dcd640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4bc103b80 0x7fb4bc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4c15ce640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4bc109e10 0x7fb4bc19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 -- 172.21.15.31:0/2015544858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4bc103b80 msgr2=0x7fb4bc19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4bc103b80 0x7fb4bc19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 -- 172.21.15.31:0/2015544858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4bc109e10 msgr2=0x7fb4bc19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4bc109e10 0x7fb4bc19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 -- 172.21.15.31:0/2015544858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1a04b0 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4bbfff640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc19aae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d750 tx=0x7fb4a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4017070 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a40054d0 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc1a07a0 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.906+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4004c30 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.907+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc1a6f00 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.907+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc103f90 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.909+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb4a4004dd0 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.910+0000 7fb4b9ffb640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb4a80b65f0 0x7fb4a80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.910+0000 7fb4c0dcd640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb4a80b65f0 0x7fb4a80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.910+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb4a40e70d0 con 0x7fb4bc1094c0 2024-09-21T02:51:33.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.910+0000 7fb4c0dcd640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb4a80b65f0 0x7fb4a80b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac031440 tx=0x7fb4ac005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:33.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:33.914+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4a40ad700 con 0x7fb4bc1094c0 2024-09-21T02:51:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.156+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb4bc02e640 con 0x7fb4bc1094c0 2024-09-21T02:51:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.161+0000 7fb4b9ffb640 1 -- 172.21.15.31:0/2015544858 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v22) ==== 74+0+296448 (secure 0 0 0) 0x7fb4a40b1ae0 con 0x7fb4bc1094c0 2024-09-21T02:51:34.166 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:34.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb4a80b65f0 msgr2=0x7fb4a80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb4a80b65f0 0x7fb4a80b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac031440 tx=0x7fb4ac005d50 comp rx=0 tx=0).stop 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 msgr2=0x7fb4bc19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc19aae0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d750 tx=0x7fb4a400dc10 comp rx=0 tx=0).stop 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 shutdown_connections 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb4a80b65f0 0x7fb4a80b8b00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4bc109e10 0x7fb4bc19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4bc1094c0 0x7fb4bc19aae0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 --2- 172.21.15.31:0/2015544858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4bc103b80 0x7fb4bc19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 >> 172.21.15.31:0/2015544858 conn(0x7fb4bc0ff610 msgr2=0x7fb4bc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 shutdown_connections 2024-09-21T02:51:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:34.166+0000 7fb4c3031640 1 -- 172.21.15.31:0/2015544858 wait complete. 2024-09-21T02:51:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:34 smithi165 ceph-mon[34818]: pgmap v285: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T02:51:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:34 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2015544858' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:51:34.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:34 smithi178 ceph-mon[34759]: pgmap v285: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T02:51:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:34 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2015544858' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:51:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:34 smithi031 ceph-mon[31209]: pgmap v285: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T02:51:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:34 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2015544858' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:51:35.200 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":44765732},{"type":"v1","addr":"172.21.15.31:6801","nonce":44765732}]},"active_addr":"172.21.15.31:6801/44765732","active_change":"2024-09-21T02:44:15.049757+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24151,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24175,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3897449423}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1752739616}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1643689360}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1196903843}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3015087263}]}]} 2024-09-21T02:51:35.206 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T02:51:35.206 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T02:51:35.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:51:35.473 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:35.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 -- 172.21.15.31:0/2243673120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 msgr2=0x7ff0081098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:35.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 --2- 172.21.15.31:0/2243673120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff0081098b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feff0009900 tx=0x7feff002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:35.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 -- 172.21.15.31:0/2243673120 shutdown_connections 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 --2- 172.21.15.31:0/2243673120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff008109df0 0x7ff008110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 --2- 172.21.15.31:0/2243673120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff0081098b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 --2- 172.21.15.31:0/2243673120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff008103b60 0x7ff008103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 -- 172.21.15.31:0/2243673120 >> 172.21.15.31:0/2243673120 conn(0x7ff00807b850 msgr2=0x7ff00807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 -- 172.21.15.31:0/2243673120 shutdown_connections 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 -- 172.21.15.31:0/2243673120 wait complete. 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.968+0000 7ff00dacd640 1 Processor -- start 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 -- start start 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff008103b60 0x7ff00819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff00819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff008109df0 0x7ff00819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff008113ab0 con 0x7ff0081094a0 2024-09-21T02:51:35.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff008113930 con 0x7ff008109df0 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff00dacd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff008113c30 con 0x7ff008103b60 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff00819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff00819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45994/0 (socket says 172.21.15.31:45994) 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 -- 172.21.15.31:0/429584195 learned_addr learned my addr 172.21.15.31:0/429584195 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 -- 172.21.15.31:0/429584195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff008103b60 msgr2=0x7ff00819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7ff0077fe640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff008103b60 0x7ff00819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff008103b60 0x7ff00819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 -- 172.21.15.31:0/429584195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff008109df0 msgr2=0x7ff00819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff008109df0 0x7ff00819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.969+0000 7feffeffd640 1 -- 172.21.15.31:0/429584195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0081a03f0 con 0x7ff0081094a0 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.970+0000 7feffeffd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff00819aa20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feff002a930 tx=0x7feff0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:35.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.970+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff003f030 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.970+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0081a0680 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.970+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff0003aa0 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.970+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0035070 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.970+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a6e40 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.971+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff008104de0 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.973+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7feff0042030 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.973+0000 7ff0057fa640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff40b65f0 0x7feff40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.974+0000 7ff0077fe640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff40b65f0 0x7feff40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.974+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7feff01080a0 con 0x7ff0081094a0 2024-09-21T02:51:35.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.974+0000 7ff0077fe640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff40b65f0 0x7feff40b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feff8005fd0 tx=0x7feff8005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:35.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:35.977+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feff00ce770 con 0x7ff0081094a0 2024-09-21T02:51:36.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.163+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff008103f70 con 0x7ff0081094a0 2024-09-21T02:51:36.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.165+0000 7ff0057fa640 1 -- 172.21.15.31:0/429584195 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+23145 (secure 0 0 0) 0x7ff008103f70 con 0x7ff0081094a0 2024-09-21T02:51:36.167 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:36.167 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:51:16.567886+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:51:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff40b65f0 msgr2=0x7feff40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff40b65f0 0x7feff40b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feff8005fd0 tx=0x7feff8005c70 comp rx=0 tx=0).stop 2024-09-21T02:51:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 msgr2=0x7ff00819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff00819aa20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feff002a930 tx=0x7feff0004270 comp rx=0 tx=0).stop 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 shutdown_connections 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff40b65f0 0x7feff40b8b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff008109df0 0x7ff00819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0081094a0 0x7ff00819aa20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 --2- 172.21.15.31:0/429584195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff008103b60 0x7ff00819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 >> 172.21.15.31:0/429584195 conn(0x7ff00807b850 msgr2=0x7ff008107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 shutdown_connections 2024-09-21T02:51:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:36.168+0000 7ff00dacd640 1 -- 172.21.15.31:0/429584195 wait complete. 2024-09-21T02:51:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:36 smithi031 ceph-mon[31209]: pgmap v286: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:36.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:36 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/429584195' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:36 smithi178 ceph-mon[34759]: pgmap v286: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:36 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/429584195' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:36.980 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T02:51:36.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:51:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:36 smithi165 ceph-mon[34818]: pgmap v286: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:36 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/429584195' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:37.242 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 -- 172.21.15.31:0/1982484148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 msgr2=0x7f7e0010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 --2- 172.21.15.31:0/1982484148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0010d8d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 -- 172.21.15.31:0/1982484148 shutdown_connections 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 --2- 172.21.15.31:0/1982484148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e0010de90 0x7f7e00110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 --2- 172.21.15.31:0/1982484148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0010d8d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 --2- 172.21.15.31:0/1982484148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e00104a30 0x7f7e00104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 -- 172.21.15.31:0/1982484148 >> 172.21.15.31:0/1982484148 conn(0x7f7e00079770 msgr2=0x7f7e00079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 -- 172.21.15.31:0/1982484148 shutdown_connections 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.784+0000 7f7e080a7640 1 -- 172.21.15.31:0/1982484148 wait complete. 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 Processor -- start 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 -- start start 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e00104a30 0x7f7e0019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0010de90 0x7f7e001a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00113af0 con 0x7f7e00105380 2024-09-21T02:51:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00113970 con 0x7f7e00104a30 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.785+0000 7f7e080a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00113c70 con 0x7f7e0010de90 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46010/0 (socket says 172.21.15.31:46010) 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 -- 172.21.15.31:0/2425088810 learned_addr learned my addr 172.21.15.31:0/2425088810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05e43640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e00104a30 0x7f7e0019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e06644640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0010de90 0x7f7e001a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 -- 172.21.15.31:0/2425088810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0010de90 msgr2=0x7f7e001a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0010de90 0x7f7e001a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 -- 172.21.15.31:0/2425088810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e00104a30 msgr2=0x7f7e0019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e00104a30 0x7f7e0019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 -- 172.21.15.31:0/2425088810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e001a2690 con 0x7f7e00105380 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e05642640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0019ccc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7df002a930 tx=0x7f7df0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df003f030 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e001a2920 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df0003aa0 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0035070 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.786+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e001a90e0 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.787+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0006c260 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.789+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f7df0042030 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.790+0000 7f7df6ffd640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7dec0b65f0 0x7f7dec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.790+0000 7f7e05e43640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7dec0b65f0 0x7f7dec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.790+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7df0107fe0 con 0x7f7e00105380 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.790+0000 7f7e05e43640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7dec0b65f0 0x7f7dec0b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005fd0 tx=0x7f7dfc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.793+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7df00d29f0 con 0x7f7e00105380 2024-09-21T02:51:37.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.979+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e00104e40 con 0x7f7e00105380 2024-09-21T02:51:37.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.981+0000 7f7df6ffd640 1 -- 172.21.15.31:0/2425088810 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+23145 (secure 0 0 0) 0x7f7df00d7930 con 0x7f7e00105380 2024-09-21T02:51:37.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:51:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:51:16.567886+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7dec0b65f0 msgr2=0x7f7dec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7dec0b65f0 0x7f7dec0b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005fd0 tx=0x7f7dfc00b390 comp rx=0 tx=0).stop 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 msgr2=0x7f7e0019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0019ccc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7df002a930 tx=0x7f7df0004270 comp rx=0 tx=0).stop 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 shutdown_connections 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7dec0b65f0 0x7f7dec0b8b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0010de90 0x7f7e001a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e00105380 0x7f7e0019ccc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 --2- 172.21.15.31:0/2425088810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e00104a30 0x7f7e0019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.983+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 >> 172.21.15.31:0/2425088810 conn(0x7f7e00079770 msgr2=0x7f7e00100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.984+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 shutdown_connections 2024-09-21T02:51:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:37.984+0000 7f7e080a7640 1 -- 172.21.15.31:0/2425088810 wait complete. 2024-09-21T02:51:38.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T02:51:38.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T02:51:38.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T02:51:38.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T02:51:38.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T02:51:38.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T02:51:38.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T02:51:38.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T02:51:38.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T02:51:38.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T02:51:38.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T02:51:38.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T02:51:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:38 smithi178 ceph-mon[34759]: pgmap v287: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:51:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:38 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2425088810' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:38 smithi031 ceph-mon[31209]: pgmap v287: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:51:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2425088810' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:38 smithi165 ceph-mon[34818]: pgmap v287: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:51:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:38 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2425088810' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:51:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:39.499 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 -- 172.21.15.31:0/2777259374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee01049d0 msgr2=0x7f2ee0104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 --2- 172.21.15.31:0/2777259374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee01049d0 0x7f2ee0104de0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009900 tx=0x7f2edc02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 -- 172.21.15.31:0/2777259374 shutdown_connections 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 --2- 172.21.15.31:0/2777259374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee010de30 0x7f2ee0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 --2- 172.21.15.31:0/2777259374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105320 0x7f2ee010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 --2- 172.21.15.31:0/2777259374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee01049d0 0x7f2ee0104de0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 -- 172.21.15.31:0/2777259374 >> 172.21.15.31:0/2777259374 conn(0x7f2ee0079740 msgr2=0x7f2ee0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 -- 172.21.15.31:0/2777259374 shutdown_connections 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.225+0000 7f2ee8674640 1 -- 172.21.15.31:0/2777259374 wait complete. 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 Processor -- start 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 -- start start 2024-09-21T02:51:40.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee01049d0 0x7f2ee019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee0105320 0x7f2ee019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee010de30 0x7f2ee019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0113ab0 con 0x7f2ee0105320 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0113930 con 0x7f2ee01049d0 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee8674640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0113c30 con 0x7f2ee010de30 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee5c0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee0105320 0x7f2ee019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee5c0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee0105320 0x7f2ee019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42294/0 (socket says 172.21.15.31:42294) 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee5c0f640 1 -- 172.21.15.31:0/3289884352 learned_addr learned my addr 172.21.15.31:0/3289884352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.226+0000 7f2ee6410640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee01049d0 0x7f2ee019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee6c11640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee010de30 0x7f2ee019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee5c0f640 1 -- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee010de30 msgr2=0x7f2ee019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee5c0f640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee010de30 0x7f2ee019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee5c0f640 1 -- 172.21.15.31:0/3289884352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee01049d0 msgr2=0x7f2ee019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee5c0f640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee01049d0 0x7f2ee019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee5c0f640 1 -- 172.21.15.31:0/3289884352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee01a0510 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee6c11640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee010de30 0x7f2ee019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee5c0f640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee0105320 0x7f2ee019ab40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c970 tx=0x7f2ed000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0007bb0 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee01a0800 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed0007d50 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0004e10 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.227+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a6f60 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.228+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ee0063ad0 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.230+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f2ed0004fb0 con 0x7f2ee0105320 2024-09-21T02:51:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.230+0000 7f2ed77fe640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ecc0b65f0 0x7f2ecc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ee6410640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ecc0b65f0 0x7f2ecc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2ed00e7870 con 0x7f2ee0105320 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ed77fe640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] conn(0x7f2ecc0c0460 0x7f2ecc0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 --> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ecc0c3020 con 0x7f2ecc0c0460 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2ed00e7cc0 con 0x7f2ee0105320 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ee6410640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ecc0b65f0 0x7f2ecc0b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.231+0000 7f2ee6c11640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] conn(0x7f2ecc0c0460 0x7f2ecc0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.232+0000 7f2ee6c11640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] conn(0x7f2ecc0c0460 0x7f2ecc0c2950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:40.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.233+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== osd.2 v2:172.21.15.178:6800/730728159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ecc0c3020 con 0x7f2ecc0c0460 2024-09-21T02:51:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.246+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 --> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ee0063ed0 con 0x7f2ecc0c0460 2024-09-21T02:51:40.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ed77fe640 1 -- 172.21.15.31:0/3289884352 <== osd.2 v2:172.21.15.178:6800/730728159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2ee0063ed0 con 0x7f2ecc0c0460 2024-09-21T02:51:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] conn(0x7f2ecc0c0460 msgr2=0x7f2ecc0c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] conn(0x7f2ecc0c0460 0x7f2ecc0c2950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ecc0b65f0 msgr2=0x7f2ecc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ecc0b65f0 0x7f2ecc0b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc031400 comp rx=0 tx=0).stop 2024-09-21T02:51:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee0105320 msgr2=0x7f2ee019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 --2- 172.21.15.31:0/3289884352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ee0105320 0x7f2ee019ab40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c970 tx=0x7f2ed000ce30 comp rx=0 tx=0).stop 2024-09-21T02:51:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee6c11640 1 -- 172.21.15.31:0/3289884352 reap_dead start 2024-09-21T02:51:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 shutdown_connections 2024-09-21T02:51:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.247+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 >> 172.21.15.31:0/3289884352 conn(0x7f2ee0079740 msgr2=0x7f2ee01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.248+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 shutdown_connections 2024-09-21T02:51:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.248+0000 7f2ee8674640 1 -- 172.21.15.31:0/3289884352 wait complete. 2024-09-21T02:51:40.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 -- 172.21.15.31:0/2921502091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c810de10 msgr2=0x7f37c8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/2921502091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c810de10 0x7f37c8110300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009ca0 tx=0x7f37c002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 -- 172.21.15.31:0/2921502091 shutdown_connections 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/2921502091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c810de10 0x7f37c8110300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/2921502091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37c8105300 0x7f37c810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/2921502091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c81049b0 0x7f37c8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 -- 172.21.15.31:0/2921502091 >> 172.21.15.31:0/2921502091 conn(0x7f37c8079730 msgr2=0x7f37c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 -- 172.21.15.31:0/2921502091 shutdown_connections 2024-09-21T02:51:40.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.672+0000 7f37cdfb8640 1 -- 172.21.15.31:0/2921502091 wait complete. 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.673+0000 7f37cdfb8640 1 Processor -- start 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.673+0000 7f37cdfb8640 1 -- start start 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c81049b0 0x7f37c819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37c8105300 0x7f37c819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c810de10 0x7f37c819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8113a70 con 0x7f37c81049b0 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81138f0 con 0x7f37c8105300 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8113bf0 con 0x7f37c810de10 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c81049b0 0x7f37c819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c81049b0 0x7f37c819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42318/0 (socket says 172.21.15.31:42318) 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 -- 172.21.15.31:0/3214143763 learned_addr learned my addr 172.21.15.31:0/3214143763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:40.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c7fff640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c810de10 0x7f37c819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37beffd640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37c8105300 0x7f37c819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 -- 172.21.15.31:0/3214143763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c810de10 msgr2=0x7f37c819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c810de10 0x7f37c819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 -- 172.21.15.31:0/3214143763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37c8105300 msgr2=0x7f37c819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37c8105300 0x7f37c819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 -- 172.21.15.31:0/3214143763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c81a0350 con 0x7f37c81049b0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c7fff640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37c810de10 0x7f37c819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37beffd640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37c8105300 0x7f37c819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c77fe640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c81049b0 0x7f37c819a4a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d950 tx=0x7f37b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017020 con 0x7f37c81049b0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c81a0640 con 0x7f37c81049b0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b8009c30 con 0x7f37c81049b0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b800f600 con 0x7f37c81049b0 2024-09-21T02:51:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.674+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81a6e60 con 0x7f37c81049b0 2024-09-21T02:51:40.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.675+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37c8100f80 con 0x7f37c81049b0 2024-09-21T02:51:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.677+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f37b801a020 con 0x7f37c81049b0 2024-09-21T02:51:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37c57fa640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37b40b6620 0x7f37b40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37beffd640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37b40b6620 0x7f37b40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f37b80e79a0 con 0x7f37c81049b0 2024-09-21T02:51:40.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37c57fa640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] conn(0x7f37b40c0490 0x7f37b40c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 --> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37b40c3050 con 0x7f37b40c0490 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f37b80e7d50 con 0x7f37c81049b0 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.678+0000 7f37c7fff640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] conn(0x7f37b40c0490 0x7f37b40c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.679+0000 7f37beffd640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37b40b6620 0x7f37b40b8b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37c810fa60 tx=0x7f37b0005d20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.679+0000 7f37c7fff640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] conn(0x7f37b40c0490 0x7f37b40c2980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.680+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== osd.0 v2:172.21.15.31:6802/3594547707 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37b40c3050 con 0x7f37b40c0490 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.694+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 --> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37c806b560 con 0x7f37b40c0490 2024-09-21T02:51:40.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37c57fa640 1 -- 172.21.15.31:0/3214143763 <== osd.0 v2:172.21.15.31:6802/3594547707 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f37c806b560 con 0x7f37b40c0490 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] conn(0x7f37b40c0490 msgr2=0x7f37b40c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] conn(0x7f37b40c0490 0x7f37b40c2980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37b40b6620 msgr2=0x7f37b40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37b40b6620 0x7f37b40b8b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37c810fa60 tx=0x7f37b0005d20 comp rx=0 tx=0).stop 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c81049b0 msgr2=0x7f37c819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 --2- 172.21.15.31:0/3214143763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c81049b0 0x7f37c819a4a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d950 tx=0x7f37b800de10 comp rx=0 tx=0).stop 2024-09-21T02:51:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37c7fff640 1 -- 172.21.15.31:0/3214143763 reap_dead start 2024-09-21T02:51:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 shutdown_connections 2024-09-21T02:51:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.695+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 >> 172.21.15.31:0/3214143763 conn(0x7f37c8079730 msgr2=0x7f37c80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.696+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 shutdown_connections 2024-09-21T02:51:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:40.696+0000 7f37cdfb8640 1 -- 172.21.15.31:0/3214143763 wait complete. 2024-09-21T02:51:40.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:40 smithi031 ceph-mon[31209]: pgmap v288: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:40 smithi178 ceph-mon[34759]: pgmap v288: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:40 smithi165 ceph-mon[34818]: pgmap v288: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:41.399 INFO:teuthology.orchestra.run.smithi031.stdout:38654705743 2024-09-21T02:51:41.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T02:51:41.435 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.437 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.439 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.448 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.467 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:41.471 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:42.438 INFO:teuthology.orchestra.run.smithi031.stdout:81604378693 2024-09-21T02:51:42.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T02:51:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:42.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:42 smithi031 ceph-mon[31209]: pgmap v289: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:51:42.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 -- 172.21.15.31:0/2714640934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4341083b0 msgr2=0x7fa4341087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:42.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 --2- 172.21.15.31:0/2714640934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4341083b0 0x7fa4341087c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009900 tx=0x7fa41c02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:42.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 -- 172.21.15.31:0/2714640934 shutdown_connections 2024-09-21T02:51:42.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 --2- 172.21.15.31:0/2714640934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa434108d00 0x7fa43410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 --2- 172.21.15.31:0/2714640934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4341083b0 0x7fa4341087c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 --2- 172.21.15.31:0/2714640934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa434102a70 0x7fa434102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 -- 172.21.15.31:0/2714640934 >> 172.21.15.31:0/2714640934 conn(0x7fa434079730 msgr2=0x7fa434079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 -- 172.21.15.31:0/2714640934 shutdown_connections 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.960+0000 7fa439966640 1 -- 172.21.15.31:0/2714640934 wait complete. 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.961+0000 7fa439966640 1 Processor -- start 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.961+0000 7fa439966640 1 -- start start 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa439966640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa434102a70 0x7fa43419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa439966640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4341083b0 0x7fa43419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa439966640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa434108d00 0x7fa43419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa439966640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa434112970 con 0x7fa434108d00 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa439966640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4341127f0 con 0x7fa4341083b0 2024-09-21T02:51:42.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa439966640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa434112af0 con 0x7fa434102a70 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa434108d00 0x7fa43419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa434108d00 0x7fa43419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42338/0 (socket says 172.21.15.31:42338) 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa432ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa434102a70 0x7fa43419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 -- 172.21.15.31:0/2196538000 learned_addr learned my addr 172.21.15.31:0/2196538000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4327fc640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4341083b0 0x7fa43419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 -- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa434102a70 msgr2=0x7fa43419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa434102a70 0x7fa43419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 -- 172.21.15.31:0/2196538000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4341083b0 msgr2=0x7fa43419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4341083b0 0x7fa43419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 -- 172.21.15.31:0/2196538000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4341a6550 con 0x7fa434108d00 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.962+0000 7fa4337fe640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa434108d00 0x7fa43419bcc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa42400eaa0 tx=0x7fa42400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.963+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42400cdb0 con 0x7fa434108d00 2024-09-21T02:51:42.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.963+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa424005020 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.963+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4341a6840 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.963+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240116d0 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.963+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4341a6d50 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.963+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa434063b10 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.966+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa4240051d0 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.966+0000 7fa438964640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4200b65f0 0x7fa4200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.966+0000 7fa432ffd640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4200b65f0 0x7fa4200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.966+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa4240e7ca0 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.967+0000 7fa438964640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] conn(0x7fa4200c0460 0x7fa4200c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.967+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 --> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa424014ec0 con 0x7fa4200c0460 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.967+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fa4240e80f0 con 0x7fa434108d00 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.967+0000 7fa432ffd640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4200b65f0 0x7fa4200b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa43406c6a0 tx=0x7fa42c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.967+0000 7fa4327fc640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] conn(0x7fa4200c0460 0x7fa4200c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.967+0000 7fa4327fc640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] conn(0x7fa4200c0460 0x7fa4200c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:42.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.968+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== osd.11 v2:172.21.15.178:6824/3803011013 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa424014ec0 con 0x7fa4200c0460 2024-09-21T02:51:42.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:42 smithi178 ceph-mon[34759]: pgmap v289: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:51:42.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.987+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 --> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa434063f10 con 0x7fa4200c0460 2024-09-21T02:51:42.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa438964640 1 -- 172.21.15.31:0/2196538000 <== osd.11 v2:172.21.15.178:6824/3803011013 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa434063f10 con 0x7fa4200c0460 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] conn(0x7fa4200c0460 msgr2=0x7fa4200c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] conn(0x7fa4200c0460 0x7fa4200c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4200b65f0 msgr2=0x7fa4200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4200b65f0 0x7fa4200b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa43406c6a0 tx=0x7fa42c00b390 comp rx=0 tx=0).stop 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa434108d00 msgr2=0x7fa43419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 --2- 172.21.15.31:0/2196538000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa434108d00 0x7fa43419bcc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa42400eaa0 tx=0x7fa42400ef60 comp rx=0 tx=0).stop 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa4337fe640 1 -- 172.21.15.31:0/2196538000 reap_dead start 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 shutdown_connections 2024-09-21T02:51:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 >> 172.21.15.31:0/2196538000 conn(0x7fa434079730 msgr2=0x7fa434104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 shutdown_connections 2024-09-21T02:51:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:42.988+0000 7fa439966640 1 -- 172.21.15.31:0/2196538000 wait complete. 2024-09-21T02:51:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:42 smithi165 ceph-mon[34818]: pgmap v289: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:51:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 -- 172.21.15.31:0/266541119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed6410de20 msgr2=0x7fed64110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 --2- 172.21.15.31:0/266541119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed6410de20 0x7fed64110330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009900 tx=0x7fed5c02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 -- 172.21.15.31:0/266541119 shutdown_connections 2024-09-21T02:51:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 --2- 172.21.15.31:0/266541119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed641052a0 0x7fed6410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 --2- 172.21.15.31:0/266541119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed6410de20 0x7fed64110330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 --2- 172.21.15.31:0/266541119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed64104870 0x7fed64104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 -- 172.21.15.31:0/266541119 >> 172.21.15.31:0/266541119 conn(0x7fed64079730 msgr2=0x7fed64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 -- 172.21.15.31:0/266541119 shutdown_connections 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 -- 172.21.15.31:0/266541119 wait complete. 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.745+0000 7fed69691640 1 Processor -- start 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 -- start start 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed64104870 0x7fed64102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed641052a0 0x7fed64102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed6410de20 0x7fed64103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed64113aa0 con 0x7fed64104870 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed64113920 con 0x7fed641052a0 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed69691640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed64113c20 con 0x7fed6410de20 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed64104870 0x7fed64102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed64104870 0x7fed64102780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42356/0 (socket says 172.21.15.31:42356) 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 -- 172.21.15.31:0/3412370961 learned_addr learned my addr 172.21.15.31:0/3412370961 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed627fc640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed641052a0 0x7fed64102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed637fe640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed6410de20 0x7fed64103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 -- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed6410de20 msgr2=0x7fed64103200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed6410de20 0x7fed64103200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 -- 172.21.15.31:0/3412370961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed641052a0 msgr2=0x7fed64102cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed641052a0 0x7fed64102cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.746+0000 7fed62ffd640 1 -- 172.21.15.31:0/3412370961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed641a8600 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.747+0000 7fed62ffd640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed64104870 0x7fed64102780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed5400cce0 tx=0x7fed5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.747+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5400f650 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.747+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed641a88f0 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.747+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5400fca0 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.747+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54018b40 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.747+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed641a8e30 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.748+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed64102020 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.750+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fed54018ce0 con 0x7fed64104870 2024-09-21T02:51:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.750+0000 7fed4bfff640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed4c0b65f0 0x7fed4c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed627fc640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed4c0b65f0 0x7fed4c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fed540e8080 con 0x7fed64104870 2024-09-21T02:51:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed4bfff640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] conn(0x7fed4c0c0460 0x7fed4c0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 --> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed5400eec0 con 0x7fed4c0c0460 2024-09-21T02:51:43.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fed540e84d0 con 0x7fed64104870 2024-09-21T02:51:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed627fc640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed4c0b65f0 0x7fed4c0b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fed6410fba0 tx=0x7fed5800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.751+0000 7fed637fe640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] conn(0x7fed4c0c0460 0x7fed4c0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.752+0000 7fed637fe640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] conn(0x7fed4c0c0460 0x7fed4c0c2950 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:43.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.752+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== osd.8 v2:172.21.15.178:6816/3837162203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed5400eec0 con 0x7fed4c0c0460 2024-09-21T02:51:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.768+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 --> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed6406b350 con 0x7fed4c0c0460 2024-09-21T02:51:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.768+0000 7fed4bfff640 1 -- 172.21.15.31:0/3412370961 <== osd.8 v2:172.21.15.178:6816/3837162203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed6406b350 con 0x7fed4c0c0460 2024-09-21T02:51:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.768+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] conn(0x7fed4c0c0460 msgr2=0x7fed4c0c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.768+0000 7fed69691640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] conn(0x7fed4c0c0460 0x7fed4c0c2950 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed4c0b65f0 msgr2=0x7fed4c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed4c0b65f0 0x7fed4c0b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fed6410fba0 tx=0x7fed5800b390 comp rx=0 tx=0).stop 2024-09-21T02:51:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed64104870 msgr2=0x7fed64102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 --2- 172.21.15.31:0/3412370961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed64104870 0x7fed64102780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed5400cce0 tx=0x7fed5400a620 comp rx=0 tx=0).stop 2024-09-21T02:51:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed637fe640 1 -- 172.21.15.31:0/3412370961 reap_dead start 2024-09-21T02:51:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 shutdown_connections 2024-09-21T02:51:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 >> 172.21.15.31:0/3412370961 conn(0x7fed64079730 msgr2=0x7fed641005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 shutdown_connections 2024-09-21T02:51:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.769+0000 7fed69691640 1 -- 172.21.15.31:0/3412370961 wait complete. 2024-09-21T02:51:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.609+0000 7f0ea67bd640 1 -- 172.21.15.31:0/860551328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea01062d0 msgr2=0x7f0ea01066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.609+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/860551328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea01062d0 0x7f0ea01066e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009900 tx=0x7f0e9402a410 comp rx=0 tx=0).stop 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 -- 172.21.15.31:0/860551328 shutdown_connections 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/860551328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ea0106c20 0x7f0ea01106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/860551328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea01062d0 0x7f0ea01066e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/860551328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ea0102080 0x7f0ea0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 -- 172.21.15.31:0/860551328 >> 172.21.15.31:0/860551328 conn(0x7f0ea0079f90 msgr2=0x7f0ea007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 -- 172.21.15.31:0/860551328 shutdown_connections 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 -- 172.21.15.31:0/860551328 wait complete. 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 Processor -- start 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.610+0000 7f0ea67bd640 1 -- start start 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea67bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea0102080 0x7f0ea019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:44.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea67bd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ea01062d0 0x7f0ea019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea67bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ea0106c20 0x7f0ea019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea67bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea0113af0 con 0x7f0ea0102080 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea67bd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea0113970 con 0x7f0ea0106c20 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea67bd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea0113c70 con 0x7f0ea01062d0 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea0102080 0x7f0ea019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea0102080 0x7f0ea019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42376/0 (socket says 172.21.15.31:42376) 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 -- 172.21.15.31:0/315991869 learned_addr learned my addr 172.21.15.31:0/315991869 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9f7fe640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ea01062d0 0x7f0ea019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea4d5a640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ea0106c20 0x7f0ea019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 -- 172.21.15.31:0/315991869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ea01062d0 msgr2=0x7f0ea019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ea01062d0 0x7f0ea019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 -- 172.21.15.31:0/315991869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ea0106c20 msgr2=0x7f0ea019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ea0106c20 0x7f0ea019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 -- 172.21.15.31:0/315991869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea01a03f0 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0ea4d5a640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ea0106c20 0x7f0ea019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9ffff640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea0102080 0x7f0ea019a480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000e9c0 tx=0x7f0e9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.611+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9000cd80 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.612+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e900035f0 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.612+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea01a06e0 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.612+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90010690 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.612+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea01a6e40 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.612+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ea0063ad0 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.614+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0e900037a0 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.615+0000 7f0e9d7fa640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e880b65f0 0x7f0e880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.615+0000 7f0e9f7fe640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e880b65f0 0x7f0e880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.615+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0e900e7ce0 con 0x7f0ea0102080 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.615+0000 7f0e9d7fa640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] conn(0x7f0e880c0460 0x7f0e880c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.615+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 --> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e9000be50 con 0x7f0e880c0460 2024-09-21T02:51:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.615+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f0e900e8130 con 0x7f0ea0102080 2024-09-21T02:51:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.616+0000 7f0ea4d5a640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] conn(0x7f0e880c0460 0x7f0e880c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.616+0000 7f0e9f7fe640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e880b65f0 0x7f0e880b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009230 tx=0x7f0e94032bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.616+0000 7f0ea4d5a640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] conn(0x7f0e880c0460 0x7f0e880c2950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.617+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== osd.6 v2:172.21.15.31:6818/787463273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e9000be50 con 0x7f0e880c0460 2024-09-21T02:51:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.631+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 --> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ea0063f50 con 0x7f0e880c0460 2024-09-21T02:51:44.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.631+0000 7f0e9d7fa640 1 -- 172.21.15.31:0/315991869 <== osd.6 v2:172.21.15.31:6818/787463273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0ea0063f50 con 0x7f0e880c0460 2024-09-21T02:51:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] conn(0x7f0e880c0460 msgr2=0x7f0e880c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] conn(0x7f0e880c0460 0x7f0e880c2950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e880b65f0 msgr2=0x7f0e880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e880b65f0 0x7f0e880b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009230 tx=0x7f0e94032bd0 comp rx=0 tx=0).stop 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea0102080 msgr2=0x7f0ea019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 --2- 172.21.15.31:0/315991869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea0102080 0x7f0ea019a480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000e9c0 tx=0x7f0e9000ee80 comp rx=0 tx=0).stop 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea4d5a640 1 -- 172.21.15.31:0/315991869 reap_dead start 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 shutdown_connections 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 >> 172.21.15.31:0/315991869 conn(0x7f0ea0079f90 msgr2=0x7f0ea0107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 shutdown_connections 2024-09-21T02:51:44.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:44.632+0000 7f0ea67bd640 1 -- 172.21.15.31:0/315991869 wait complete. 2024-09-21T02:51:44.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:44 smithi031 ceph-mon[31209]: pgmap v290: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:51:44.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:44.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:44 smithi165 ceph-mon[34818]: pgmap v290: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:51:44.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:44.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:44 smithi178 ceph-mon[34759]: pgmap v290: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:51:44.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:45.563 INFO:teuthology.orchestra.run.smithi031.stdout:300647710743 2024-09-21T02:51:45.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T02:51:45.657 INFO:teuthology.orchestra.run.smithi031.stdout:184683593777 2024-09-21T02:51:45.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T02:51:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 -- 172.21.15.31:0/3409367916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac1094c0 msgr2=0x7f1aac1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 --2- 172.21.15.31:0/3409367916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac1094c0 0x7f1aac1098d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009900 tx=0x7f1aa002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 -- 172.21.15.31:0/3409367916 shutdown_connections 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 --2- 172.21.15.31:0/3409367916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aac109e10 0x7f1aac110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 --2- 172.21.15.31:0/3409367916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac1094c0 0x7f1aac1098d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 --2- 172.21.15.31:0/3409367916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1aac103b80 0x7f1aac103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 -- 172.21.15.31:0/3409367916 >> 172.21.15.31:0/3409367916 conn(0x7f1aac0ff5d0 msgr2=0x7f1aac101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 -- 172.21.15.31:0/3409367916 shutdown_connections 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.633+0000 7f1ab18da640 1 -- 172.21.15.31:0/3409367916 wait complete. 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.634+0000 7f1ab18da640 1 Processor -- start 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.634+0000 7f1ab18da640 1 -- start start 2024-09-21T02:51:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.634+0000 7f1ab18da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac103b80 0x7f1aac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.634+0000 7f1ab18da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aac1094c0 0x7f1aac075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.634+0000 7f1ab18da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1aac109e10 0x7f1aac076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.634+0000 7f1ab18da640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac113ad0 con 0x7f1aac103b80 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab18da640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac113950 con 0x7f1aac109e10 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab18da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac113c50 con 0x7f1aac1094c0 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac103b80 0x7f1aac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac103b80 0x7f1aac075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42348/0 (socket says 172.21.15.31:42348) 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 -- 172.21.15.31:0/2923480579 learned_addr learned my addr 172.21.15.31:0/2923480579 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aab7fe640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1aac109e10 0x7f1aac076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaa7fc640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aac1094c0 0x7f1aac075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 -- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aac1094c0 msgr2=0x7f1aac075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aac1094c0 0x7f1aac075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 -- 172.21.15.31:0/2923480579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1aac109e10 msgr2=0x7f1aac076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1aac109e10 0x7f1aac076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 -- 172.21.15.31:0/2923480579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac1a8760 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aab7fe640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1aac109e10 0x7f1aac076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1aaaffd640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac103b80 0x7f1aac075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d950 tx=0x7f1a9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c017030 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c004020 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c0047a0 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac1a8a50 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.635+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac1a8f10 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.636+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1aac063b50 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.638+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1a9c009a30 con 0x7f1aac103b80 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1ab08d8640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a940b6670 0x7f1a940b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1aaa7fc640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a940b6670 0x7f1a940b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1a9c0e7790 con 0x7f1aac103b80 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1ab08d8640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] conn(0x7f1a940c04e0 0x7f1a940c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 --> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a940c30a0 con 0x7f1a940c04e0 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c0e7be0 con 0x7f1aac103b80 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1aab7fe640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] conn(0x7f1a940c04e0 0x7f1a940c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.639+0000 7f1aaa7fc640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a940b6670 0x7f1a940b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009230 tx=0x7f1aa0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.640+0000 7f1aab7fe640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] conn(0x7f1a940c04e0 0x7f1a940c29d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.641+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== osd.5 v2:172.21.15.178:6808/1160605172 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a940c30a0 con 0x7f1a940c04e0 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.655+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 --> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1aac064020 con 0x7f1a940c04e0 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.655+0000 7f1ab08d8640 1 -- 172.21.15.31:0/2923480579 <== osd.5 v2:172.21.15.178:6808/1160605172 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1aac064020 con 0x7f1a940c04e0 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] conn(0x7f1a940c04e0 msgr2=0x7f1a940c29d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] conn(0x7f1a940c04e0 0x7f1a940c29d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a940b6670 msgr2=0x7f1a940b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a940b6670 0x7f1a940b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009230 tx=0x7f1aa0031400 comp rx=0 tx=0).stop 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac103b80 msgr2=0x7f1aac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 --2- 172.21.15.31:0/2923480579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1aac103b80 0x7f1aac075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d950 tx=0x7f1a9c00de10 comp rx=0 tx=0).stop 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1aab7fe640 1 -- 172.21.15.31:0/2923480579 reap_dead start 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 shutdown_connections 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 >> 172.21.15.31:0/2923480579 conn(0x7f1aac0ff5d0 msgr2=0x7f1aac1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 shutdown_connections 2024-09-21T02:51:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:43.656+0000 7f1ab18da640 1 -- 172.21.15.31:0/2923480579 wait complete. 2024-09-21T02:51:46.465 INFO:teuthology.orchestra.run.smithi031.stdout:158913790005 2024-09-21T02:51:46.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T02:51:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:46 smithi165 ceph-mon[34818]: pgmap v291: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T02:51:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:46 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.698 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:46 smithi031 ceph-mon[31209]: pgmap v291: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T02:51:46.698 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.698 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.698 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.698 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:46 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:46 smithi178 ceph-mon[34759]: pgmap v291: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-21T02:51:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:46 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:51:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.212+0000 7f039fa72640 1 -- 172.21.15.31:0/1194714640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03981094f0 msgr2=0x7f0398109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:47.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.212+0000 7f039fa72640 1 --2- 172.21.15.31:0/1194714640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03981094f0 0x7f0398109900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0388009900 tx=0x7f038802a3f0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 -- 172.21.15.31:0/1194714640 shutdown_connections 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 --2- 172.21.15.31:0/1194714640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0398109e40 0x7f0398110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 --2- 172.21.15.31:0/1194714640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03981094f0 0x7f0398109900 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 --2- 172.21.15.31:0/1194714640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0398103bb0 0x7f0398103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 -- 172.21.15.31:0/1194714640 >> 172.21.15.31:0/1194714640 conn(0x7f03980ff600 msgr2=0x7f0398101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 -- 172.21.15.31:0/1194714640 shutdown_connections 2024-09-21T02:51:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 -- 172.21.15.31:0/1194714640 wait complete. 2024-09-21T02:51:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 Processor -- start 2024-09-21T02:51:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.213+0000 7f039fa72640 1 -- start start 2024-09-21T02:51:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039fa72640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0398103bb0 0x7f0398075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039fa72640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03981094f0 0x7f0398075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039fa72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398109e40 0x7f0398076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039fa72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0398113b00 con 0x7f0398109e40 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039fa72640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0398113980 con 0x7f0398103bb0 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039fa72640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0398113c80 con 0x7f03981094f0 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398109e40 0x7f0398076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398109e40 0x7f0398076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42408/0 (socket says 172.21.15.31:42408) 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 -- 172.21.15.31:0/4149518045 learned_addr learned my addr 172.21.15.31:0/4149518045 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039d80e640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0398103bb0 0x7f0398075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039d00d640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03981094f0 0x7f0398075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:47.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 -- 172.21.15.31:0/4149518045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03981094f0 msgr2=0x7f0398075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03981094f0 0x7f0398075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 -- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0398103bb0 msgr2=0x7f0398075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0398103bb0 0x7f0398075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039e00f640 1 -- 172.21.15.31:0/4149518045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03981a86e0 con 0x7f0398109e40 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.214+0000 7f039d80e640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0398103bb0 0x7f0398075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.215+0000 7f039e00f640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398109e40 0x7f0398076190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f039000d750 tx=0x7f039000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.215+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390017070 con 0x7f0398109e40 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.215+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03981a89d0 con 0x7f0398109e40 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.215+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03900054d0 con 0x7f0398109e40 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.215+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390004c60 con 0x7f0398109e40 2024-09-21T02:51:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.215+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a8e90 con 0x7f0398109e40 2024-09-21T02:51:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.216+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0398063b50 con 0x7f0398109e40 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.217+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0390010460 con 0x7f0398109e40 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.218+0000 7f038effd640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f03840b65f0 0x7f03840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.218+0000 7f039d80e640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f03840b65f0 0x7f03840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.218+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f03900e7640 con 0x7f0398109e40 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.219+0000 7f038effd640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] conn(0x7f03840c0460 0x7f03840c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.219+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 --> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03840c3020 con 0x7f03840c0460 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.219+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f03900e7a90 con 0x7f0398109e40 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.219+0000 7f039d80e640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f03840b65f0 0x7f03840b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0398077160 tx=0x7f0394007480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:47.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.219+0000 7f039d00d640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] conn(0x7f03840c0460 0x7f03840c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.219+0000 7f039d00d640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] conn(0x7f03840c0460 0x7f03840c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:47.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.220+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== osd.7 v2:172.21.15.165:6816/2423292259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03840c3020 con 0x7f03840c0460 2024-09-21T02:51:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.234+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 --> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0398063f80 con 0x7f03840c0460 2024-09-21T02:51:47.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.234+0000 7f038effd640 1 -- 172.21.15.31:0/4149518045 <== osd.7 v2:172.21.15.165:6816/2423292259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0398063f80 con 0x7f03840c0460 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] conn(0x7f03840c0460 msgr2=0x7f03840c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] conn(0x7f03840c0460 0x7f03840c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f03840b65f0 msgr2=0x7f03840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f03840b65f0 0x7f03840b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0398077160 tx=0x7f0394007480 comp rx=0 tx=0).stop 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398109e40 msgr2=0x7f0398076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 --2- 172.21.15.31:0/4149518045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398109e40 0x7f0398076190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f039000d750 tx=0x7f039000dc10 comp rx=0 tx=0).stop 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039e00f640 1 -- 172.21.15.31:0/4149518045 reap_dead start 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 shutdown_connections 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 >> 172.21.15.31:0/4149518045 conn(0x7f03980ff600 msgr2=0x7f0398101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 shutdown_connections 2024-09-21T02:51:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:47.235+0000 7f039fa72640 1 -- 172.21.15.31:0/4149518045 wait complete. 2024-09-21T02:51:48.099 INFO:teuthology.orchestra.run.smithi031.stdout:231928234022 2024-09-21T02:51:48.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T02:51:48.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 -- 172.21.15.31:0/2885964944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241051c0 msgr2=0x7f5b241055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 --2- 172.21.15.31:0/2885964944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241051c0 0x7f5b241055d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009900 tx=0x7f5b1802a410 comp rx=0 tx=0).stop 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 -- 172.21.15.31:0/2885964944 shutdown_connections 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 --2- 172.21.15.31:0/2885964944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b24105b10 0x7f5b2410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 --2- 172.21.15.31:0/2885964944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241051c0 0x7f5b241055d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 --2- 172.21.15.31:0/2885964944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b241031f0 0x7f5b24103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 -- 172.21.15.31:0/2885964944 >> 172.21.15.31:0/2885964944 conn(0x7f5b24077cc0 msgr2=0x7f5b24078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 -- 172.21.15.31:0/2885964944 shutdown_connections 2024-09-21T02:51:48.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.464+0000 7f5b2a121640 1 -- 172.21.15.31:0/2885964944 wait complete. 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 Processor -- start 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 -- start start 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241031f0 0x7f5b2419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b241051c0 0x7f5b2419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b24105b10 0x7f5b2419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113af0 con 0x7f5b241031f0 2024-09-21T02:51:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113970 con 0x7f5b241051c0 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b2a121640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113c70 con 0x7f5b24105b10 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241031f0 0x7f5b2419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.465+0000 7f5b237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241031f0 0x7f5b2419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42428/0 (socket says 172.21.15.31:42428) 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 -- 172.21.15.31:0/3958843980 learned_addr learned my addr 172.21.15.31:0/3958843980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b23fff640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b24105b10 0x7f5b2419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b22ffd640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b241051c0 0x7f5b2419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 -- 172.21.15.31:0/3958843980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b24105b10 msgr2=0x7f5b2419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b24105b10 0x7f5b2419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 -- 172.21.15.31:0/3958843980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b241051c0 msgr2=0x7f5b2419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b241051c0 0x7f5b2419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 -- 172.21.15.31:0/3958843980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b241a0510 con 0x7f5b241031f0 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b23fff640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b24105b10 0x7f5b2419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b237fe640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241031f0 0x7f5b2419a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c890 tx=0x7f5b1400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:48.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004220 con 0x7f5b241031f0 2024-09-21T02:51:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b140043c0 con 0x7f5b241031f0 2024-09-21T02:51:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004a00 con 0x7f5b241031f0 2024-09-21T02:51:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.466+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b241a0800 con 0x7f5b241031f0 2024-09-21T02:51:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.467+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2410fc20 con 0x7f5b241031f0 2024-09-21T02:51:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.467+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b24063a90 con 0x7f5b241031f0 2024-09-21T02:51:48.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.469+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5b140079b0 con 0x7f5b241031f0 2024-09-21T02:51:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b20ff9640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b0c0b6670 0x7f5b0c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b22ffd640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b0c0b6670 0x7f5b0c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5b140e77f0 con 0x7f5b241031f0 2024-09-21T02:51:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b20ff9640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] conn(0x7f5b0c0c04e0 0x7f5b0c0c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 --> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b0c0c30a0 con 0x7f5b0c0c04e0 2024-09-21T02:51:48.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f5b140e7ba0 con 0x7f5b241031f0 2024-09-21T02:51:48.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.470+0000 7f5b23fff640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] conn(0x7f5b0c0c04e0 0x7f5b0c0c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.471+0000 7f5b22ffd640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b0c0b6670 0x7f5b0c0b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:48.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.471+0000 7f5b23fff640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] conn(0x7f5b0c0c04e0 0x7f5b0c0c29d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:48.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.471+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== osd.3 v2:172.21.15.31:6810/3748731073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b0c0c30a0 con 0x7f5b0c0c04e0 2024-09-21T02:51:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.485+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 --> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b24063ee0 con 0x7f5b0c0c04e0 2024-09-21T02:51:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.485+0000 7f5b20ff9640 1 -- 172.21.15.31:0/3958843980 <== osd.3 v2:172.21.15.31:6810/3748731073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5b24063ee0 con 0x7f5b0c0c04e0 2024-09-21T02:51:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] conn(0x7f5b0c0c04e0 msgr2=0x7f5b0c0c29d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] conn(0x7f5b0c0c04e0 0x7f5b0c0c29d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b0c0b6670 msgr2=0x7f5b0c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b0c0b6670 0x7f5b0c0b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18031400 comp rx=0 tx=0).stop 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241031f0 msgr2=0x7f5b2419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 --2- 172.21.15.31:0/3958843980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b241031f0 0x7f5b2419a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c890 tx=0x7f5b1400cd50 comp rx=0 tx=0).stop 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b23fff640 1 -- 172.21.15.31:0/3958843980 reap_dead start 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 shutdown_connections 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 >> 172.21.15.31:0/3958843980 conn(0x7f5b24077cc0 msgr2=0x7f5b24106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 shutdown_connections 2024-09-21T02:51:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.486+0000 7f5b2a121640 1 -- 172.21.15.31:0/3958843980 wait complete. 2024-09-21T02:51:48.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:48 smithi031 ceph-mon[31209]: pgmap v292: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.655+0000 7f2556546640 1 -- 172.21.15.31:0/2949798849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 msgr2=0x7f2550112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.655+0000 7f2556546640 1 --2- 172.21.15.31:0/2949798849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f2550112560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2548009ca0 tx=0x7f254802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 -- 172.21.15.31:0/2949798849 shutdown_connections 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 --2- 172.21.15.31:0/2949798849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2550076080 0x7f255010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 --2- 172.21.15.31:0/2949798849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f2550112560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 --2- 172.21.15.31:0/2949798849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550075730 0x7f2550075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 -- 172.21.15.31:0/2949798849 >> 172.21.15.31:0/2949798849 conn(0x7f255007b850 msgr2=0x7f255007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 -- 172.21.15.31:0/2949798849 shutdown_connections 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.656+0000 7f2556546640 1 -- 172.21.15.31:0/2949798849 wait complete. 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 Processor -- start 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 -- start start 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550075730 0x7f255019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2550076080 0x7f255019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f25501a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2550115c80 con 0x7f2550110140 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2550115b00 con 0x7f2550075730 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.657+0000 7f2556546640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2550115e00 con 0x7f2550076080 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f25501a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f25501a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42448/0 (socket says 172.21.15.31:42448) 2024-09-21T02:51:48.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 -- 172.21.15.31:0/2953011931 learned_addr learned my addr 172.21.15.31:0/2953011931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f254f7fe640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2550076080 0x7f255019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f254ffff640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550075730 0x7f255019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 -- 172.21.15.31:0/2953011931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2550076080 msgr2=0x7f255019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2550076080 0x7f255019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 -- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550075730 msgr2=0x7f255019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550075730 0x7f255019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 -- 172.21.15.31:0/2953011931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25501a2460 con 0x7f2550110140 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2554ae3640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f25501a1df0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2548005cc0 tx=0x7f254802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548003cf0 con 0x7f2550110140 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25480044f0 con 0x7f2550110140 2024-09-21T02:51:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501a26f0 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.658+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254802fc30 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.659+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25501a2c30 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.659+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2550109610 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.661+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f25480046a0 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254d7fa640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25380b6620 0x7f25380b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254ffff640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25380b6620 0x7f25380b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2548108d20 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254d7fa640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] conn(0x7f25380c0490 0x7f25380c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 --> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25380c3050 con 0x7f25380c0490 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2548109170 con 0x7f2550110140 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.662+0000 7f254ffff640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25380b6620 0x7f25380b8b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2540005e10 tx=0x7f2540005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.663+0000 7f254f7fe640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] conn(0x7f25380c0490 0x7f25380c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.663+0000 7f254f7fe640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] conn(0x7f25380c0490 0x7f25380c2980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.664+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== osd.1 v2:172.21.15.165:6800/3056017088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25380c3050 con 0x7f25380c0490 2024-09-21T02:51:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.678+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 --> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2550076c40 con 0x7f25380c0490 2024-09-21T02:51:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f254d7fa640 1 -- 172.21.15.31:0/2953011931 <== osd.1 v2:172.21.15.165:6800/3056017088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2550076c40 con 0x7f25380c0490 2024-09-21T02:51:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] conn(0x7f25380c0490 msgr2=0x7f25380c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] conn(0x7f25380c0490 0x7f25380c2980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25380b6620 msgr2=0x7f25380b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25380b6620 0x7f25380b8b30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2540005e10 tx=0x7f2540005da0 comp rx=0 tx=0).stop 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 msgr2=0x7f25501a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 --2- 172.21.15.31:0/2953011931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2550110140 0x7f25501a1df0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2548005cc0 tx=0x7f254802f470 comp rx=0 tx=0).stop 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2554ae3640 1 -- 172.21.15.31:0/2953011931 reap_dead start 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 shutdown_connections 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 >> 172.21.15.31:0/2953011931 conn(0x7f255007b850 msgr2=0x7f2550107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 shutdown_connections 2024-09-21T02:51:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:48.679+0000 7f2556546640 1 -- 172.21.15.31:0/2953011931 wait complete. 2024-09-21T02:51:48.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:48 smithi165 ceph-mon[34818]: pgmap v292: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:51:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:48 smithi178 ceph-mon[34759]: pgmap v292: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:51:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.092+0000 7f28aab1f640 1 -- 172.21.15.31:0/3922958336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107300 msgr2=0x7f28a4107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.092+0000 7f28aab1f640 1 --2- 172.21.15.31:0/3922958336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107300 0x7f28a4107710 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2890009900 tx=0x7f289002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 -- 172.21.15.31:0/3922958336 shutdown_connections 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 --2- 172.21.15.31:0/3922958336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28a4107c50 0x7f28a4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 --2- 172.21.15.31:0/3922958336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107300 0x7f28a4107710 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 --2- 172.21.15.31:0/3922958336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a4102100 0x7f28a4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 -- 172.21.15.31:0/3922958336 >> 172.21.15.31:0/3922958336 conn(0x7f28a4079e40 msgr2=0x7f28a407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 -- 172.21.15.31:0/3922958336 shutdown_connections 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 -- 172.21.15.31:0/3922958336 wait complete. 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 Processor -- start 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.093+0000 7f28aab1f640 1 -- start start 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28aab1f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a4102100 0x7f28a419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28aab1f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28a4107300 0x7f28a419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28aab1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107c50 0x7f28a419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28aab1f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4113ac0 con 0x7f28a4107c50 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28aab1f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4113940 con 0x7f28a4107300 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28aab1f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4113c40 con 0x7f28a4102100 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107c50 0x7f28a419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107c50 0x7f28a419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41396/0 (socket says 172.21.15.31:41396) 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 -- 172.21.15.31:0/2777385064 learned_addr learned my addr 172.21.15.31:0/2777385064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a88bb640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a4102100 0x7f28a419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a3fff640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28a4107300 0x7f28a419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 -- 172.21.15.31:0/2777385064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a4102100 msgr2=0x7f28a419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a4102100 0x7f28a419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 -- 172.21.15.31:0/2777385064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28a4107300 msgr2=0x7f28a419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28a4107300 0x7f28a419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 -- 172.21.15.31:0/2777385064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a41a0440 con 0x7f28a4107c50 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a3fff640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28a4107300 0x7f28a419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a90bc640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107c50 0x7f28a419fd10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f289c009fd0 tx=0x7f289c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.094+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c017070 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.095+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a41a0730 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.095+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c003e20 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.095+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c010460 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.095+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a41a6e90 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.096+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28a4063b50 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.097+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f289c003970 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.098+0000 7f28a1ffb640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f28940b65f0 0x7f28940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.098+0000 7f28a88bb640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f28940b65f0 0x7f28940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.098+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f289c0e7850 con 0x7f28a4107c50 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.098+0000 7f28a1ffb640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] conn(0x7f28940c0460 0x7f28940c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.098+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 --> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28940c3020 con 0x7f28940c0460 2024-09-21T02:51:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.098+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f289c0e7ca0 con 0x7f28a4107c50 2024-09-21T02:51:49.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.099+0000 7f28a3fff640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] conn(0x7f28940c0460 0x7f28940c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.099+0000 7f28a88bb640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f28940b65f0 0x7f28940b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2898005fd0 tx=0x7f2898005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:49.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.099+0000 7f28a3fff640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] conn(0x7f28940c0460 0x7f28940c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:49.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.100+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== osd.9 v2:172.21.15.31:6826/1204611917 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28940c3020 con 0x7f28940c0460 2024-09-21T02:51:49.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.113+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 --> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28a4063fd0 con 0x7f28940c0460 2024-09-21T02:51:49.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28a1ffb640 1 -- 172.21.15.31:0/2777385064 <== osd.9 v2:172.21.15.31:6826/1204611917 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f28a4063fd0 con 0x7f28940c0460 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] conn(0x7f28940c0460 msgr2=0x7f28940c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] conn(0x7f28940c0460 0x7f28940c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f28940b65f0 msgr2=0x7f28940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f28940b65f0 0x7f28940b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2898005fd0 tx=0x7f2898005850 comp rx=0 tx=0).stop 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107c50 msgr2=0x7f28a419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 --2- 172.21.15.31:0/2777385064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4107c50 0x7f28a419fd10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f289c009fd0 tx=0x7f289c00b260 comp rx=0 tx=0).stop 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28a90bc640 1 -- 172.21.15.31:0/2777385064 reap_dead start 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 shutdown_connections 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 >> 172.21.15.31:0/2777385064 conn(0x7f28a4079e40 msgr2=0x7f28a41061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.114+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 shutdown_connections 2024-09-21T02:51:49.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.115+0000 7f28aab1f640 1 -- 172.21.15.31:0/2777385064 wait complete. 2024-09-21T02:51:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 -- 172.21.15.31:0/3610738800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 msgr2=0x7f8718110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 --2- 172.21.15.31:0/3610738800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f8718110620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8710009ca0 tx=0x7f871002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 -- 172.21.15.31:0/3610738800 shutdown_connections 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 --2- 172.21.15.31:0/3610738800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f8718110620 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 --2- 172.21.15.31:0/3610738800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87181094a0 0x7f87181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 --2- 172.21.15.31:0/3610738800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8718103b60 0x7f8718103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 -- 172.21.15.31:0/3610738800 >> 172.21.15.31:0/3610738800 conn(0x7f871807b850 msgr2=0x7f871807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 -- 172.21.15.31:0/3610738800 shutdown_connections 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.309+0000 7f871e652640 1 -- 172.21.15.31:0/3610738800 wait complete. 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.310+0000 7f871e652640 1 Processor -- start 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.310+0000 7f871e652640 1 -- start start 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871e652640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8718103b60 0x7f871819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871e652640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87181094a0 0x7f871819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871e652640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f871819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871e652640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8718113ab0 con 0x7f8718109df0 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871e652640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8718113930 con 0x7f8718103b60 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871e652640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8718113c30 con 0x7f87181094a0 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f871819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f871819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41422/0 (socket says 172.21.15.31:41422) 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 -- 172.21.15.31:0/1627004767 learned_addr learned my addr 172.21.15.31:0/1627004767 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f8717fff640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8718103b60 0x7f871819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f870ffff640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87181094a0 0x7f871819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 -- 172.21.15.31:0/1627004767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87181094a0 msgr2=0x7f871819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87181094a0 0x7f871819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 -- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8718103b60 msgr2=0x7f871819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8718103b60 0x7f871819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 -- 172.21.15.31:0/1627004767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87181a0350 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.311+0000 7f871cbef640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f871819fce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8710005cc0 tx=0x7f871002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.312+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710003bf0 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.312+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181a05e0 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.312+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8710003d90 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.312+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871002fb20 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.312+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181a0b20 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.313+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8718063b50 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.314+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8710004580 con 0x7f8718109df0 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.315+0000 7f8715ffb640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87040b6620 0x7f87040b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.315+0000 7f8717fff640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87040b6620 0x7f87040b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.315+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8710108c10 con 0x7f8718109df0 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.316+0000 7f8715ffb640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] conn(0x7f87040c0490 0x7f87040c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.316+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 --> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87040c3050 con 0x7f87040c0490 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.316+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f8710109060 con 0x7f8718109df0 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.316+0000 7f8717fff640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87040b6620 0x7f87040b8b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f870800abe0 tx=0x7f870800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.316+0000 7f870ffff640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] conn(0x7f87040c0490 0x7f87040c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.316+0000 7f870ffff640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] conn(0x7f87040c0490 0x7f87040c2980 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.317+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== osd.4 v2:172.21.15.165:6808/3903622433 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87040c3050 con 0x7f87040c0490 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.331+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 --> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8718063fd0 con 0x7f87040c0490 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f8715ffb640 1 -- 172.21.15.31:0/1627004767 <== osd.4 v2:172.21.15.165:6808/3903622433 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8718063fd0 con 0x7f87040c0490 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] conn(0x7f87040c0490 msgr2=0x7f87040c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] conn(0x7f87040c0490 0x7f87040c2980 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87040b6620 msgr2=0x7f87040b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f87040b6620 0x7f87040b8b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f870800abe0 tx=0x7f870800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 msgr2=0x7f871819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 --2- 172.21.15.31:0/1627004767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8718109df0 0x7f871819fce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8710005cc0 tx=0x7f871002f380 comp rx=0 tx=0).stop 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871cbef640 1 -- 172.21.15.31:0/1627004767 reap_dead start 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 shutdown_connections 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 >> 172.21.15.31:0/1627004767 conn(0x7f871807b850 msgr2=0x7f87181054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 shutdown_connections 2024-09-21T02:51:49.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:49.332+0000 7f871e652640 1 -- 172.21.15.31:0/1627004767 wait complete. 2024-09-21T02:51:50.237 INFO:teuthology.orchestra.run.smithi031.stdout:137438953533 2024-09-21T02:51:50.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T02:51:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:50 smithi031 ceph-mon[31209]: pgmap v293: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-09-21T02:51:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:50 smithi165 ceph-mon[34818]: pgmap v293: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-09-21T02:51:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:50 smithi178 ceph-mon[34759]: pgmap v293: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-09-21T02:51:51.096 INFO:teuthology.orchestra.run.smithi031.stdout:60129542219 2024-09-21T02:51:51.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T02:51:51.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 -- 172.21.15.31:0/2445824400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 msgr2=0x7fe4b8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:51.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/2445824400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b8102e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009980 tx=0x7fe4a802e830 comp rx=0 tx=0).stop 2024-09-21T02:51:51.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 -- 172.21.15.31:0/2445824400 shutdown_connections 2024-09-21T02:51:51.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/2445824400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4b8108cf0 0x7fe4b810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:51.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/2445824400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b81083a0 0x7fe4b81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/2445824400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b8102e70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 -- 172.21.15.31:0/2445824400 >> 172.21.15.31:0/2445824400 conn(0x7fe4b8079730 msgr2=0x7fe4b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 -- 172.21.15.31:0/2445824400 shutdown_connections 2024-09-21T02:51:51.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.276+0000 7fe4be4fe640 1 -- 172.21.15.31:0/2445824400 wait complete. 2024-09-21T02:51:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.277+0000 7fe4be4fe640 1 Processor -- start 2024-09-21T02:51:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.277+0000 7fe4be4fe640 1 -- start start 2024-09-21T02:51:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4be4fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4be4fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b81083a0 0x7fe4b8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4be4fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4b8108cf0 0x7fe4b819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4be4fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8111860 con 0x7fe4b8102a60 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4be4fe640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81116e0 con 0x7fe4b81083a0 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4be4fe640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81119e0 con 0x7fe4b8108cf0 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41442/0 (socket says 172.21.15.31:41442) 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 -- 172.21.15.31:0/3232470935 learned_addr learned my addr 172.21.15.31:0/3232470935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b77fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b81083a0 0x7fe4b8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4bca9b640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4b8108cf0 0x7fe4b819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 -- 172.21.15.31:0/3232470935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4b8108cf0 msgr2=0x7fe4b819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4b8108cf0 0x7fe4b819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 -- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b81083a0 msgr2=0x7fe4b8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b81083a0 0x7fe4b8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b7fff640 1 -- 172.21.15.31:0/3232470935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b819e160 con 0x7fe4b8102a60 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.278+0000 7fe4b77fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b81083a0 0x7fe4b8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.279+0000 7fe4b7fff640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b81981f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000c00 tx=0x7fe4a8036990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.279+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8044030 con 0x7fe4b8102a60 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.279+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b819e3f0 con 0x7fe4b8102a60 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.279+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4a8004410 con 0x7fe4b8102a60 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.279+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a803a070 con 0x7fe4b8102a60 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.279+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81a4bb0 con 0x7fe4b8102a60 2024-09-21T02:51:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.280+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4b8063b50 con 0x7fe4b8102a60 2024-09-21T02:51:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.282+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe4a8047030 con 0x7fe4b8102a60 2024-09-21T02:51:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.282+0000 7fe4b57fa640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe4a00b65f0 0x7fe4a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4b77fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe4a00b65f0 0x7fe4a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe4a810be10 con 0x7fe4b8102a60 2024-09-21T02:51:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4b57fa640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] conn(0x7fe4a00c0460 0x7fe4a00c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 --> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4a00c3020 con 0x7fe4a00c0460 2024-09-21T02:51:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fe4a810c260 con 0x7fe4b8102a60 2024-09-21T02:51:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4b77fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe4a00b65f0 0x7fe4a00b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8100cc0 tx=0x7fe4ac00d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.283+0000 7fe4bca9b640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] conn(0x7fe4a00c0460 0x7fe4a00c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.284+0000 7fe4bca9b640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] conn(0x7fe4a00c0460 0x7fe4a00c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:51.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.285+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== osd.10 v2:172.21.15.165:6824/3486886499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4a00c3020 con 0x7fe4a00c0460 2024-09-21T02:51:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.299+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 --> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4b8063f80 con 0x7fe4a00c0460 2024-09-21T02:51:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.300+0000 7fe4b57fa640 1 -- 172.21.15.31:0/3232470935 <== osd.10 v2:172.21.15.165:6824/3486886499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe4b8063f80 con 0x7fe4a00c0460 2024-09-21T02:51:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.300+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] conn(0x7fe4a00c0460 msgr2=0x7fe4a00c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.300+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] conn(0x7fe4a00c0460 0x7fe4a00c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe4a00b65f0 msgr2=0x7fe4a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe4a00b65f0 0x7fe4a00b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8100cc0 tx=0x7fe4ac00d040 comp rx=0 tx=0).stop 2024-09-21T02:51:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 msgr2=0x7fe4b81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 --2- 172.21.15.31:0/3232470935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b8102a60 0x7fe4b81981f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000c00 tx=0x7fe4a8036990 comp rx=0 tx=0).stop 2024-09-21T02:51:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4bca9b640 1 -- 172.21.15.31:0/3232470935 reap_dead start 2024-09-21T02:51:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 shutdown_connections 2024-09-21T02:51:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 >> 172.21.15.31:0/3232470935 conn(0x7fe4b8079730 msgr2=0x7fe4b81013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 shutdown_connections 2024-09-21T02:51:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:51.301+0000 7fe4be4fe640 1 -- 172.21.15.31:0/3232470935 wait complete. 2024-09-21T02:51:52.024 INFO:teuthology.orchestra.run.smithi031.stdout:257698037795 2024-09-21T02:51:52.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T02:51:52.458 INFO:teuthology.orchestra.run.smithi031.stdout:111669149762 2024-09-21T02:51:52.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T02:51:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:52 smithi165 ceph-mon[34818]: pgmap v294: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-21T02:51:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:52 smithi178 ceph-mon[34759]: pgmap v294: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-21T02:51:52.829 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:52 smithi031 ceph-mon[31209]: pgmap v294: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-21T02:51:52.862 INFO:teuthology.orchestra.run.smithi031.stdout:206158430252 2024-09-21T02:51:52.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T02:51:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:53.307 INFO:teuthology.orchestra.run.smithi031.stdout:279172874269 2024-09-21T02:51:53.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T02:51:53.322 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:53.373 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:54.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 -- 172.21.15.31:0/1032432197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc1094a0 msgr2=0x7f9bfc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 --2- 172.21.15.31:0/1032432197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc1094a0 0x7f9bfc1098b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009900 tx=0x7f9bf002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:54.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 -- 172.21.15.31:0/1032432197 shutdown_connections 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 --2- 172.21.15.31:0/1032432197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bfc109df0 0x7f9bfc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 --2- 172.21.15.31:0/1032432197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc1094a0 0x7f9bfc1098b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 --2- 172.21.15.31:0/1032432197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bfc103b60 0x7f9bfc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 -- 172.21.15.31:0/1032432197 >> 172.21.15.31:0/1032432197 conn(0x7f9bfc07b850 msgr2=0x7f9bfc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 -- 172.21.15.31:0/1032432197 shutdown_connections 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.072+0000 7f9c02904640 1 -- 172.21.15.31:0/1032432197 wait complete. 2024-09-21T02:51:54.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 Processor -- start 2024-09-21T02:51:54.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 -- start start 2024-09-21T02:51:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 0x7f9bfc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bfc1094a0 0x7f9bfc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bfc109df0 0x7f9bfc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc113ab0 con 0x7f9bfc103b60 2024-09-21T02:51:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc113930 con 0x7f9bfc109df0 2024-09-21T02:51:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.073+0000 7f9c02904640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc113c30 con 0x7f9bfc1094a0 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 0x7f9bfc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 0x7f9bfc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41464/0 (socket says 172.21.15.31:41464) 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 -- 172.21.15.31:0/2893735026 learned_addr learned my addr 172.21.15.31:0/2893735026 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfb7fe640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bfc1094a0 0x7f9bfc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9c00ea1640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bfc109df0 0x7f9bfc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 -- 172.21.15.31:0/2893735026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bfc1094a0 msgr2=0x7f9bfc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bfc1094a0 0x7f9bfc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 -- 172.21.15.31:0/2893735026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bfc109df0 msgr2=0x7f9bfc19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bfc109df0 0x7f9bfc19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 -- 172.21.15.31:0/2893735026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc1a0450 con 0x7f9bfc103b60 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bfbfff640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 0x7f9bfc19a5a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00d950 tx=0x7f9bec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec017070 con 0x7f9bfc103b60 2024-09-21T02:51:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec009c30 con 0x7f9bfc103b60 2024-09-21T02:51:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc1a0740 con 0x7f9bfc103b60 2024-09-21T02:51:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.074+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00f620 con 0x7f9bfc103b60 2024-09-21T02:51:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.075+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc1a6f60 con 0x7f9bfc103b60 2024-09-21T02:51:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.076+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bfc104de0 con 0x7f9bfc103b60 2024-09-21T02:51:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.077+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9bec010460 con 0x7f9bfc103b60 2024-09-21T02:51:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.078+0000 7f9bf97fa640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9be40b6620 0x7f9be40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.078+0000 7f9bfb7fe640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9be40b6620 0x7f9be40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.078+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9bec0e71f0 con 0x7f9bfc103b60 2024-09-21T02:51:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.079+0000 7f9bfb7fe640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9be40b6620 0x7f9be40b8b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009230 tx=0x7f9bf0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.082+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9bec0b1ba0 con 0x7f9bfc103b60 2024-09-21T02:51:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.273+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9bfc10bbc0 con 0x7f9bfc103b60 2024-09-21T02:51:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.273+0000 7f9bf97fa640 1 -- 172.21.15.31:0/2893735026 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9bec0b6ae0 con 0x7f9bfc103b60 2024-09-21T02:51:54.275 INFO:teuthology.orchestra.run.smithi031.stdout:184683593779 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9be40b6620 msgr2=0x7f9be40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9be40b6620 0x7f9be40b8b30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009230 tx=0x7f9bf0031400 comp rx=0 tx=0).stop 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 msgr2=0x7f9bfc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 0x7f9bfc19a5a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00d950 tx=0x7f9bec00de10 comp rx=0 tx=0).stop 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 shutdown_connections 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9be40b6620 0x7f9be40b8b30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bfc1094a0 0x7f9bfc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bfc109df0 0x7f9bfc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 --2- 172.21.15.31:0/2893735026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bfc103b60 0x7f9bfc19a5a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 >> 172.21.15.31:0/2893735026 conn(0x7f9bfc07b850 msgr2=0x7f9bfc105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 shutdown_connections 2024-09-21T02:51:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.275+0000 7f9c02904640 1 -- 172.21.15.31:0/2893735026 wait complete. 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 -- 172.21.15.31:0/1899579823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 msgr2=0x7f3978103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 --2- 172.21.15.31:0/1899579823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f3978103fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3974009900 tx=0x7f397402a410 comp rx=0 tx=0).stop 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 -- 172.21.15.31:0/1899579823 shutdown_connections 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 --2- 172.21.15.31:0/1899579823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39781094e0 0x7f39781098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 --2- 172.21.15.31:0/1899579823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3978109e30 0x7f3978110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 --2- 172.21.15.31:0/1899579823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f3978103fb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.289+0000 7f398078c640 1 -- 172.21.15.31:0/1899579823 >> 172.21.15.31:0/1899579823 conn(0x7f39780ff630 msgr2=0x7f3978101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.290+0000 7f398078c640 1 -- 172.21.15.31:0/1899579823 shutdown_connections 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.290+0000 7f398078c640 1 -- 172.21.15.31:0/1899579823 wait complete. 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.290+0000 7f398078c640 1 Processor -- start 2024-09-21T02:51:54.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.290+0000 7f398078c640 1 -- start start 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f398078c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f397819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f398078c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39781094e0 0x7f397819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f398078c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978109e30 0x7f397819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f398078c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3978113b70 con 0x7f3978103ba0 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f398078c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39781139f0 con 0x7f39781094e0 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f398078c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3978113cf0 con 0x7f3978109e30 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f397819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f397819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41486/0 (socket says 172.21.15.31:41486) 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 -- 172.21.15.31:0/1941028725 learned_addr learned my addr 172.21.15.31:0/1941028725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397dd27640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39781094e0 0x7f397819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397ed29640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978109e30 0x7f397819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 -- 172.21.15.31:0/1941028725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978109e30 msgr2=0x7f397819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978109e30 0x7f397819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 -- 172.21.15.31:0/1941028725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39781094e0 msgr2=0x7f397819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39781094e0 0x7f397819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 -- 172.21.15.31:0/1941028725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781a0530 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.291+0000 7f397e528640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f397819a5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3974005940 tx=0x7f39740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.292+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397403f030 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.292+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781a07c0 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.292+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3974003cb0 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.292+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974035070 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.292+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a6f80 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.293+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3978103fb0 con 0x7f3978103ba0 2024-09-21T02:51:54.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.296+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3974042030 con 0x7f3978103ba0 2024-09-21T02:51:54.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.297+0000 7f396f7fe640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f39640b65f0 0x7f39640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:54.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.297+0000 7f397dd27640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f39640b65f0 0x7f39640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:54.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.297+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f39741080a0 con 0x7f3978103ba0 2024-09-21T02:51:54.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.298+0000 7f397dd27640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f39640b65f0 0x7f39640b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3978102d30 tx=0x7f396800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:54.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.302+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39740ce770 con 0x7f3978103ba0 2024-09-21T02:51:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.497+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f397819b6f0 con 0x7f3978103ba0 2024-09-21T02:51:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.497+0000 7f396f7fe640 1 -- 172.21.15.31:0/1941028725 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f397819b6f0 con 0x7f3978103ba0 2024-09-21T02:51:54.499 INFO:teuthology.orchestra.run.smithi031.stdout:60129542220 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f39640b65f0 msgr2=0x7f39640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f39640b65f0 0x7f39640b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3978102d30 tx=0x7f396800b390 comp rx=0 tx=0).stop 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 msgr2=0x7f397819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f397819a5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3974005940 tx=0x7f39740042f0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 shutdown_connections 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f39640b65f0 0x7f39640b8b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3978109e30 0x7f397819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39781094e0 0x7f397819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 --2- 172.21.15.31:0/1941028725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3978103ba0 0x7f397819a5c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 >> 172.21.15.31:0/1941028725 conn(0x7f39780ff630 msgr2=0x7f3978108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 shutdown_connections 2024-09-21T02:51:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:54.500+0000 7f398078c640 1 -- 172.21.15.31:0/1941028725 wait complete. 2024-09-21T02:51:54.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:54 smithi031 ceph-mon[31209]: pgmap v295: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:51:54.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:54 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2893735026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:51:54.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:54 smithi165 ceph-mon[34818]: pgmap v295: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:51:54.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:54 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2893735026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:51:54.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:54 smithi178 ceph-mon[34759]: pgmap v295: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:51:54.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:54 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2893735026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:51:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 -- 172.21.15.31:0/725767902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4075730 msgr2=0x7fd9d4075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 --2- 172.21.15.31:0/725767902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4075730 0x7fd9d4075b40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009900 tx=0x7fd9c402a410 comp rx=0 tx=0).stop 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 -- 172.21.15.31:0/725767902 shutdown_connections 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 --2- 172.21.15.31:0/725767902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4110140 0x7fd9d4112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 --2- 172.21.15.31:0/725767902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4076080 0x7fd9d410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 --2- 172.21.15.31:0/725767902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4075730 0x7fd9d4075b40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 -- 172.21.15.31:0/725767902 >> 172.21.15.31:0/725767902 conn(0x7fd9d407b850 msgr2=0x7fd9d407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 -- 172.21.15.31:0/725767902 shutdown_connections 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.288+0000 7fd9da51e640 1 -- 172.21.15.31:0/725767902 wait complete. 2024-09-21T02:51:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 Processor -- start 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 -- start start 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4075730 0x7fd9d419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4076080 0x7fd9d419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 0x7fd9d41a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4115c80 con 0x7fd9d4110140 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4115b00 con 0x7fd9d4075730 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.289+0000 7fd9da51e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4115e00 con 0x7fd9d4076080 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 0x7fd9d41a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4075730 0x7fd9d419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 0x7fd9d41a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41494/0 (socket says 172.21.15.31:41494) 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 -- 172.21.15.31:0/3450263446 learned_addr learned my addr 172.21.15.31:0/3450263446 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4076080 0x7fd9d419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 -- 172.21.15.31:0/3450263446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4076080 msgr2=0x7fd9d419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4076080 0x7fd9d419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 -- 172.21.15.31:0/3450263446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4075730 msgr2=0x7fd9d419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4075730 0x7fd9d419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 -- 172.21.15.31:0/3450263446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d41a25c0 con 0x7fd9d4110140 2024-09-21T02:51:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d8abb640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 0x7fd9d41a1e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00eaa0 tx=0x7fd9cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00cdb0 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc005020 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d41a28b0 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0116d0 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.290+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d41a9010 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.292+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d4075b40 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.293+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd9cc0051d0 con 0x7fd9d4110140 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.294+0000 7fd9d17fa640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b65f0 0x7fd9bc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.294+0000 7fd9d3fff640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b65f0 0x7fd9bc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.294+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd9cc0e7750 con 0x7fd9d4110140 2024-09-21T02:51:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.294+0000 7fd9d3fff640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b65f0 0x7fd9bc0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4031440 tx=0x7fd9c4009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.298+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9cc016030 con 0x7fd9d4110140 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:55 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1941028725' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:51:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.497+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd9d419d780 con 0x7fd9d4110140 2024-09-21T02:51:55.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.497+0000 7fd9d17fa640 1 -- 172.21.15.31:0/3450263446 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd9cc0b2160 con 0x7fd9d4110140 2024-09-21T02:51:55.498 INFO:teuthology.orchestra.run.smithi031.stdout:137438953534 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b65f0 msgr2=0x7fd9bc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b65f0 0x7fd9bc0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4031440 tx=0x7fd9c4009560 comp rx=0 tx=0).stop 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 msgr2=0x7fd9d41a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 0x7fd9d41a1e90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00eaa0 tx=0x7fd9cc00ef60 comp rx=0 tx=0).stop 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 shutdown_connections 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9bc0b65f0 0x7fd9bc0b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d4076080 0x7fd9d419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d4110140 0x7fd9d41a1e90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 --2- 172.21.15.31:0/3450263446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d4075730 0x7fd9d419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:55.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 >> 172.21.15.31:0/3450263446 conn(0x7fd9d407b850 msgr2=0x7fd9d41018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 shutdown_connections 2024-09-21T02:51:55.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:55.499+0000 7fd9da51e640 1 -- 172.21.15.31:0/3450263446 wait complete. 2024-09-21T02:51:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:55.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:55 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1941028725' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:51:55.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542219 got 60129542220 for osd.1 2024-09-21T02:51:55.706 DEBUG:teuthology.parallel:result is None 2024-09-21T02:51:55.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:55 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1941028725' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:51:55.900 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:56.370 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:56.621 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:56 smithi031 ceph-mon[31209]: pgmap v296: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:56.621 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:56 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3450263446' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:51:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:56 smithi165 ceph-mon[34818]: pgmap v296: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:56.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:56 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3450263446' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:51:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:56 smithi178 ceph-mon[34759]: pgmap v296: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:51:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:56 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3450263446' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:51:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 -- 172.21.15.31:0/883069525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c10de10 msgr2=0x7f996c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 --2- 172.21.15.31:0/883069525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c10de10 0x7f996c110300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9964009ca0 tx=0x7f996402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 -- 172.21.15.31:0/883069525 shutdown_connections 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 --2- 172.21.15.31:0/883069525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c10de10 0x7f996c110300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 --2- 172.21.15.31:0/883069525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c105300 0x7f996c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 --2- 172.21.15.31:0/883069525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1049b0 0x7f996c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 -- 172.21.15.31:0/883069525 >> 172.21.15.31:0/883069525 conn(0x7f996c079730 msgr2=0x7f996c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 -- 172.21.15.31:0/883069525 shutdown_connections 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 -- 172.21.15.31:0/883069525 wait complete. 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.769+0000 7f997447d640 1 Processor -- start 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 -- start start 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1049b0 0x7f996c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 0x7f996c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c10de10 0x7f996c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:56.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113a70 con 0x7f996c105300 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1138f0 con 0x7f996c10de10 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f997447d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f996c113bf0 con 0x7f996c1049b0 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 0x7f996c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 0x7f996c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41514/0 (socket says 172.21.15.31:41514) 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 -- 172.21.15.31:0/1388522999 learned_addr learned my addr 172.21.15.31:0/1388522999 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 -- 172.21.15.31:0/1388522999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1049b0 msgr2=0x7f996c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9972219640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1049b0 0x7f996c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1049b0 0x7f996c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9972a1a640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c10de10 0x7f996c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 -- 172.21.15.31:0/1388522999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c10de10 msgr2=0x7f996c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c10de10 0x7f996c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 -- 172.21.15.31:0/1388522999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1a0410 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9972a1a640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c10de10 0x7f996c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.770+0000 7f9971a18640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 0x7f996c19aa40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f995c00d740 tx=0x7f995c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.771+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c017070 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.771+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c00de50 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.771+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1a0700 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.771+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00f620 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.771+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1a6e60 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.772+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c104dc0 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.774+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f995c01a070 con 0x7f996c105300 2024-09-21T02:51:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.774+0000 7f99637fe640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f99580b6670 0x7f99580b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.774+0000 7f9972219640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f99580b6670 0x7f99580b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.775+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f995c0e6ff0 con 0x7f996c105300 2024-09-21T02:51:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.775+0000 7f9972219640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f99580b6670 0x7f99580b8b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9968005fd0 tx=0x7f9968005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:56.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.779+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f995c0ad620 con 0x7f996c105300 2024-09-21T02:51:56.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.969+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f996c1a7150 con 0x7f996c105300 2024-09-21T02:51:56.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.970+0000 7f99637fe640 1 -- 172.21.15.31:0/1388522999 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f995c0b1a00 con 0x7f996c105300 2024-09-21T02:51:56.972 INFO:teuthology.orchestra.run.smithi031.stdout:158913790008 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f99580b6670 msgr2=0x7f99580b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f99580b6670 0x7f99580b8b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9968005fd0 tx=0x7f9968005850 comp rx=0 tx=0).stop 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 msgr2=0x7f996c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 0x7f996c19aa40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f995c00d740 tx=0x7f995c00dc00 comp rx=0 tx=0).stop 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 shutdown_connections 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f99580b6670 0x7f99580b8b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f996c10de10 0x7f996c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f996c105300 0x7f996c19aa40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 --2- 172.21.15.31:0/1388522999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1049b0 0x7f996c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:56.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 >> 172.21.15.31:0/1388522999 conn(0x7f996c079730 msgr2=0x7f996c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:56.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 shutdown_connections 2024-09-21T02:51:56.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:56.972+0000 7f997447d640 1 -- 172.21.15.31:0/1388522999 wait complete. 2024-09-21T02:51:57.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593777 got 184683593779 for osd.6 2024-09-21T02:51:57.252 DEBUG:teuthology.parallel:result is None 2024-09-21T02:51:57.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 -- 172.21.15.31:0/14680818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5281094f0 msgr2=0x7fb528109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:57.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 --2- 172.21.15.31:0/14680818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5281094f0 0x7fb528109900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb51802a410 comp rx=0 tx=0).stop 2024-09-21T02:51:57.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 -- 172.21.15.31:0/14680818 shutdown_connections 2024-09-21T02:51:57.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 --2- 172.21.15.31:0/14680818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb528109e40 0x7fb528110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 --2- 172.21.15.31:0/14680818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5281094f0 0x7fb528109900 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 --2- 172.21.15.31:0/14680818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb528103bb0 0x7fb528103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 -- 172.21.15.31:0/14680818 >> 172.21.15.31:0/14680818 conn(0x7fb5280ff600 msgr2=0x7fb528101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 -- 172.21.15.31:0/14680818 shutdown_connections 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 -- 172.21.15.31:0/14680818 wait complete. 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.465+0000 7fb530724640 1 Processor -- start 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 -- start start 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 0x7fb52819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5281094f0 0x7fb52819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb528109e40 0x7fb52819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb528113b00 con 0x7fb528103bb0 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb528113980 con 0x7fb528109e40 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb530724640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb528113c80 con 0x7fb5281094f0 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 0x7fb52819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:57.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 0x7fb52819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41536/0 (socket says 172.21.15.31:41536) 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 -- 172.21.15.31:0/2985151581 learned_addr learned my addr 172.21.15.31:0/2985151581 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52dcbf640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5281094f0 0x7fb52819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 -- 172.21.15.31:0/2985151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5281094f0 msgr2=0x7fb52819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5281094f0 0x7fb52819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52ecc1640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb528109e40 0x7fb52819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 -- 172.21.15.31:0/2985151581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb528109e40 msgr2=0x7fb52819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb528109e40 0x7fb52819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 -- 172.21.15.31:0/2985151581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5281a0460 con 0x7fb528103bb0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.466+0000 7fb52e4c0640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 0x7fb52819a4f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d950 tx=0x7fb52400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.467+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524017070 con 0x7fb528103bb0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.467+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281a0750 con 0x7fb528103bb0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.467+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb524009c30 con 0x7fb528103bb0 2024-09-21T02:51:57.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.467+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52400f620 con 0x7fb528103bb0 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.467+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a6eb0 con 0x7fb528103bb0 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.468+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb528103fc0 con 0x7fb528103bb0 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.469+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb524005030 con 0x7fb528103bb0 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.470+0000 7fb51f7fe640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb5140b65f0 0x7fb5140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.470+0000 7fb52dcbf640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb5140b65f0 0x7fb5140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.470+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb5240e7310 con 0x7fb528103bb0 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.471+0000 7fb52dcbf640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb5140b65f0 0x7fb5140b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb518000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:57.477 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:57 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1388522999' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:51:57.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.473+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5240ad8e0 con 0x7fb528103bb0 2024-09-21T02:51:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.665+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb52802e640 con 0x7fb528103bb0 2024-09-21T02:51:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.665+0000 7fb51f7fe640 1 -- 172.21.15.31:0/2985151581 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb524009de0 con 0x7fb528103bb0 2024-09-21T02:51:57.667 INFO:teuthology.orchestra.run.smithi031.stdout:300647710746 2024-09-21T02:51:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb5140b65f0 msgr2=0x7fb5140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb5140b65f0 0x7fb5140b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb518000bc0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 msgr2=0x7fb52819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 0x7fb52819a4f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d950 tx=0x7fb52400de10 comp rx=0 tx=0).stop 2024-09-21T02:51:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 shutdown_connections 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb5140b65f0 0x7fb5140b8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5281094f0 0x7fb52819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb528109e40 0x7fb52819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 --2- 172.21.15.31:0/2985151581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb528103bb0 0x7fb52819a4f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 >> 172.21.15.31:0/2985151581 conn(0x7fb5280ff600 msgr2=0x7fb528101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 shutdown_connections 2024-09-21T02:51:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:57.668+0000 7fb530724640 1 -- 172.21.15.31:0/2985151581 wait complete. 2024-09-21T02:51:57.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:57 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1388522999' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:51:57.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:57 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1388522999' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:51:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 -- 172.21.15.31:0/3437391469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 msgr2=0x7fed14104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/3437391469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed14104dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fed10009900 tx=0x7fed1002a410 comp rx=0 tx=0).stop 2024-09-21T02:51:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 -- 172.21.15.31:0/3437391469 shutdown_connections 2024-09-21T02:51:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/3437391469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed14105310 0x7fed1410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/3437391469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed1410de20 0x7fed14110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/3437391469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed14104dd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 -- 172.21.15.31:0/3437391469 >> 172.21.15.31:0/3437391469 conn(0x7fed14079710 msgr2=0x7fed14079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:58.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 -- 172.21.15.31:0/3437391469 shutdown_connections 2024-09-21T02:51:58.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.147+0000 7fed1bf4d640 1 -- 172.21.15.31:0/3437391469 wait complete. 2024-09-21T02:51:58.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 Processor -- start 2024-09-21T02:51:58.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 -- start start 2024-09-21T02:51:58.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed1419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed14105310 0x7fed1419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed1410de20 0x7fed1419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed14113a80 con 0x7fed141049c0 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed14113900 con 0x7fed1410de20 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed1bf4d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed14113c00 con 0x7fed14105310 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed19ce9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed1419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed19ce9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed1419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41546/0 (socket says 172.21.15.31:41546) 2024-09-21T02:51:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed19ce9640 1 -- 172.21.15.31:0/4006622658 learned_addr learned my addr 172.21.15.31:0/4006622658 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.148+0000 7fed194e8640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed14105310 0x7fed1419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed1a4ea640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed1410de20 0x7fed1419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed19ce9640 1 -- 172.21.15.31:0/4006622658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed14105310 msgr2=0x7fed1419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed19ce9640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed14105310 0x7fed1419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed19ce9640 1 -- 172.21.15.31:0/4006622658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed1410de20 msgr2=0x7fed1419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed19ce9640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed1410de20 0x7fed1419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed19ce9640 1 -- 172.21.15.31:0/4006622658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed141a03c0 con 0x7fed141049c0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed19ce9640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed1419a510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed10005940 tx=0x7fed100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1003f030 con 0x7fed141049c0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed10003cb0 con 0x7fed141049c0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141a0650 con 0x7fed141049c0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10035070 con 0x7fed141049c0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.149+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141a0b80 con 0x7fed141049c0 2024-09-21T02:51:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.150+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1406c280 con 0x7fed141049c0 2024-09-21T02:51:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.152+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fed10042030 con 0x7fed141049c0 2024-09-21T02:51:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.153+0000 7fed0affd640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed000b66a0 0x7fed000b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.153+0000 7fed194e8640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed000b66a0 0x7fed000b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.153+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fed10107d50 con 0x7fed141049c0 2024-09-21T02:51:58.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.154+0000 7fed194e8640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed000b66a0 0x7fed000b8bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fed1410fcc0 tx=0x7fed0400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:58.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.157+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed141a0b80 con 0x7fed141049c0 2024-09-21T02:51:58.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.348+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fed14104dd0 con 0x7fed141049c0 2024-09-21T02:51:58.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.349+0000 7fed0affd640 1 -- 172.21.15.31:0/4006622658 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fed100ce380 con 0x7fed141049c0 2024-09-21T02:51:58.350 INFO:teuthology.orchestra.run.smithi031.stdout:231928234025 2024-09-21T02:51:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed000b66a0 msgr2=0x7fed000b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:58.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed000b66a0 0x7fed000b8bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fed1410fcc0 tx=0x7fed0400b410 comp rx=0 tx=0).stop 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 msgr2=0x7fed1419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed1419a510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed10005940 tx=0x7fed100042f0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 shutdown_connections 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed000b66a0 0x7fed000b8bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed14105310 0x7fed1419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed1410de20 0x7fed1419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 --2- 172.21.15.31:0/4006622658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed141049c0 0x7fed1419a510 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 >> 172.21.15.31:0/4006622658 conn(0x7fed14079710 msgr2=0x7fed141068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 shutdown_connections 2024-09-21T02:51:58.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.351+0000 7fed1bf4d640 1 -- 172.21.15.31:0/4006622658 wait complete. 2024-09-21T02:51:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:58 smithi031 ceph-mon[31209]: pgmap v297: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T02:51:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2985151581' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:51:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4006622658' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:51:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:58 smithi165 ceph-mon[34818]: pgmap v297: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T02:51:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2985151581' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:51:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/4006622658' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:51:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:58 smithi178 ceph-mon[34759]: pgmap v297: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-21T02:51:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2985151581' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:51:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/4006622658' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:51:58.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790008 for osd.5 2024-09-21T02:51:58.964 DEBUG:teuthology.parallel:result is None 2024-09-21T02:51:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 -- 172.21.15.31:0/2884696335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 msgr2=0x7f07c4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 --2- 172.21.15.31:0/2884696335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4102e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009900 tx=0x7f07ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:51:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 -- 172.21.15.31:0/2884696335 shutdown_connections 2024-09-21T02:51:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 --2- 172.21.15.31:0/2884696335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07c41083a0 0x7f07c41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 --2- 172.21.15.31:0/2884696335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c4108cf0 0x7f07c410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 --2- 172.21.15.31:0/2884696335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4102e70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 -- 172.21.15.31:0/2884696335 >> 172.21.15.31:0/2884696335 conn(0x7f07c4079730 msgr2=0x7f07c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 -- 172.21.15.31:0/2884696335 shutdown_connections 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 -- 172.21.15.31:0/2884696335 wait complete. 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.407+0000 7f07cab56640 1 Processor -- start 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 -- start start 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c41083a0 0x7f07c4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07c4108cf0 0x7f07c4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4112960 con 0x7f07c4102a60 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41127e0 con 0x7f07c41083a0 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07cab56640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4112ae0 con 0x7f07c4108cf0 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41564/0 (socket says 172.21.15.31:41564) 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 -- 172.21.15.31:0/3430957162 learned_addr learned my addr 172.21.15.31:0/3430957162 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c3fff640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c41083a0 0x7f07c4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c90f3640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07c4108cf0 0x7f07c4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 -- 172.21.15.31:0/3430957162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07c4108cf0 msgr2=0x7f07c4199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07c4108cf0 0x7f07c4199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 -- 172.21.15.31:0/3430957162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c41083a0 msgr2=0x7f07c4199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c41083a0 0x7f07c4199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 -- 172.21.15.31:0/3430957162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c41a86f0 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c3fff640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c41083a0 0x7f07c4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.408+0000 7f07c88f2640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4199040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005940 tx=0x7f07ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.409+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac03f030 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.409+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07ac0044c0 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.409+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c41a8980 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.409+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac035070 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.409+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a8e60 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.410+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c4102e70 con 0x7f07c4102a60 2024-09-21T02:51:59.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.412+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f07ac042030 con 0x7f07c4102a60 2024-09-21T02:51:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.413+0000 7f07c1ffb640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07b00b65f0 0x7f07b00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:51:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.413+0000 7f07c3fff640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07b00b65f0 0x7f07b00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:51:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.413+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f07ac107d20 con 0x7f07c4102a60 2024-09-21T02:51:59.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.413+0000 7f07c3fff640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07b00b65f0 0x7f07b00b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f07c4100b70 tx=0x7f07b4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:51:59.049 INFO:teuthology.orchestra.run.smithi031.stdout:111669149763 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.416+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07ac0ce350 con 0x7f07c4102a60 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.609+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f07c402e640 con 0x7f07c4102a60 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.610+0000 7f07c1ffb640 1 -- 172.21.15.31:0/3430957162 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f07ac0d2730 con 0x7f07c4102a60 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07b00b65f0 msgr2=0x7f07b00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07b00b65f0 0x7f07b00b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f07c4100b70 tx=0x7f07b4005f90 comp rx=0 tx=0).stop 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 msgr2=0x7f07c4199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4199040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005940 tx=0x7f07ac0040c0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 shutdown_connections 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07b00b65f0 0x7f07b00b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07c4108cf0 0x7f07c4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07c41083a0 0x7f07c4199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 --2- 172.21.15.31:0/3430957162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07c4102a60 0x7f07c4199040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 >> 172.21.15.31:0/3430957162 conn(0x7f07c4079730 msgr2=0x7f07c4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 shutdown_connections 2024-09-21T02:51:59.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:58.612+0000 7f07cab56640 1 -- 172.21.15.31:0/3430957162 wait complete. 2024-09-21T02:51:59.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:51:59 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3430957162' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:51:59.809 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:51:59 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3430957162' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:51:59.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:51:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3430957162' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:52:00.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953533 got 137438953534 for osd.4 2024-09-21T02:52:00.512 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:00.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:00 smithi031 ceph-mon[31209]: pgmap v298: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:00.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:00 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/123646135' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:52:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:00 smithi178 ceph-mon[34759]: pgmap v298: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:00 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/123646135' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:52:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:00 smithi165 ceph-mon[34818]: pgmap v298: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:00 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/123646135' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:52:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:01.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149762 got 111669149763 for osd.3 2024-09-21T02:52:01.073 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 -- 172.21.15.31:0/4284995399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4103bc0 msgr2=0x7f93b4103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 --2- 172.21.15.31:0/4284995399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4103bc0 0x7f93b4103fd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009900 tx=0x7f93b002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 -- 172.21.15.31:0/4284995399 shutdown_connections 2024-09-21T02:52:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 --2- 172.21.15.31:0/4284995399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b4109e00 0x7f93b4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 --2- 172.21.15.31:0/4284995399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b41094b0 0x7f93b41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 --2- 172.21.15.31:0/4284995399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4103bc0 0x7f93b4103fd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 -- 172.21.15.31:0/4284995399 >> 172.21.15.31:0/4284995399 conn(0x7f93b407b910 msgr2=0x7f93b407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 -- 172.21.15.31:0/4284995399 shutdown_connections 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.903+0000 7f93bb67d640 1 -- 172.21.15.31:0/4284995399 wait complete. 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 Processor -- start 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 -- start start 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b4103bc0 0x7f93b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b41094b0 0x7f93b419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 0x7f93b419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4113ac0 con 0x7f93b4109e00 2024-09-21T02:52:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4113940 con 0x7f93b41094b0 2024-09-21T02:52:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93bb67d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4113c40 con 0x7f93b4103bc0 2024-09-21T02:52:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93b9c1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 0x7f93b419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93b9c1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 0x7f93b419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47218/0 (socket says 172.21.15.31:47218) 2024-09-21T02:52:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93b9c1a640 1 -- 172.21.15.31:0/123646135 learned_addr learned my addr 172.21.15.31:0/123646135 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93b9419640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b4103bc0 0x7f93b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.904+0000 7f93b8c18640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b41094b0 0x7f93b419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93b9c1a640 1 -- 172.21.15.31:0/123646135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b4103bc0 msgr2=0x7f93b419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93b9c1a640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b4103bc0 0x7f93b419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93b9c1a640 1 -- 172.21.15.31:0/123646135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b41094b0 msgr2=0x7f93b419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93b9c1a640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b41094b0 0x7f93b419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93b9c1a640 1 -- 172.21.15.31:0/123646135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b41a0360 con 0x7f93b4109e00 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93b9c1a640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 0x7f93b419fcf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00cad0 tx=0x7f93ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00ae20 con 0x7f93b4109e00 2024-09-21T02:52:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ac005020 con 0x7f93b4109e00 2024-09-21T02:52:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41a0650 con 0x7f93b4109e00 2024-09-21T02:52:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00f6b0 con 0x7f93b4109e00 2024-09-21T02:52:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.905+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41a6e70 con 0x7f93b4109e00 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.906+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b4103fd0 con 0x7f93b4109e00 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.908+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f93ac0051d0 con 0x7f93b4109e00 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.909+0000 7f93aa7fc640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93a00b6620 0x7f93a00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.909+0000 7f93b9419640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93a00b6620 0x7f93a00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.909+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f93ac0e7980 con 0x7f93b4109e00 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.909+0000 7f93b9419640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93a00b6620 0x7f93a00b8b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93b0031440 tx=0x7f93b0005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:51:59.912+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93ac014030 con 0x7f93b4109e00 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.105+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f93b419b5e0 con 0x7f93b4109e00 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi031.stdout:279172874271 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.105+0000 7f93aa7fc640 1 -- 172.21.15.31:0/123646135 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f93b419b5e0 con 0x7f93b4109e00 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93a00b6620 msgr2=0x7f93a00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93a00b6620 0x7f93a00b8b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93b0031440 tx=0x7f93b0005e80 comp rx=0 tx=0).stop 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 msgr2=0x7f93b419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 0x7f93b419fcf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00cad0 tx=0x7f93ac00cf90 comp rx=0 tx=0).stop 2024-09-21T02:52:01.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 shutdown_connections 2024-09-21T02:52:01.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93a00b6620 0x7f93a00b8b30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b4109e00 0x7f93b419fcf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b41094b0 0x7f93b419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 --2- 172.21.15.31:0/123646135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b4103bc0 0x7f93b419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 >> 172.21.15.31:0/123646135 conn(0x7f93b407b910 msgr2=0x7f93b4107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 shutdown_connections 2024-09-21T02:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.108+0000 7f93bb67d640 1 -- 172.21.15.31:0/123646135 wait complete. 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.726+0000 7f5ba19d2640 1 -- 172.21.15.31:0/2744720454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 msgr2=0x7f5b9c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.726+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/2744720454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c10f520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009900 tx=0x7f5b9402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 -- 172.21.15.31:0/2744720454 shutdown_connections 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/2744720454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b9c1083a0 0x7f5b9c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/2744720454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c10f520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/2744720454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 0x7f5b9c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 -- 172.21.15.31:0/2744720454 >> 172.21.15.31:0/2744720454 conn(0x7f5b9c079730 msgr2=0x7f5b9c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 -- 172.21.15.31:0/2744720454 shutdown_connections 2024-09-21T02:52:01.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.727+0000 7f5ba19d2640 1 -- 172.21.15.31:0/2744720454 wait complete. 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.728+0000 7f5ba19d2640 1 Processor -- start 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.728+0000 7f5ba19d2640 1 -- start start 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5ba19d2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 0x7f5b9c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5ba19d2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b9c1083a0 0x7f5b9c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5ba19d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5ba19d2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c111860 con 0x7f5b9c108cf0 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5ba19d2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1116e0 con 0x7f5b9c102a60 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5ba19d2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1119e0 con 0x7f5b9c1083a0 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47234/0 (socket says 172.21.15.31:47234) 2024-09-21T02:52:01.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 -- 172.21.15.31:0/1483332715 learned_addr learned my addr 172.21.15.31:0/1483332715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9affd640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 0x7f5b9c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9a7fc640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b9c1083a0 0x7f5b9c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 -- 172.21.15.31:0/1483332715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b9c1083a0 msgr2=0x7f5b9c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b9c1083a0 0x7f5b9c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 -- 172.21.15.31:0/1483332715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 msgr2=0x7f5b9c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 0x7f5b9c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.729+0000 7f5b9b7fe640 1 -- 172.21.15.31:0/1483332715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c19e160 con 0x7f5b9c108cf0 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5b9affd640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 0x7f5b9c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5b9b7fe640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c19da30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b94004770 tx=0x7f5b94004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9403f030 con 0x7f5b9c108cf0 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b94004440 con 0x7f5b9c108cf0 2024-09-21T02:52:01.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94035070 con 0x7f5b9c108cf0 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c19e3f0 con 0x7f5b9c108cf0 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.730+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a4bb0 con 0x7f5b9c108cf0 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.731+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c102e70 con 0x7f5b9c108cf0 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.733+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5b94042030 con 0x7f5b9c108cf0 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.734+0000 7f5ba09d0640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b880baa00 0x7f5b880bcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.734+0000 7f5b9affd640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b880baa00 0x7f5b880bcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stdout:38654705746 2024-09-21T02:52:01.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.734+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5b94107cc0 con 0x7f5b9c108cf0 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.735+0000 7f5b9affd640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b880baa00 0x7f5b880bcf10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00bfd0 tx=0x7f5b8c00d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.738+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b940ce2f0 con 0x7f5b9c108cf0 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.929+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5b9c10aae0 con 0x7f5b9c108cf0 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.930+0000 7f5ba09d0640 1 -- 172.21.15.31:0/1483332715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b940d26d0 con 0x7f5b9c108cf0 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b880baa00 msgr2=0x7f5b880bcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b880baa00 0x7f5b880bcf10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00bfd0 tx=0x7f5b8c00d040 comp rx=0 tx=0).stop 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 msgr2=0x7f5b9c19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c19da30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b94004770 tx=0x7f5b94004040 comp rx=0 tx=0).stop 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 shutdown_connections 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b880baa00 0x7f5b880bcf10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b9c1083a0 0x7f5b9c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b9c108cf0 0x7f5b9c19da30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 --2- 172.21.15.31:0/1483332715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b9c102a60 0x7f5b9c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 >> 172.21.15.31:0/1483332715 conn(0x7f5b9c079730 msgr2=0x7f5b9c104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 shutdown_connections 2024-09-21T02:52:01.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:00.932+0000 7f5ba19d2640 1 -- 172.21.15.31:0/1483332715 wait complete. 2024-09-21T02:52:01.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:01 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1483332715' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:52:01.790 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:01 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1483332715' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:52:01.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:01 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1483332715' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 -- 172.21.15.31:0/3890843904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328108d20 msgr2=0x7ff32810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3890843904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328108d20 0x7ff32810f550 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff320009ca0 tx=0x7ff32002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 -- 172.21.15.31:0/3890843904 shutdown_connections 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3890843904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328108d20 0x7ff32810f550 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3890843904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3281083d0 0x7ff3281087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3890843904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff328102ae0 0x7ff328102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 -- 172.21.15.31:0/3890843904 >> 172.21.15.31:0/3890843904 conn(0x7ff328079790 msgr2=0x7ff328079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:02.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 -- 172.21.15.31:0/3890843904 shutdown_connections 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.144+0000 7ff32ed71640 1 -- 172.21.15.31:0/3890843904 wait complete. 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.145+0000 7ff32ed71640 1 Processor -- start 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.145+0000 7ff32ed71640 1 -- start start 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32ed71640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 0x7ff32819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32ed71640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3281083d0 0x7ff32819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32ed71640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff328108d20 0x7ff32819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32ed71640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff328113b00 con 0x7ff328102ae0 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32ed71640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff328113980 con 0x7ff328108d20 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32ed71640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff328113c80 con 0x7ff3281083d0 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 0x7ff32819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 0x7ff32819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47260/0 (socket says 172.21.15.31:47260) 2024-09-21T02:52:02.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 -- 172.21.15.31:0/3538319378 learned_addr learned my addr 172.21.15.31:0/3538319378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff327fff640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3281083d0 0x7ff32819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32d30e640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff328108d20 0x7ff32819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 -- 172.21.15.31:0/3538319378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3281083d0 msgr2=0x7ff32819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3281083d0 0x7ff32819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 -- 172.21.15.31:0/3538319378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff328108d20 msgr2=0x7ff32819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff328108d20 0x7ff32819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 -- 172.21.15.31:0/3538319378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3281a02e0 con 0x7ff328102ae0 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32d30e640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff328108d20 0x7ff32819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff327fff640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3281083d0 0x7ff32819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.146+0000 7ff32cb0d640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 0x7ff32819a430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00d770 tx=0x7ff31c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.147+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c017070 con 0x7ff328102ae0 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.147+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff31c0054d0 con 0x7ff328102ae0 2024-09-21T02:52:02.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.147+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3281a05d0 con 0x7ff328102ae0 2024-09-21T02:52:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.147+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c004c30 con 0x7ff328102ae0 2024-09-21T02:52:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.147+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3281a6df0 con 0x7ff328102ae0 2024-09-21T02:52:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.148+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff328102ef0 con 0x7ff328102ae0 2024-09-21T02:52:02.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.150+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff31c004dd0 con 0x7ff328102ae0 2024-09-21T02:52:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.151+0000 7ff325ffb640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3180b6620 0x7ff3180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.151+0000 7ff327fff640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3180b6620 0x7ff3180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:02.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.151+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff31c0e7540 con 0x7ff328102ae0 2024-09-21T02:52:02.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.152+0000 7ff327fff640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3180b6620 0x7ff3180b8b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff328079d30 tx=0x7ff31400a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.155+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff31c0adb70 con 0x7ff328102ae0 2024-09-21T02:52:02.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.346+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff32819b560 con 0x7ff328102ae0 2024-09-21T02:52:02.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.347+0000 7ff325ffb640 1 -- 172.21.15.31:0/3538319378 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff31c0b1f50 con 0x7ff328102ae0 2024-09-21T02:52:02.349 INFO:teuthology.orchestra.run.smithi031.stdout:206158430255 2024-09-21T02:52:02.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3180b6620 msgr2=0x7ff3180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:02.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3180b6620 0x7ff3180b8b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff328079d30 tx=0x7ff31400a5c0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 msgr2=0x7ff32819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 0x7ff32819a430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00d770 tx=0x7ff31c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 shutdown_connections 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3180b6620 0x7ff3180b8b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3281083d0 0x7ff32819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff328108d20 0x7ff32819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 --2- 172.21.15.31:0/3538319378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff328102ae0 0x7ff32819a430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 >> 172.21.15.31:0/3538319378 conn(0x7ff328079790 msgr2=0x7ff328104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 shutdown_connections 2024-09-21T02:52:02.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:02.350+0000 7ff32ed71640 1 -- 172.21.15.31:0/3538319378 wait complete. 2024-09-21T02:52:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:02 smithi031 ceph-mon[31209]: pgmap v299: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-21T02:52:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:02 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3538319378' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:52:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:02 smithi178 ceph-mon[34759]: pgmap v299: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-21T02:52:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:02 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3538319378' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:52:02.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:02 smithi165 ceph-mon[34818]: pgmap v299: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-21T02:52:02.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:02 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3538319378' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:52:03.221 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234022 got 231928234025 for osd.8 2024-09-21T02:52:03.221 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:03.648 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710743 got 300647710746 for osd.11 2024-09-21T02:52:03.648 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:04.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874269 got 279172874271 for osd.10 2024-09-21T02:52:04.216 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:04 smithi178 ceph-mon[34759]: pgmap v300: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T02:52:04.779 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:04 smithi031 ceph-mon[31209]: pgmap v300: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T02:52:04.814 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705743 got 38654705746 for osd.0 2024-09-21T02:52:04.814 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:04.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:04 smithi165 ceph-mon[34818]: pgmap v300: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T02:52:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:05.694 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430252 got 206158430255 for osd.7 2024-09-21T02:52:05.695 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 -- 172.21.15.31:0/3382336124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1201094a0 msgr2=0x7ff1201098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 --2- 172.21.15.31:0/3382336124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1201094a0 0x7ff1201098b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff110009900 tx=0x7ff11002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 -- 172.21.15.31:0/3382336124 shutdown_connections 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 --2- 172.21.15.31:0/3382336124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff120109df0 0x7ff120110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 --2- 172.21.15.31:0/3382336124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1201094a0 0x7ff1201098b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 --2- 172.21.15.31:0/3382336124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120103b60 0x7ff120103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 -- 172.21.15.31:0/3382336124 >> 172.21.15.31:0/3382336124 conn(0x7ff12007b850 msgr2=0x7ff12007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 -- 172.21.15.31:0/3382336124 shutdown_connections 2024-09-21T02:52:06.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.143+0000 7ff126236640 1 -- 172.21.15.31:0/3382336124 wait complete. 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.144+0000 7ff126236640 1 Processor -- start 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.144+0000 7ff126236640 1 -- start start 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff126236640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 0x7ff12019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff126236640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1201094a0 0x7ff12019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff126236640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120109df0 0x7ff1201a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff126236640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff120076240 con 0x7ff120103b60 2024-09-21T02:52:06.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff126236640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1200760c0 con 0x7ff1201094a0 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff126236640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1200763c0 con 0x7ff120109df0 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 0x7ff12019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 0x7ff12019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47284/0 (socket says 172.21.15.31:47284) 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 -- 172.21.15.31:0/1507699484 learned_addr learned my addr 172.21.15.31:0/1507699484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11effd640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1201094a0 0x7ff12019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11ffff640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120109df0 0x7ff1201a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 -- 172.21.15.31:0/1507699484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120109df0 msgr2=0x7ff1201a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120109df0 0x7ff1201a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 -- 172.21.15.31:0/1507699484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1201094a0 msgr2=0x7ff12019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1201094a0 0x7ff12019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 -- 172.21.15.31:0/1507699484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1201a25a0 con 0x7ff120103b60 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.145+0000 7ff11f7fe640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 0x7ff12019c630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff11400d950 tx=0x7ff11400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.146+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114017070 con 0x7ff120103b60 2024-09-21T02:52:06.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.146+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1201a2890 con 0x7ff120103b60 2024-09-21T02:52:06.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.146+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201a8ff0 con 0x7ff120103b60 2024-09-21T02:52:06.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.146+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff114009c30 con 0x7ff120103b60 2024-09-21T02:52:06.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.146+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400f620 con 0x7ff120103b60 2024-09-21T02:52:06.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.147+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10c0052c0 con 0x7ff120103b60 2024-09-21T02:52:06.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.148+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff114010460 con 0x7ff120103b60 2024-09-21T02:52:06.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.149+0000 7ff11cff9640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1080b6670 0x7ff1080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.149+0000 7ff11effd640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1080b6670 0x7ff1080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.149+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff1140e6ff0 con 0x7ff120103b60 2024-09-21T02:52:06.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.150+0000 7ff11effd640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1080b6670 0x7ff1080b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff110009230 tx=0x7ff110031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:06.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.154+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff114009de0 con 0x7ff120103b60 2024-09-21T02:52:06.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.348+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff10c005880 con 0x7ff120103b60 2024-09-21T02:52:06.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.348+0000 7ff11cff9640 1 -- 172.21.15.31:0/1507699484 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff1140b19a0 con 0x7ff120103b60 2024-09-21T02:52:06.350 INFO:teuthology.orchestra.run.smithi031.stdout:81604378698 2024-09-21T02:52:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.350+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1080b6670 msgr2=0x7ff1080b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.350+0000 7ff126236640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1080b6670 0x7ff1080b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff110009230 tx=0x7ff110031400 comp rx=0 tx=0).stop 2024-09-21T02:52:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.350+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 msgr2=0x7ff12019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.350+0000 7ff126236640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 0x7ff12019c630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff11400d950 tx=0x7ff11400de10 comp rx=0 tx=0).stop 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 shutdown_connections 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1080b6670 0x7ff1080b8b80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120109df0 0x7ff1201a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1201094a0 0x7ff12019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 --2- 172.21.15.31:0/1507699484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff120103b60 0x7ff12019c630 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 >> 172.21.15.31:0/1507699484 conn(0x7ff12007b850 msgr2=0x7ff120102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 shutdown_connections 2024-09-21T02:52:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.351+0000 7ff126236640 1 -- 172.21.15.31:0/1507699484 wait complete. 2024-09-21T02:52:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:06 smithi031 ceph-mon[31209]: pgmap v301: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:06.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:06 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1507699484' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:52:06.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:06 smithi178 ceph-mon[34759]: pgmap v301: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:06.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:06 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1507699484' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 -- 172.21.15.31:0/806339546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d2010de20 msgr2=0x7f5d20110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 --2- 172.21.15.31:0/806339546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d2010de20 0x7f5d20110300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009900 tx=0x7f5d1802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 -- 172.21.15.31:0/806339546 shutdown_connections 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 --2- 172.21.15.31:0/806339546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d2010de20 0x7f5d20110300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 --2- 172.21.15.31:0/806339546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d201052a0 0x7f5d2010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 --2- 172.21.15.31:0/806339546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d20104870 0x7f5d20104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 -- 172.21.15.31:0/806339546 >> 172.21.15.31:0/806339546 conn(0x7f5d20079730 msgr2=0x7f5d20079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 -- 172.21.15.31:0/806339546 shutdown_connections 2024-09-21T02:52:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.786+0000 7f5d272e3640 1 -- 172.21.15.31:0/806339546 wait complete. 2024-09-21T02:52:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 Processor -- start 2024-09-21T02:52:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 -- start start 2024-09-21T02:52:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 0x7f5d2019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d201052a0 0x7f5d2019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d2010de20 0x7f5d2019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113a90 con 0x7f5d20104870 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113910 con 0x7f5d201052a0 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d272e3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113c10 con 0x7f5d2010de20 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d2507f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 0x7f5d2019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d2507f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 0x7f5d2019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47300/0 (socket says 172.21.15.31:47300) 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d2507f640 1 -- 172.21.15.31:0/1928168870 learned_addr learned my addr 172.21.15.31:0/1928168870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d2487e640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d201052a0 0x7f5d2019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.787+0000 7f5d25880640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d2010de20 0x7f5d2019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d2507f640 1 -- 172.21.15.31:0/1928168870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d2010de20 msgr2=0x7f5d2019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d2507f640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d2010de20 0x7f5d2019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d2507f640 1 -- 172.21.15.31:0/1928168870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d201052a0 msgr2=0x7f5d2019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d2507f640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d201052a0 0x7f5d2019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d2507f640 1 -- 172.21.15.31:0/1928168870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d201a0370 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d2507f640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 0x7f5d2019a400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00ea80 tx=0x7f5d1c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00ce40 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201a0660 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c005020 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0116d0 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.788+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201a6dc0 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.789+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d040052c0 con 0x7f5d20104870 2024-09-21T02:52:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.791+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5d1c004a40 con 0x7f5d20104870 2024-09-21T02:52:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.791+0000 7f5d167fc640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d0c0b65f0 0x7f5d0c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.792+0000 7f5d2487e640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d0c0b65f0 0x7f5d0c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.792+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5d1c0e7a20 con 0x7f5d20104870 2024-09-21T02:52:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.792+0000 7f5d2487e640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d0c0b65f0 0x7f5d0c0b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d20102280 tx=0x7f5d1000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.795+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d1c016030 con 0x7f5d20104870 2024-09-21T02:52:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:06 smithi165 ceph-mon[34818]: pgmap v301: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:06 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1507699484' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:52:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.990+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5d04005880 con 0x7f5d20104870 2024-09-21T02:52:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.991+0000 7f5d167fc640 1 -- 172.21.15.31:0/1928168870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d1c0ae050 con 0x7f5d20104870 2024-09-21T02:52:06.992 INFO:teuthology.orchestra.run.smithi031.stdout:257698037798 2024-09-21T02:52:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d0c0b65f0 msgr2=0x7f5d0c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d0c0b65f0 0x7f5d0c0b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d20102280 tx=0x7f5d1000b390 comp rx=0 tx=0).stop 2024-09-21T02:52:06.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 msgr2=0x7f5d2019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 0x7f5d2019a400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00ea80 tx=0x7f5d1c00ef40 comp rx=0 tx=0).stop 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 shutdown_connections 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d0c0b65f0 0x7f5d0c0b8b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d2010de20 0x7f5d2019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d201052a0 0x7f5d2019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 --2- 172.21.15.31:0/1928168870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d20104870 0x7f5d2019a400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 >> 172.21.15.31:0/1928168870 conn(0x7f5d20079730 msgr2=0x7f5d201007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 shutdown_connections 2024-09-21T02:52:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:06.993+0000 7f5d272e3640 1 -- 172.21.15.31:0/1928168870 wait complete. 2024-09-21T02:52:07.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378693 got 81604378698 for osd.2 2024-09-21T02:52:07.270 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:07.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:07 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1928168870' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:52:07.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:07 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1928168870' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:52:07.940 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037795 got 257698037798 for osd.9 2024-09-21T02:52:07.940 DEBUG:teuthology.parallel:result is None 2024-09-21T02:52:07.940 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T02:52:07.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:52:07.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:07 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1928168870' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:52:08.222 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:08.672 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:08 smithi031 ceph-mon[31209]: pgmap v302: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:08.672 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:08 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:08.673 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:08 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:08 smithi178 ceph-mon[34759]: pgmap v302: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:08 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:08 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 -- 172.21.15.31:0/446628098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c108cf0 msgr2=0x7fe39c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/446628098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c108cf0 0x7fe39c10f520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe394009ca0 tx=0x7fe39402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 -- 172.21.15.31:0/446628098 shutdown_connections 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/446628098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c108cf0 0x7fe39c10f520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/446628098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c1083a0 0x7fe39c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/446628098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c102a60 0x7fe39c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 -- 172.21.15.31:0/446628098 >> 172.21.15.31:0/446628098 conn(0x7fe39c079730 msgr2=0x7fe39c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 -- 172.21.15.31:0/446628098 shutdown_connections 2024-09-21T02:52:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.764+0000 7fe3a1f79640 1 -- 172.21.15.31:0/446628098 wait complete. 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.765+0000 7fe3a1f79640 1 Processor -- start 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.765+0000 7fe3a1f79640 1 -- start start 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe3a1f79640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c102a60 0x7fe39c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe3a1f79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 0x7fe39c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe3a1f79640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c108cf0 0x7fe39c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe3a1f79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c113a80 con 0x7fe39c1083a0 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe3a1f79640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c113900 con 0x7fe39c108cf0 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe3a1f79640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c113c00 con 0x7fe39c102a60 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 0x7fe39c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 0x7fe39c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47324/0 (socket says 172.21.15.31:47324) 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 -- 172.21.15.31:0/2962312468 learned_addr learned my addr 172.21.15.31:0/2962312468 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:08.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39bfff640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c108cf0 0x7fe39c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39b7fe640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c102a60 0x7fe39c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 -- 172.21.15.31:0/2962312468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c102a60 msgr2=0x7fe39c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c102a60 0x7fe39c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 -- 172.21.15.31:0/2962312468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c108cf0 msgr2=0x7fe39c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c108cf0 0x7fe39c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 -- 172.21.15.31:0/2962312468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c1a0340 con 0x7fe39c1083a0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39bfff640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c108cf0 0x7fe39c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.766+0000 7fe39affd640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 0x7fe39c19a970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d770 tx=0x7fe39000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.767+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390017070 con 0x7fe39c1083a0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.767+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe390009c30 con 0x7fe39c1083a0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.767+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c1a0630 con 0x7fe39c1083a0 2024-09-21T02:52:08.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.767+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000f620 con 0x7fe39c1083a0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.767+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a6d90 con 0x7fe39c1083a0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.768+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c102e70 con 0x7fe39c1083a0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.770+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe39001a070 con 0x7fe39c1083a0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.770+0000 7fe398ff9640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3840b65f0 0x7fe3840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.770+0000 7fe39b7fe640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3840b65f0 0x7fe3840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.770+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe3900e7190 con 0x7fe39c1083a0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.771+0000 7fe39b7fe640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3840b65f0 0x7fe3840b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe38c005fd0 tx=0x7fe38c005e20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:08.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.773+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe390009de0 con 0x7fe39c1083a0 2024-09-21T02:52:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:08 smithi165 ceph-mon[34818]: pgmap v302: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:08 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:08 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:08.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.957+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe39c063800 con 0x7fe3840b65f0 2024-09-21T02:52:08.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.962+0000 7fe398ff9640 1 -- 172.21.15.31:0/2962312468 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259057 (secure 0 0 0) 0x7fe39c063800 con 0x7fe3840b65f0 2024-09-21T02:52:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:08.966 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3840b65f0 msgr2=0x7fe3840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3840b65f0 0x7fe3840b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe38c005fd0 tx=0x7fe38c005e20 comp rx=0 tx=0).stop 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 msgr2=0x7fe39c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 0x7fe39c19a970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d770 tx=0x7fe39000dc30 comp rx=0 tx=0).stop 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 shutdown_connections 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3840b65f0 0x7fe3840b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe39c108cf0 0x7fe39c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe39c1083a0 0x7fe39c19a970 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.966+0000 7fe3a1f79640 1 --2- 172.21.15.31:0/2962312468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c102a60 0x7fe39c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:08.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.967+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 >> 172.21.15.31:0/2962312468 conn(0x7fe39c079730 msgr2=0x7fe39c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:08.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.967+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 shutdown_connections 2024-09-21T02:52:08.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:08.967+0000 7fe3a1f79640 1 -- 172.21.15.31:0/2962312468 wait complete. 2024-09-21T02:52:09.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:09.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:09.653 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:09.688 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-21T02:52:07.159650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661546,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":760,"num_read_kb":491,"num_write":365,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376492,"kb_used_data":10560,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565332,"statfs":{"total":1151940427776,"available":1151554899968,"internally_reserved":0,"allocated":10813440,"data_stored":7018602,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574159+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:51:16.574159+0000","last_peered":"2024-09-21T02:51:16.574159+0000","last_clean":"2024-09-21T02:51:16.574159+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:51:16.574159+0000","last_undegraded":"2024-09-21T02:51:16.574159+0000","last_fullsized":"2024-09-21T02:51:16.574159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576740+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:51:16.576740+0000","last_peered":"2024-09-21T02:51:16.576740+0000","last_clean":"2024-09-21T02:51:16.576740+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:51:16.576740+0000","last_undegraded":"2024-09-21T02:51:16.576740+0000","last_fullsized":"2024-09-21T02:51:16.576740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576913+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:51:16.576913+0000","last_peered":"2024-09-21T02:51:16.576913+0000","last_clean":"2024-09-21T02:51:16.576913+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:51:16.576913+0000","last_undegraded":"2024-09-21T02:51:16.576913+0000","last_fullsized":"2024-09-21T02:51:16.576913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573194+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:51:16.573194+0000","last_peered":"2024-09-21T02:51:16.573194+0000","last_clean":"2024-09-21T02:51:16.573194+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:51:16.573194+0000","last_undegraded":"2024-09-21T02:51:16.573194+0000","last_fullsized":"2024-09-21T02:51:16.573194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572183+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:51:16.572183+0000","last_peered":"2024-09-21T02:51:16.572183+0000","last_clean":"2024-09-21T02:51:16.572183+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:51:16.572183+0000","last_undegraded":"2024-09-21T02:51:16.572183+0000","last_fullsized":"2024-09-21T02:51:16.572183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578336+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:51:16.578336+0000","last_peered":"2024-09-21T02:51:16.578336+0000","last_clean":"2024-09-21T02:51:16.578336+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:51:16.578336+0000","last_undegraded":"2024-09-21T02:51:16.578336+0000","last_fullsized":"2024-09-21T02:51:16.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576570+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:51:16.576570+0000","last_peered":"2024-09-21T02:51:16.576570+0000","last_clean":"2024-09-21T02:51:16.576570+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:51:16.576570+0000","last_undegraded":"2024-09-21T02:51:16.576570+0000","last_fullsized":"2024-09-21T02:51:16.576570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976088+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:51:16.976088+0000","last_peered":"2024-09-21T02:51:16.976088+0000","last_clean":"2024-09-21T02:51:16.976088+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:51:16.976088+0000","last_undegraded":"2024-09-21T02:51:16.976088+0000","last_fullsized":"2024-09-21T02:51:16.976088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977495+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:51:16.977495+0000","last_peered":"2024-09-21T02:51:16.977495+0000","last_clean":"2024-09-21T02:51:16.977495+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:51:16.977495+0000","last_undegraded":"2024-09-21T02:51:16.977495+0000","last_fullsized":"2024-09-21T02:51:16.977495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574699+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:51:16.574699+0000","last_peered":"2024-09-21T02:51:16.574699+0000","last_clean":"2024-09-21T02:51:16.574699+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:51:16.574699+0000","last_undegraded":"2024-09-21T02:51:16.574699+0000","last_fullsized":"2024-09-21T02:51:16.574699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576637+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:51:16.576637+0000","last_peered":"2024-09-21T02:51:16.576637+0000","last_clean":"2024-09-21T02:51:16.576637+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:51:16.576637+0000","last_undegraded":"2024-09-21T02:51:16.576637+0000","last_fullsized":"2024-09-21T02:51:16.576637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576458+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:51:16.576458+0000","last_peered":"2024-09-21T02:51:16.576458+0000","last_clean":"2024-09-21T02:51:16.576458+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:51:16.576458+0000","last_undegraded":"2024-09-21T02:51:16.576458+0000","last_fullsized":"2024-09-21T02:51:16.576458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573699+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:16.573699+0000","last_peered":"2024-09-21T02:51:16.573699+0000","last_clean":"2024-09-21T02:51:16.573699+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:16.573699+0000","last_undegraded":"2024-09-21T02:51:16.573699+0000","last_fullsized":"2024-09-21T02:51:16.573699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576623+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:51:16.576623+0000","last_peered":"2024-09-21T02:51:16.576623+0000","last_clean":"2024-09-21T02:51:16.576623+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:51:16.576623+0000","last_undegraded":"2024-09-21T02:51:16.576623+0000","last_fullsized":"2024-09-21T02:51:16.576623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576412+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:51:16.576412+0000","last_peered":"2024-09-21T02:51:16.576412+0000","last_clean":"2024-09-21T02:51:16.576412+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:51:16.576412+0000","last_undegraded":"2024-09-21T02:51:16.576412+0000","last_fullsized":"2024-09-21T02:51:16.576412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976749+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:51:16.976749+0000","last_peered":"2024-09-21T02:51:16.976749+0000","last_clean":"2024-09-21T02:51:16.976749+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:51:16.976749+0000","last_undegraded":"2024-09-21T02:51:16.976749+0000","last_fullsized":"2024-09-21T02:51:16.976749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574274+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:51:16.574274+0000","last_peered":"2024-09-21T02:51:16.574274+0000","last_clean":"2024-09-21T02:51:16.574274+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:51:16.574274+0000","last_undegraded":"2024-09-21T02:51:16.574274+0000","last_fullsized":"2024-09-21T02:51:16.574274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571557+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:16.571557+0000","last_peered":"2024-09-21T02:51:16.571557+0000","last_clean":"2024-09-21T02:51:16.571557+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:16.571557+0000","last_undegraded":"2024-09-21T02:51:16.571557+0000","last_fullsized":"2024-09-21T02:51:16.571557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575827+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:51:16.575827+0000","last_peered":"2024-09-21T02:51:16.575827+0000","last_clean":"2024-09-21T02:51:16.575827+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:51:16.575827+0000","last_undegraded":"2024-09-21T02:51:16.575827+0000","last_fullsized":"2024-09-21T02:51:16.575827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573836+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:51:16.573836+0000","last_peered":"2024-09-21T02:51:16.573836+0000","last_clean":"2024-09-21T02:51:16.573836+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:51:16.573836+0000","last_undegraded":"2024-09-21T02:51:16.573836+0000","last_fullsized":"2024-09-21T02:51:16.573836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573770+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:16.573770+0000","last_peered":"2024-09-21T02:51:16.573770+0000","last_clean":"2024-09-21T02:51:16.573770+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:16.573770+0000","last_undegraded":"2024-09-21T02:51:16.573770+0000","last_fullsized":"2024-09-21T02:51:16.573770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976953+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:51:16.976953+0000","last_peered":"2024-09-21T02:51:16.976953+0000","last_clean":"2024-09-21T02:51:16.976953+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:51:16.976953+0000","last_undegraded":"2024-09-21T02:51:16.976953+0000","last_fullsized":"2024-09-21T02:51:16.976953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975815+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:51:16.975815+0000","last_peered":"2024-09-21T02:51:16.975815+0000","last_clean":"2024-09-21T02:51:16.975815+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:51:16.975815+0000","last_undegraded":"2024-09-21T02:51:16.975815+0000","last_fullsized":"2024-09-21T02:51:16.975815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977399+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:51:16.977399+0000","last_peered":"2024-09-21T02:51:16.977399+0000","last_clean":"2024-09-21T02:51:16.977399+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:51:16.977399+0000","last_undegraded":"2024-09-21T02:51:16.977399+0000","last_fullsized":"2024-09-21T02:51:16.977399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976482+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:51:16.976482+0000","last_peered":"2024-09-21T02:51:16.976482+0000","last_clean":"2024-09-21T02:51:16.976482+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:51:16.976482+0000","last_undegraded":"2024-09-21T02:51:16.976482+0000","last_fullsized":"2024-09-21T02:51:16.976482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576864+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:51:16.576864+0000","last_peered":"2024-09-21T02:51:16.576864+0000","last_clean":"2024-09-21T02:51:16.576864+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:51:16.576864+0000","last_undegraded":"2024-09-21T02:51:16.576864+0000","last_fullsized":"2024-09-21T02:51:16.576864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575552+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:51:16.575552+0000","last_peered":"2024-09-21T02:51:16.575552+0000","last_clean":"2024-09-21T02:51:16.575552+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:51:16.575552+0000","last_undegraded":"2024-09-21T02:51:16.575552+0000","last_fullsized":"2024-09-21T02:51:16.575552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574009+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:16.574009+0000","last_peered":"2024-09-21T02:51:16.574009+0000","last_clean":"2024-09-21T02:51:16.574009+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:16.574009+0000","last_undegraded":"2024-09-21T02:51:16.574009+0000","last_fullsized":"2024-09-21T02:51:16.574009+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580839+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:51:16.580839+0000","last_peered":"2024-09-21T02:51:16.580839+0000","last_clean":"2024-09-21T02:51:16.580839+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:51:16.580839+0000","last_undegraded":"2024-09-21T02:51:16.580839+0000","last_fullsized":"2024-09-21T02:51:16.580839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575696+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:51:16.575696+0000","last_peered":"2024-09-21T02:51:16.575696+0000","last_clean":"2024-09-21T02:51:16.575696+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:51:16.575696+0000","last_undegraded":"2024-09-21T02:51:16.575696+0000","last_fullsized":"2024-09-21T02:51:16.575696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576084+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:51:16.576084+0000","last_peered":"2024-09-21T02:51:16.576084+0000","last_clean":"2024-09-21T02:51:16.576084+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:51:16.576084+0000","last_undegraded":"2024-09-21T02:51:16.576084+0000","last_fullsized":"2024-09-21T02:51:16.576084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574101+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:51:16.574101+0000","last_peered":"2024-09-21T02:51:16.574101+0000","last_clean":"2024-09-21T02:51:16.574101+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:51:16.574101+0000","last_undegraded":"2024-09-21T02:51:16.574101+0000","last_fullsized":"2024-09-21T02:51:16.574101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572143+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:51:16.572143+0000","last_peered":"2024-09-21T02:51:16.572143+0000","last_clean":"2024-09-21T02:51:16.572143+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:51:16.572143+0000","last_undegraded":"2024-09-21T02:51:16.572143+0000","last_fullsized":"2024-09-21T02:51:16.572143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":108,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:02.648658+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:52:02.648658+0000","last_peered":"2024-09-21T02:52:02.648658+0000","last_clean":"2024-09-21T02:52:02.648658+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:52:02.648658+0000","last_undegraded":"2024-09-21T02:52:02.648658+0000","last_fullsized":"2024-09-21T02:52:02.648658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578401+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:51:16.578401+0000","last_peered":"2024-09-21T02:51:16.578401+0000","last_clean":"2024-09-21T02:51:16.578401+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:51:16.578401+0000","last_undegraded":"2024-09-21T02:51:16.578401+0000","last_fullsized":"2024-09-21T02:51:16.578401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976219+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:51:16.976219+0000","last_peered":"2024-09-21T02:51:16.976219+0000","last_clean":"2024-09-21T02:51:16.976219+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:51:16.976219+0000","last_undegraded":"2024-09-21T02:51:16.976219+0000","last_fullsized":"2024-09-21T02:51:16.976219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571256+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:16.571256+0000","last_peered":"2024-09-21T02:51:16.571256+0000","last_clean":"2024-09-21T02:51:16.571256+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:16.571256+0000","last_undegraded":"2024-09-21T02:51:16.571256+0000","last_fullsized":"2024-09-21T02:51:16.571256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":159,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:06.626167+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:52:06.626167+0000","last_peered":"2024-09-21T02:52:06.626167+0000","last_clean":"2024-09-21T02:52:06.626167+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:52:06.626167+0000","last_undegraded":"2024-09-21T02:52:06.626167+0000","last_fullsized":"2024-09-21T02:52:06.626167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977390+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:51:16.977390+0000","last_peered":"2024-09-21T02:51:16.977390+0000","last_clean":"2024-09-21T02:51:16.977390+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:51:16.977390+0000","last_undegraded":"2024-09-21T02:51:16.977390+0000","last_fullsized":"2024-09-21T02:51:16.977390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577020+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:51:16.577020+0000","last_peered":"2024-09-21T02:51:16.577020+0000","last_clean":"2024-09-21T02:51:16.577020+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:51:16.577020+0000","last_undegraded":"2024-09-21T02:51:16.577020+0000","last_fullsized":"2024-09-21T02:51:16.577020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977117+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:51:16.977117+0000","last_peered":"2024-09-21T02:51:16.977117+0000","last_clean":"2024-09-21T02:51:16.977117+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:51:16.977117+0000","last_undegraded":"2024-09-21T02:51:16.977117+0000","last_fullsized":"2024-09-21T02:51:16.977117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":190,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:06.654332+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:52:06.654332+0000","last_peered":"2024-09-21T02:52:06.654332+0000","last_clean":"2024-09-21T02:52:06.654332+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:52:06.654332+0000","last_undegraded":"2024-09-21T02:52:06.654332+0000","last_fullsized":"2024-09-21T02:52:06.654332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63349,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":130,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572844+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:51:16.572844+0000","last_peered":"2024-09-21T02:51:16.572844+0000","last_clean":"2024-09-21T02:51:16.572844+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:51:16.572844+0000","last_undegraded":"2024-09-21T02:51:16.572844+0000","last_fullsized":"2024-09-21T02:51:16.572844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578577+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:51:16.578577+0000","last_peered":"2024-09-21T02:51:16.578577+0000","last_clean":"2024-09-21T02:51:16.578577+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:51:16.578577+0000","last_undegraded":"2024-09-21T02:51:16.578577+0000","last_fullsized":"2024-09-21T02:51:16.578577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576983+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:51:16.576983+0000","last_peered":"2024-09-21T02:51:16.576983+0000","last_clean":"2024-09-21T02:51:16.576983+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:51:16.576983+0000","last_undegraded":"2024-09-21T02:51:16.576983+0000","last_fullsized":"2024-09-21T02:51:16.576983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:02.620237+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:52:02.620237+0000","last_peered":"2024-09-21T02:52:02.620237+0000","last_clean":"2024-09-21T02:52:02.620237+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:52:02.620237+0000","last_undegraded":"2024-09-21T02:52:02.620237+0000","last_fullsized":"2024-09-21T02:52:02.620237+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580840+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:51:16.580840+0000","last_peered":"2024-09-21T02:51:16.580840+0000","last_clean":"2024-09-21T02:51:16.580840+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:51:16.580840+0000","last_undegraded":"2024-09-21T02:51:16.580840+0000","last_fullsized":"2024-09-21T02:51:16.580840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577762+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:51:16.577762+0000","last_peered":"2024-09-21T02:51:16.577762+0000","last_clean":"2024-09-21T02:51:16.577762+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:51:16.577762+0000","last_undegraded":"2024-09-21T02:51:16.577762+0000","last_fullsized":"2024-09-21T02:51:16.577762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573938+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:51:16.573938+0000","last_peered":"2024-09-21T02:51:16.573938+0000","last_clean":"2024-09-21T02:51:16.573938+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:51:16.573938+0000","last_undegraded":"2024-09-21T02:51:16.573938+0000","last_fullsized":"2024-09-21T02:51:16.573938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576883+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:51:16.576883+0000","last_peered":"2024-09-21T02:51:16.576883+0000","last_clean":"2024-09-21T02:51:16.576883+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:51:16.576883+0000","last_undegraded":"2024-09-21T02:51:16.576883+0000","last_fullsized":"2024-09-21T02:51:16.576883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576759+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:51:16.576759+0000","last_peered":"2024-09-21T02:51:16.576759+0000","last_clean":"2024-09-21T02:51:16.576759+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:51:16.576759+0000","last_undegraded":"2024-09-21T02:51:16.576759+0000","last_fullsized":"2024-09-21T02:51:16.576759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:02.620181+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:52:02.620181+0000","last_peered":"2024-09-21T02:52:02.620181+0000","last_clean":"2024-09-21T02:52:02.620181+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:52:02.620181+0000","last_undegraded":"2024-09-21T02:52:02.620181+0000","last_fullsized":"2024-09-21T02:52:02.620181+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576701+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:51:16.576701+0000","last_peered":"2024-09-21T02:51:16.576701+0000","last_clean":"2024-09-21T02:51:16.576701+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:51:16.576701+0000","last_undegraded":"2024-09-21T02:51:16.576701+0000","last_fullsized":"2024-09-21T02:51:16.576701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573447+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573447+0000","last_peered":"2024-09-21T02:51:16.573447+0000","last_clean":"2024-09-21T02:51:16.573447+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573447+0000","last_undegraded":"2024-09-21T02:51:16.573447+0000","last_fullsized":"2024-09-21T02:51:16.573447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976061+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:51:16.976061+0000","last_peered":"2024-09-21T02:51:16.976061+0000","last_clean":"2024-09-21T02:51:16.976061+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:51:16.976061+0000","last_undegraded":"2024-09-21T02:51:16.976061+0000","last_fullsized":"2024-09-21T02:51:16.976061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576775+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576775+0000","last_peered":"2024-09-21T02:51:16.576775+0000","last_clean":"2024-09-21T02:51:16.576775+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576775+0000","last_undegraded":"2024-09-21T02:51:16.576775+0000","last_fullsized":"2024-09-21T02:51:16.576775+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573136+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:16.573136+0000","last_peered":"2024-09-21T02:51:16.573136+0000","last_clean":"2024-09-21T02:51:16.573136+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:16.573136+0000","last_undegraded":"2024-09-21T02:51:16.573136+0000","last_fullsized":"2024-09-21T02:51:16.573136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:04.622912+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:52:04.622912+0000","last_peered":"2024-09-21T02:52:04.622912+0000","last_clean":"2024-09-21T02:52:04.622912+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:52:04.622912+0000","last_undegraded":"2024-09-21T02:52:04.622912+0000","last_fullsized":"2024-09-21T02:52:04.622912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578540+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:51:16.578540+0000","last_peered":"2024-09-21T02:51:16.578540+0000","last_clean":"2024-09-21T02:51:16.578540+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:51:16.578540+0000","last_undegraded":"2024-09-21T02:51:16.578540+0000","last_fullsized":"2024-09-21T02:51:16.578540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:02.619667+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:52:02.619667+0000","last_peered":"2024-09-21T02:52:02.619667+0000","last_clean":"2024-09-21T02:52:02.619667+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:52:02.619667+0000","last_undegraded":"2024-09-21T02:52:02.619667+0000","last_fullsized":"2024-09-21T02:52:02.619667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574384+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:51:16.574384+0000","last_peered":"2024-09-21T02:51:16.574384+0000","last_clean":"2024-09-21T02:51:16.574384+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:51:16.574384+0000","last_undegraded":"2024-09-21T02:51:16.574384+0000","last_fullsized":"2024-09-21T02:51:16.574384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571552+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571552+0000","last_peered":"2024-09-21T02:51:16.571552+0000","last_clean":"2024-09-21T02:51:16.571552+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571552+0000","last_undegraded":"2024-09-21T02:51:16.571552+0000","last_fullsized":"2024-09-21T02:51:16.571552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574339+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:51:16.574339+0000","last_peered":"2024-09-21T02:51:16.574339+0000","last_clean":"2024-09-21T02:51:16.574339+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:51:16.574339+0000","last_undegraded":"2024-09-21T02:51:16.574339+0000","last_fullsized":"2024-09-21T02:51:16.574339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574116+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:16.574116+0000","last_peered":"2024-09-21T02:51:16.574116+0000","last_clean":"2024-09-21T02:51:16.574116+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:16.574116+0000","last_undegraded":"2024-09-21T02:51:16.574116+0000","last_fullsized":"2024-09-21T02:51:16.574116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571740+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:16.571740+0000","last_peered":"2024-09-21T02:51:16.571740+0000","last_clean":"2024-09-21T02:51:16.571740+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:16.571740+0000","last_undegraded":"2024-09-21T02:51:16.571740+0000","last_fullsized":"2024-09-21T02:51:16.571740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976502+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:51:16.976502+0000","last_peered":"2024-09-21T02:51:16.976502+0000","last_clean":"2024-09-21T02:51:16.976502+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:51:16.976502+0000","last_undegraded":"2024-09-21T02:51:16.976502+0000","last_fullsized":"2024-09-21T02:51:16.976502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573953+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.573953+0000","last_peered":"2024-09-21T02:51:16.573953+0000","last_clean":"2024-09-21T02:51:16.573953+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.573953+0000","last_undegraded":"2024-09-21T02:51:16.573953+0000","last_fullsized":"2024-09-21T02:51:16.573953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976881+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976881+0000","last_peered":"2024-09-21T02:51:16.976881+0000","last_clean":"2024-09-21T02:51:16.976881+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976881+0000","last_undegraded":"2024-09-21T02:51:16.976881+0000","last_fullsized":"2024-09-21T02:51:16.976881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571934+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:51:16.571934+0000","last_peered":"2024-09-21T02:51:16.571934+0000","last_clean":"2024-09-21T02:51:16.571934+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:51:16.571934+0000","last_undegraded":"2024-09-21T02:51:16.571934+0000","last_fullsized":"2024-09-21T02:51:16.571934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975953+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.975953+0000","last_peered":"2024-09-21T02:51:16.975953+0000","last_clean":"2024-09-21T02:51:16.975953+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.975953+0000","last_undegraded":"2024-09-21T02:51:16.975953+0000","last_fullsized":"2024-09-21T02:51:16.975953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573705+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:51:16.573705+0000","last_peered":"2024-09-21T02:51:16.573705+0000","last_clean":"2024-09-21T02:51:16.573705+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:51:16.573705+0000","last_undegraded":"2024-09-21T02:51:16.573705+0000","last_fullsized":"2024-09-21T02:51:16.573705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575838+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:16.575838+0000","last_peered":"2024-09-21T02:51:16.575838+0000","last_clean":"2024-09-21T02:51:16.575838+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:16.575838+0000","last_undegraded":"2024-09-21T02:51:16.575838+0000","last_fullsized":"2024-09-21T02:51:16.575838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:51:16.976034+0000","last_peered":"2024-09-21T02:51:16.976034+0000","last_clean":"2024-09-21T02:51:16.976034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:51:16.976034+0000","last_undegraded":"2024-09-21T02:51:16.976034+0000","last_fullsized":"2024-09-21T02:51:16.976034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576863+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:51:16.576863+0000","last_peered":"2024-09-21T02:51:16.576863+0000","last_clean":"2024-09-21T02:51:16.576863+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:51:16.576863+0000","last_undegraded":"2024-09-21T02:51:16.576863+0000","last_fullsized":"2024-09-21T02:51:16.576863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578106+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:51:16.578106+0000","last_peered":"2024-09-21T02:51:16.578106+0000","last_clean":"2024-09-21T02:51:16.578106+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:51:16.578106+0000","last_undegraded":"2024-09-21T02:51:16.578106+0000","last_fullsized":"2024-09-21T02:51:16.578106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977018+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:51:16.977018+0000","last_peered":"2024-09-21T02:51:16.977018+0000","last_clean":"2024-09-21T02:51:16.977018+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:51:16.977018+0000","last_undegraded":"2024-09-21T02:51:16.977018+0000","last_fullsized":"2024-09-21T02:51:16.977018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577070+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:51:16.577070+0000","last_peered":"2024-09-21T02:51:16.577070+0000","last_clean":"2024-09-21T02:51:16.577070+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:51:16.577070+0000","last_undegraded":"2024-09-21T02:51:16.577070+0000","last_fullsized":"2024-09-21T02:51:16.577070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976572+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976572+0000","last_peered":"2024-09-21T02:51:16.976572+0000","last_clean":"2024-09-21T02:51:16.976572+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976572+0000","last_undegraded":"2024-09-21T02:51:16.976572+0000","last_fullsized":"2024-09-21T02:51:16.976572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976425+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:51:16.976425+0000","last_peered":"2024-09-21T02:51:16.976425+0000","last_clean":"2024-09-21T02:51:16.976425+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:51:16.976425+0000","last_undegraded":"2024-09-21T02:51:16.976425+0000","last_fullsized":"2024-09-21T02:51:16.976425+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":102,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:04.622201+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:52:04.622201+0000","last_peered":"2024-09-21T02:52:04.622201+0000","last_clean":"2024-09-21T02:52:04.622201+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:52:04.622201+0000","last_undegraded":"2024-09-21T02:52:04.622201+0000","last_fullsized":"2024-09-21T02:52:04.622201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70944,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":679,"num_read_kb":422,"num_write":240,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":212832,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":860160,"data_stored":421192,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075067,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":1013906,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":860160,"data_stored":420727,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013193,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":421210,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":484587,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":484989,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":422066,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65492,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65090,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65085,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:52:09.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:52:09.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:09.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:09.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:09.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:09.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:09.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:52:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 -- 172.21.15.31:0/295625299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f8103bb0 msgr2=0x7fa2f8103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:10.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/295625299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f8103bb0 0x7fa2f8103fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009900 tx=0x7fa2ec02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:10.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 -- 172.21.15.31:0/295625299 shutdown_connections 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/295625299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2f81094f0 0x7fa2f8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/295625299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f8109e40 0x7fa2f8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/295625299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f8103bb0 0x7fa2f8103fc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 -- 172.21.15.31:0/295625299 >> 172.21.15.31:0/295625299 conn(0x7fa2f80ff600 msgr2=0x7fa2f8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 -- 172.21.15.31:0/295625299 shutdown_connections 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.505+0000 7fa2fef9a640 1 -- 172.21.15.31:0/295625299 wait complete. 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 Processor -- start 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 -- start start 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2f8103bb0 0x7fa2f819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 0x7fa2f819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f8109e40 0x7fa2f819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8113b00 con 0x7fa2f81094f0 2024-09-21T02:52:10.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8113980 con 0x7fa2f8109e40 2024-09-21T02:52:10.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fef9a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8113c80 con 0x7fa2f8103bb0 2024-09-21T02:52:10.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 0x7fa2f819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:10.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 0x7fa2f819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49818/0 (socket says 172.21.15.31:49818) 2024-09-21T02:52:10.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 -- 172.21.15.31:0/1120710354 learned_addr learned my addr 172.21.15.31:0/1120710354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:10.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fd537640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f8109e40 0x7fa2f819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:10.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2fcd36640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2f8103bb0 0x7fa2f819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 -- 172.21.15.31:0/1120710354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2f8103bb0 msgr2=0x7fa2f819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2f8103bb0 0x7fa2f819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 -- 172.21.15.31:0/1120710354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f8109e40 msgr2=0x7fa2f819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f8109e40 0x7fa2f819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.506+0000 7fa2f7fff640 1 -- 172.21.15.31:0/1120710354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f81a0440 con 0x7fa2f81094f0 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.507+0000 7fa2f7fff640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 0x7fa2f819aa70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400c970 tx=0x7fa2e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:10.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.507+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4004220 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.507+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e40043c0 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.507+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f81a0730 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.507+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81a6e90 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.507+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4004a00 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.508+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f8103fc0 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.510+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa2e40079b0 con 0x7fa2f81094f0 2024-09-21T02:52:10.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.510+0000 7fa2f5ffb640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa2e80b6670 0x7fa2e80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:10.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.510+0000 7fa2fcd36640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa2e80b6670 0x7fa2e80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:10.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.510+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa2e40e7250 con 0x7fa2f81094f0 2024-09-21T02:52:10.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.511+0000 7fa2fcd36640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa2e80b6670 0x7fa2e80b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec031440 tx=0x7fa2ec009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:10.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.515+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2e4014030 con 0x7fa2f81094f0 2024-09-21T02:52:10.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.702+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2f810bc70 con 0x7fa2e80b6670 2024-09-21T02:52:10.704 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:10 smithi031 ceph-mon[31209]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:52:10.704 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:10 smithi031 ceph-mon[31209]: pgmap v303: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:10.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.708+0000 7fa2f5ffb640 1 -- 172.21.15.31:0/1120710354 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259058 (secure 0 0 0) 0x7fa2f810bc70 con 0x7fa2e80b6670 2024-09-21T02:52:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:10.712 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:52:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa2e80b6670 msgr2=0x7fa2e80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa2e80b6670 0x7fa2e80b8b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec031440 tx=0x7fa2ec009560 comp rx=0 tx=0).stop 2024-09-21T02:52:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 msgr2=0x7fa2f819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 0x7fa2f819aa70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400c970 tx=0x7fa2e400ce30 comp rx=0 tx=0).stop 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 shutdown_connections 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa2e80b6670 0x7fa2e80b8b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f8109e40 0x7fa2f819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2f81094f0 0x7fa2f819aa70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 --2- 172.21.15.31:0/1120710354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2f8103bb0 0x7fa2f819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 >> 172.21.15.31:0/1120710354 conn(0x7fa2f80ff600 msgr2=0x7fa2f8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 shutdown_connections 2024-09-21T02:52:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:10.712+0000 7fa2fef9a640 1 -- 172.21.15.31:0/1120710354 wait complete. 2024-09-21T02:52:10.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:10 smithi165 ceph-mon[34818]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:52:10.942 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:10 smithi165 ceph-mon[34818]: pgmap v303: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:10 smithi178 ceph-mon[34759]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:52:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:10 smithi178 ceph-mon[34759]: pgmap v303: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:11.382 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-21T02:52:09.160263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661546,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":775,"num_read_kb":491,"num_write":365,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376492,"kb_used_data":10560,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565332,"statfs":{"total":1151940427776,"available":1151554899968,"internally_reserved":0,"allocated":10813440,"data_stored":7018602,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003610"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574159+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:51:16.574159+0000","last_peered":"2024-09-21T02:51:16.574159+0000","last_clean":"2024-09-21T02:51:16.574159+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:51:16.574159+0000","last_undegraded":"2024-09-21T02:51:16.574159+0000","last_fullsized":"2024-09-21T02:51:16.574159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576740+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:51:16.576740+0000","last_peered":"2024-09-21T02:51:16.576740+0000","last_clean":"2024-09-21T02:51:16.576740+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:51:16.576740+0000","last_undegraded":"2024-09-21T02:51:16.576740+0000","last_fullsized":"2024-09-21T02:51:16.576740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576913+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:51:16.576913+0000","last_peered":"2024-09-21T02:51:16.576913+0000","last_clean":"2024-09-21T02:51:16.576913+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:51:16.576913+0000","last_undegraded":"2024-09-21T02:51:16.576913+0000","last_fullsized":"2024-09-21T02:51:16.576913+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573194+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:51:16.573194+0000","last_peered":"2024-09-21T02:51:16.573194+0000","last_clean":"2024-09-21T02:51:16.573194+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:51:16.573194+0000","last_undegraded":"2024-09-21T02:51:16.573194+0000","last_fullsized":"2024-09-21T02:51:16.573194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572183+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:51:16.572183+0000","last_peered":"2024-09-21T02:51:16.572183+0000","last_clean":"2024-09-21T02:51:16.572183+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:51:16.572183+0000","last_undegraded":"2024-09-21T02:51:16.572183+0000","last_fullsized":"2024-09-21T02:51:16.572183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578336+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:51:16.578336+0000","last_peered":"2024-09-21T02:51:16.578336+0000","last_clean":"2024-09-21T02:51:16.578336+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:51:16.578336+0000","last_undegraded":"2024-09-21T02:51:16.578336+0000","last_fullsized":"2024-09-21T02:51:16.578336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576570+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:51:16.576570+0000","last_peered":"2024-09-21T02:51:16.576570+0000","last_clean":"2024-09-21T02:51:16.576570+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:51:16.576570+0000","last_undegraded":"2024-09-21T02:51:16.576570+0000","last_fullsized":"2024-09-21T02:51:16.576570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976088+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:51:16.976088+0000","last_peered":"2024-09-21T02:51:16.976088+0000","last_clean":"2024-09-21T02:51:16.976088+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:51:16.976088+0000","last_undegraded":"2024-09-21T02:51:16.976088+0000","last_fullsized":"2024-09-21T02:51:16.976088+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977495+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:51:16.977495+0000","last_peered":"2024-09-21T02:51:16.977495+0000","last_clean":"2024-09-21T02:51:16.977495+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:51:16.977495+0000","last_undegraded":"2024-09-21T02:51:16.977495+0000","last_fullsized":"2024-09-21T02:51:16.977495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574699+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:51:16.574699+0000","last_peered":"2024-09-21T02:51:16.574699+0000","last_clean":"2024-09-21T02:51:16.574699+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:51:16.574699+0000","last_undegraded":"2024-09-21T02:51:16.574699+0000","last_fullsized":"2024-09-21T02:51:16.574699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576637+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:51:16.576637+0000","last_peered":"2024-09-21T02:51:16.576637+0000","last_clean":"2024-09-21T02:51:16.576637+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:51:16.576637+0000","last_undegraded":"2024-09-21T02:51:16.576637+0000","last_fullsized":"2024-09-21T02:51:16.576637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576458+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:51:16.576458+0000","last_peered":"2024-09-21T02:51:16.576458+0000","last_clean":"2024-09-21T02:51:16.576458+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:51:16.576458+0000","last_undegraded":"2024-09-21T02:51:16.576458+0000","last_fullsized":"2024-09-21T02:51:16.576458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573699+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:51:16.573699+0000","last_peered":"2024-09-21T02:51:16.573699+0000","last_clean":"2024-09-21T02:51:16.573699+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:51:16.573699+0000","last_undegraded":"2024-09-21T02:51:16.573699+0000","last_fullsized":"2024-09-21T02:51:16.573699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576623+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:51:16.576623+0000","last_peered":"2024-09-21T02:51:16.576623+0000","last_clean":"2024-09-21T02:51:16.576623+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:51:16.576623+0000","last_undegraded":"2024-09-21T02:51:16.576623+0000","last_fullsized":"2024-09-21T02:51:16.576623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576412+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:51:16.576412+0000","last_peered":"2024-09-21T02:51:16.576412+0000","last_clean":"2024-09-21T02:51:16.576412+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:51:16.576412+0000","last_undegraded":"2024-09-21T02:51:16.576412+0000","last_fullsized":"2024-09-21T02:51:16.576412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976749+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:51:16.976749+0000","last_peered":"2024-09-21T02:51:16.976749+0000","last_clean":"2024-09-21T02:51:16.976749+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:51:16.976749+0000","last_undegraded":"2024-09-21T02:51:16.976749+0000","last_fullsized":"2024-09-21T02:51:16.976749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574274+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:51:16.574274+0000","last_peered":"2024-09-21T02:51:16.574274+0000","last_clean":"2024-09-21T02:51:16.574274+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:51:16.574274+0000","last_undegraded":"2024-09-21T02:51:16.574274+0000","last_fullsized":"2024-09-21T02:51:16.574274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571557+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:51:16.571557+0000","last_peered":"2024-09-21T02:51:16.571557+0000","last_clean":"2024-09-21T02:51:16.571557+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:51:16.571557+0000","last_undegraded":"2024-09-21T02:51:16.571557+0000","last_fullsized":"2024-09-21T02:51:16.571557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575827+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:51:16.575827+0000","last_peered":"2024-09-21T02:51:16.575827+0000","last_clean":"2024-09-21T02:51:16.575827+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:51:16.575827+0000","last_undegraded":"2024-09-21T02:51:16.575827+0000","last_fullsized":"2024-09-21T02:51:16.575827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573836+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:51:16.573836+0000","last_peered":"2024-09-21T02:51:16.573836+0000","last_clean":"2024-09-21T02:51:16.573836+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:51:16.573836+0000","last_undegraded":"2024-09-21T02:51:16.573836+0000","last_fullsized":"2024-09-21T02:51:16.573836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573770+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:51:16.573770+0000","last_peered":"2024-09-21T02:51:16.573770+0000","last_clean":"2024-09-21T02:51:16.573770+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:51:16.573770+0000","last_undegraded":"2024-09-21T02:51:16.573770+0000","last_fullsized":"2024-09-21T02:51:16.573770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976953+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:51:16.976953+0000","last_peered":"2024-09-21T02:51:16.976953+0000","last_clean":"2024-09-21T02:51:16.976953+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:51:16.976953+0000","last_undegraded":"2024-09-21T02:51:16.976953+0000","last_fullsized":"2024-09-21T02:51:16.976953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975815+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:51:16.975815+0000","last_peered":"2024-09-21T02:51:16.975815+0000","last_clean":"2024-09-21T02:51:16.975815+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:51:16.975815+0000","last_undegraded":"2024-09-21T02:51:16.975815+0000","last_fullsized":"2024-09-21T02:51:16.975815+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977399+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:51:16.977399+0000","last_peered":"2024-09-21T02:51:16.977399+0000","last_clean":"2024-09-21T02:51:16.977399+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:51:16.977399+0000","last_undegraded":"2024-09-21T02:51:16.977399+0000","last_fullsized":"2024-09-21T02:51:16.977399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976482+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:51:16.976482+0000","last_peered":"2024-09-21T02:51:16.976482+0000","last_clean":"2024-09-21T02:51:16.976482+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:51:16.976482+0000","last_undegraded":"2024-09-21T02:51:16.976482+0000","last_fullsized":"2024-09-21T02:51:16.976482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576864+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:51:16.576864+0000","last_peered":"2024-09-21T02:51:16.576864+0000","last_clean":"2024-09-21T02:51:16.576864+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:51:16.576864+0000","last_undegraded":"2024-09-21T02:51:16.576864+0000","last_fullsized":"2024-09-21T02:51:16.576864+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575552+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:51:16.575552+0000","last_peered":"2024-09-21T02:51:16.575552+0000","last_clean":"2024-09-21T02:51:16.575552+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:51:16.575552+0000","last_undegraded":"2024-09-21T02:51:16.575552+0000","last_fullsized":"2024-09-21T02:51:16.575552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574009+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:51:16.574009+0000","last_peered":"2024-09-21T02:51:16.574009+0000","last_clean":"2024-09-21T02:51:16.574009+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:51:16.574009+0000","last_undegraded":"2024-09-21T02:51:16.574009+0000","last_fullsized":"2024-09-21T02:51:16.574009+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580839+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:51:16.580839+0000","last_peered":"2024-09-21T02:51:16.580839+0000","last_clean":"2024-09-21T02:51:16.580839+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:51:16.580839+0000","last_undegraded":"2024-09-21T02:51:16.580839+0000","last_fullsized":"2024-09-21T02:51:16.580839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575696+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:51:16.575696+0000","last_peered":"2024-09-21T02:51:16.575696+0000","last_clean":"2024-09-21T02:51:16.575696+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:51:16.575696+0000","last_undegraded":"2024-09-21T02:51:16.575696+0000","last_fullsized":"2024-09-21T02:51:16.575696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576084+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:51:16.576084+0000","last_peered":"2024-09-21T02:51:16.576084+0000","last_clean":"2024-09-21T02:51:16.576084+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:51:16.576084+0000","last_undegraded":"2024-09-21T02:51:16.576084+0000","last_fullsized":"2024-09-21T02:51:16.576084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574101+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:51:16.574101+0000","last_peered":"2024-09-21T02:51:16.574101+0000","last_clean":"2024-09-21T02:51:16.574101+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:51:16.574101+0000","last_undegraded":"2024-09-21T02:51:16.574101+0000","last_fullsized":"2024-09-21T02:51:16.574101+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572143+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:51:16.572143+0000","last_peered":"2024-09-21T02:51:16.572143+0000","last_clean":"2024-09-21T02:51:16.572143+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:51:16.572143+0000","last_undegraded":"2024-09-21T02:51:16.572143+0000","last_fullsized":"2024-09-21T02:51:16.572143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'6","reported_seq":113,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:07.656472+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:52:07.656472+0000","last_peered":"2024-09-21T02:52:07.656472+0000","last_clean":"2024-09-21T02:52:07.656472+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:52:07.656472+0000","last_undegraded":"2024-09-21T02:52:07.656472+0000","last_fullsized":"2024-09-21T02:52:07.656472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578401+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:51:16.578401+0000","last_peered":"2024-09-21T02:51:16.578401+0000","last_clean":"2024-09-21T02:51:16.578401+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:51:16.578401+0000","last_undegraded":"2024-09-21T02:51:16.578401+0000","last_fullsized":"2024-09-21T02:51:16.578401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976219+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:51:16.976219+0000","last_peered":"2024-09-21T02:51:16.976219+0000","last_clean":"2024-09-21T02:51:16.976219+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:51:16.976219+0000","last_undegraded":"2024-09-21T02:51:16.976219+0000","last_fullsized":"2024-09-21T02:51:16.976219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571256+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:51:16.571256+0000","last_peered":"2024-09-21T02:51:16.571256+0000","last_clean":"2024-09-21T02:51:16.571256+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:51:16.571256+0000","last_undegraded":"2024-09-21T02:51:16.571256+0000","last_fullsized":"2024-09-21T02:51:16.571256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'4","reported_seq":159,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:06.626167+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:52:06.626167+0000","last_peered":"2024-09-21T02:52:06.626167+0000","last_clean":"2024-09-21T02:52:06.626167+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:52:06.626167+0000","last_undegraded":"2024-09-21T02:52:06.626167+0000","last_fullsized":"2024-09-21T02:52:06.626167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977390+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:51:16.977390+0000","last_peered":"2024-09-21T02:51:16.977390+0000","last_clean":"2024-09-21T02:51:16.977390+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:51:16.977390+0000","last_undegraded":"2024-09-21T02:51:16.977390+0000","last_fullsized":"2024-09-21T02:51:16.977390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577020+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:51:16.577020+0000","last_peered":"2024-09-21T02:51:16.577020+0000","last_clean":"2024-09-21T02:51:16.577020+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:51:16.577020+0000","last_undegraded":"2024-09-21T02:51:16.577020+0000","last_fullsized":"2024-09-21T02:51:16.577020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977117+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:51:16.977117+0000","last_peered":"2024-09-21T02:51:16.977117+0000","last_clean":"2024-09-21T02:51:16.977117+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:51:16.977117+0000","last_undegraded":"2024-09-21T02:51:16.977117+0000","last_fullsized":"2024-09-21T02:51:16.977117+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'28","reported_seq":190,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:06.654332+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:52:06.654332+0000","last_peered":"2024-09-21T02:52:06.654332+0000","last_clean":"2024-09-21T02:52:06.654332+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:52:06.654332+0000","last_undegraded":"2024-09-21T02:52:06.654332+0000","last_fullsized":"2024-09-21T02:52:06.654332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63349,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":130,"num_write":30,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572844+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:51:16.572844+0000","last_peered":"2024-09-21T02:51:16.572844+0000","last_clean":"2024-09-21T02:51:16.572844+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:51:16.572844+0000","last_undegraded":"2024-09-21T02:51:16.572844+0000","last_fullsized":"2024-09-21T02:51:16.572844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578577+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:51:16.578577+0000","last_peered":"2024-09-21T02:51:16.578577+0000","last_clean":"2024-09-21T02:51:16.578577+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:51:16.578577+0000","last_undegraded":"2024-09-21T02:51:16.578577+0000","last_fullsized":"2024-09-21T02:51:16.578577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576983+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:51:16.576983+0000","last_peered":"2024-09-21T02:51:16.576983+0000","last_clean":"2024-09-21T02:51:16.576983+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:51:16.576983+0000","last_undegraded":"2024-09-21T02:51:16.576983+0000","last_fullsized":"2024-09-21T02:51:16.576983+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'5","reported_seq":98,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:07.627795+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:52:07.627795+0000","last_peered":"2024-09-21T02:52:07.627795+0000","last_clean":"2024-09-21T02:52:07.627795+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:52:07.627795+0000","last_undegraded":"2024-09-21T02:52:07.627795+0000","last_fullsized":"2024-09-21T02:52:07.627795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.580840+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:51:16.580840+0000","last_peered":"2024-09-21T02:51:16.580840+0000","last_clean":"2024-09-21T02:51:16.580840+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:51:16.580840+0000","last_undegraded":"2024-09-21T02:51:16.580840+0000","last_fullsized":"2024-09-21T02:51:16.580840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577762+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:51:16.577762+0000","last_peered":"2024-09-21T02:51:16.577762+0000","last_clean":"2024-09-21T02:51:16.577762+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:51:16.577762+0000","last_undegraded":"2024-09-21T02:51:16.577762+0000","last_fullsized":"2024-09-21T02:51:16.577762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573938+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:51:16.573938+0000","last_peered":"2024-09-21T02:51:16.573938+0000","last_clean":"2024-09-21T02:51:16.573938+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:51:16.573938+0000","last_undegraded":"2024-09-21T02:51:16.573938+0000","last_fullsized":"2024-09-21T02:51:16.573938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576883+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:51:16.576883+0000","last_peered":"2024-09-21T02:51:16.576883+0000","last_clean":"2024-09-21T02:51:16.576883+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:51:16.576883+0000","last_undegraded":"2024-09-21T02:51:16.576883+0000","last_fullsized":"2024-09-21T02:51:16.576883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576759+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:51:16.576759+0000","last_peered":"2024-09-21T02:51:16.576759+0000","last_clean":"2024-09-21T02:51:16.576759+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:51:16.576759+0000","last_undegraded":"2024-09-21T02:51:16.576759+0000","last_fullsized":"2024-09-21T02:51:16.576759+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'6","reported_seq":100,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:07.628110+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:52:07.628110+0000","last_peered":"2024-09-21T02:52:07.628110+0000","last_clean":"2024-09-21T02:52:07.628110+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:52:07.628110+0000","last_undegraded":"2024-09-21T02:52:07.628110+0000","last_fullsized":"2024-09-21T02:52:07.628110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576701+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:51:16.576701+0000","last_peered":"2024-09-21T02:51:16.576701+0000","last_clean":"2024-09-21T02:51:16.576701+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:51:16.576701+0000","last_undegraded":"2024-09-21T02:51:16.576701+0000","last_fullsized":"2024-09-21T02:51:16.576701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573447+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573447+0000","last_peered":"2024-09-21T02:51:16.573447+0000","last_clean":"2024-09-21T02:51:16.573447+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573447+0000","last_undegraded":"2024-09-21T02:51:16.573447+0000","last_fullsized":"2024-09-21T02:51:16.573447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976061+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:51:16.976061+0000","last_peered":"2024-09-21T02:51:16.976061+0000","last_clean":"2024-09-21T02:51:16.976061+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:51:16.976061+0000","last_undegraded":"2024-09-21T02:51:16.976061+0000","last_fullsized":"2024-09-21T02:51:16.976061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576775+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576775+0000","last_peered":"2024-09-21T02:51:16.576775+0000","last_clean":"2024-09-21T02:51:16.576775+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576775+0000","last_undegraded":"2024-09-21T02:51:16.576775+0000","last_fullsized":"2024-09-21T02:51:16.576775+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573136+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:51:16.573136+0000","last_peered":"2024-09-21T02:51:16.573136+0000","last_clean":"2024-09-21T02:51:16.573136+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:51:16.573136+0000","last_undegraded":"2024-09-21T02:51:16.573136+0000","last_fullsized":"2024-09-21T02:51:16.573136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'5","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:04.622912+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:52:04.622912+0000","last_peered":"2024-09-21T02:52:04.622912+0000","last_clean":"2024-09-21T02:52:04.622912+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:52:04.622912+0000","last_undegraded":"2024-09-21T02:52:04.622912+0000","last_fullsized":"2024-09-21T02:52:04.622912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578540+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:51:16.578540+0000","last_peered":"2024-09-21T02:51:16.578540+0000","last_clean":"2024-09-21T02:51:16.578540+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:51:16.578540+0000","last_undegraded":"2024-09-21T02:51:16.578540+0000","last_fullsized":"2024-09-21T02:51:16.578540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"76'6","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:02.619667+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:52:02.619667+0000","last_peered":"2024-09-21T02:52:02.619667+0000","last_clean":"2024-09-21T02:52:02.619667+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:52:02.619667+0000","last_undegraded":"2024-09-21T02:52:02.619667+0000","last_fullsized":"2024-09-21T02:52:02.619667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574384+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:51:16.574384+0000","last_peered":"2024-09-21T02:51:16.574384+0000","last_clean":"2024-09-21T02:51:16.574384+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:51:16.574384+0000","last_undegraded":"2024-09-21T02:51:16.574384+0000","last_fullsized":"2024-09-21T02:51:16.574384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571552+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571552+0000","last_peered":"2024-09-21T02:51:16.571552+0000","last_clean":"2024-09-21T02:51:16.571552+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571552+0000","last_undegraded":"2024-09-21T02:51:16.571552+0000","last_fullsized":"2024-09-21T02:51:16.571552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574339+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:51:16.574339+0000","last_peered":"2024-09-21T02:51:16.574339+0000","last_clean":"2024-09-21T02:51:16.574339+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:51:16.574339+0000","last_undegraded":"2024-09-21T02:51:16.574339+0000","last_fullsized":"2024-09-21T02:51:16.574339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574116+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:51:16.574116+0000","last_peered":"2024-09-21T02:51:16.574116+0000","last_clean":"2024-09-21T02:51:16.574116+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:51:16.574116+0000","last_undegraded":"2024-09-21T02:51:16.574116+0000","last_fullsized":"2024-09-21T02:51:16.574116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'14","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571740+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:16.571740+0000","last_peered":"2024-09-21T02:51:16.571740+0000","last_clean":"2024-09-21T02:51:16.571740+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:16.571740+0000","last_undegraded":"2024-09-21T02:51:16.571740+0000","last_fullsized":"2024-09-21T02:51:16.571740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976502+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:51:16.976502+0000","last_peered":"2024-09-21T02:51:16.976502+0000","last_clean":"2024-09-21T02:51:16.976502+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:51:16.976502+0000","last_undegraded":"2024-09-21T02:51:16.976502+0000","last_fullsized":"2024-09-21T02:51:16.976502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573953+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.573953+0000","last_peered":"2024-09-21T02:51:16.573953+0000","last_clean":"2024-09-21T02:51:16.573953+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.573953+0000","last_undegraded":"2024-09-21T02:51:16.573953+0000","last_fullsized":"2024-09-21T02:51:16.573953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576841+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:51:16.576841+0000","last_peered":"2024-09-21T02:51:16.576841+0000","last_clean":"2024-09-21T02:51:16.576841+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:51:16.576841+0000","last_undegraded":"2024-09-21T02:51:16.576841+0000","last_fullsized":"2024-09-21T02:51:16.576841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976881+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976881+0000","last_peered":"2024-09-21T02:51:16.976881+0000","last_clean":"2024-09-21T02:51:16.976881+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976881+0000","last_undegraded":"2024-09-21T02:51:16.976881+0000","last_fullsized":"2024-09-21T02:51:16.976881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571934+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:51:16.571934+0000","last_peered":"2024-09-21T02:51:16.571934+0000","last_clean":"2024-09-21T02:51:16.571934+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:51:16.571934+0000","last_undegraded":"2024-09-21T02:51:16.571934+0000","last_fullsized":"2024-09-21T02:51:16.571934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.975953+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.975953+0000","last_peered":"2024-09-21T02:51:16.975953+0000","last_clean":"2024-09-21T02:51:16.975953+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.975953+0000","last_undegraded":"2024-09-21T02:51:16.975953+0000","last_fullsized":"2024-09-21T02:51:16.975953+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573705+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:51:16.573705+0000","last_peered":"2024-09-21T02:51:16.573705+0000","last_clean":"2024-09-21T02:51:16.573705+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:51:16.573705+0000","last_undegraded":"2024-09-21T02:51:16.573705+0000","last_fullsized":"2024-09-21T02:51:16.573705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.575838+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:51:16.575838+0000","last_peered":"2024-09-21T02:51:16.575838+0000","last_clean":"2024-09-21T02:51:16.575838+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:51:16.575838+0000","last_undegraded":"2024-09-21T02:51:16.575838+0000","last_fullsized":"2024-09-21T02:51:16.575838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976034+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:51:16.976034+0000","last_peered":"2024-09-21T02:51:16.976034+0000","last_clean":"2024-09-21T02:51:16.976034+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:51:16.976034+0000","last_undegraded":"2024-09-21T02:51:16.976034+0000","last_fullsized":"2024-09-21T02:51:16.976034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576863+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:51:16.576863+0000","last_peered":"2024-09-21T02:51:16.576863+0000","last_clean":"2024-09-21T02:51:16.576863+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:51:16.576863+0000","last_undegraded":"2024-09-21T02:51:16.576863+0000","last_fullsized":"2024-09-21T02:51:16.576863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578106+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:51:16.578106+0000","last_peered":"2024-09-21T02:51:16.578106+0000","last_clean":"2024-09-21T02:51:16.578106+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:51:16.578106+0000","last_undegraded":"2024-09-21T02:51:16.578106+0000","last_fullsized":"2024-09-21T02:51:16.578106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.977018+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:51:16.977018+0000","last_peered":"2024-09-21T02:51:16.977018+0000","last_clean":"2024-09-21T02:51:16.977018+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:51:16.977018+0000","last_undegraded":"2024-09-21T02:51:16.977018+0000","last_fullsized":"2024-09-21T02:51:16.977018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577070+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:51:16.577070+0000","last_peered":"2024-09-21T02:51:16.577070+0000","last_clean":"2024-09-21T02:51:16.577070+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:51:16.577070+0000","last_undegraded":"2024-09-21T02:51:16.577070+0000","last_fullsized":"2024-09-21T02:51:16.577070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976572+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976572+0000","last_peered":"2024-09-21T02:51:16.976572+0000","last_clean":"2024-09-21T02:51:16.976572+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976572+0000","last_undegraded":"2024-09-21T02:51:16.976572+0000","last_fullsized":"2024-09-21T02:51:16.976572+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976425+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:51:16.976425+0000","last_peered":"2024-09-21T02:51:16.976425+0000","last_clean":"2024-09-21T02:51:16.976425+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:51:16.976425+0000","last_undegraded":"2024-09-21T02:51:16.976425+0000","last_fullsized":"2024-09-21T02:51:16.976425+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'13","reported_seq":102,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:04.622201+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:52:04.622201+0000","last_peered":"2024-09-21T02:52:04.622201+0000","last_clean":"2024-09-21T02:52:04.622201+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:52:04.622201+0000","last_undegraded":"2024-09-21T02:52:04.622201+0000","last_fullsized":"2024-09-21T02:52:04.622201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70944,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":694,"num_read_kb":422,"num_write":240,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":212832,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710748,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":860160,"data_stored":421192,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075067,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":1013906,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593782,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":860160,"data_stored":420727,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013193,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":421210,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":484587,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":484989,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":880640,"data_stored":422066,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65492,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65090,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65085,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:52:11.387 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T02:52:11.387 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T02:52:11.387 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T02:52:11.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T02:52:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:11.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:11 smithi031 ceph-mon[31209]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:52:11.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:11 smithi165 ceph-mon[34818]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:52:11.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:11 smithi178 ceph-mon[34759]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:52:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.146+0000 7f71083b7640 1 -- 172.21.15.31:0/4189525942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7100105b10 msgr2=0x7f710010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.146+0000 7f71083b7640 1 --2- 172.21.15.31:0/4189525942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7100105b10 0x7f710010f5a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009ca0 tx=0x7f70f802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 -- 172.21.15.31:0/4189525942 shutdown_connections 2024-09-21T02:52:12.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 --2- 172.21.15.31:0/4189525942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7100105b10 0x7f710010f5a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 --2- 172.21.15.31:0/4189525942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71001051c0 0x7f71001055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 --2- 172.21.15.31:0/4189525942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71001031b0 0x7f71001035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 -- 172.21.15.31:0/4189525942 >> 172.21.15.31:0/4189525942 conn(0x7f7100079e40 msgr2=0x7f710007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 -- 172.21.15.31:0/4189525942 shutdown_connections 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 -- 172.21.15.31:0/4189525942 wait complete. 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 Processor -- start 2024-09-21T02:52:12.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.147+0000 7f71083b7640 1 -- start start 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f71083b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 0x7f710019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f71083b7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71001051c0 0x7f710019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f71083b7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7100105b10 0x7f71001a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f71083b7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7100113af0 con 0x7f71001031b0 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f71083b7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7100113970 con 0x7f7100105b10 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f71083b7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7100113c70 con 0x7f71001051c0 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 0x7f710019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 0x7f710019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49848/0 (socket says 172.21.15.31:49848) 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 -- 172.21.15.31:0/1702415620 learned_addr learned my addr 172.21.15.31:0/1702415620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106954640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7100105b10 0x7f71001a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7105952640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71001051c0 0x7f710019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 -- 172.21.15.31:0/1702415620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71001051c0 msgr2=0x7f710019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71001051c0 0x7f710019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 -- 172.21.15.31:0/1702415620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7100105b10 msgr2=0x7f71001a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7100105b10 0x7f71001a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 -- 172.21.15.31:0/1702415620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71001a2690 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f7106153640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 0x7f710019c720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00d770 tx=0x7f70fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.148+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc017070 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.149+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71001a2980 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.149+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70fc009c30 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.149+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc003cc0 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.149+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001a90e0 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.150+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71001035c0 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.151+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f70fc003e60 con 0x7f71001031b0 2024-09-21T02:52:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.152+0000 7f70f77fe640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70ec0b65f0 0x7f70ec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.152+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f70fc0e73a0 con 0x7f71001031b0 2024-09-21T02:52:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.152+0000 7f7105952640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70ec0b65f0 0x7f70ec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.152+0000 7f7105952640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70ec0b65f0 0x7f70ec0b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005fd0 tx=0x7f70f0005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.155+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70fc009de0 con 0x7f71001031b0 2024-09-21T02:52:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.382+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f71001a9450 con 0x7f71001031b0 2024-09-21T02:52:12.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.383+0000 7f70f77fe640 1 -- 172.21.15.31:0/1702415620 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f70fc0ad9d0 con 0x7f71001031b0 2024-09-21T02:52:12.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:12.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T02:52:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70ec0b65f0 msgr2=0x7f70ec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70ec0b65f0 0x7f70ec0b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005fd0 tx=0x7f70f0005850 comp rx=0 tx=0).stop 2024-09-21T02:52:12.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 msgr2=0x7f710019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 0x7f710019c720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00d770 tx=0x7f70fc00dc30 comp rx=0 tx=0).stop 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 shutdown_connections 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70ec0b65f0 0x7f70ec0b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71001051c0 0x7f710019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7100105b10 0x7f71001a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 --2- 172.21.15.31:0/1702415620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71001031b0 0x7f710019c720 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 >> 172.21.15.31:0/1702415620 conn(0x7f7100079e40 msgr2=0x7f7100106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 shutdown_connections 2024-09-21T02:52:12.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:12.385+0000 7f71083b7640 1 -- 172.21.15.31:0/1702415620 wait complete. 2024-09-21T02:52:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:12 smithi165 ceph-mon[34818]: pgmap v304: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:12 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1702415620' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:52:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:12 smithi178 ceph-mon[34759]: pgmap v304: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:12 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1702415620' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:52:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:12 smithi031 ceph-mon[31209]: pgmap v304: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:13.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:12 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1702415620' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:52:13.034 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T02:52:13.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:52:13.292 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:13.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 -- 172.21.15.31:0/3020636386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0104a30 msgr2=0x7fedf0104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:13.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 --2- 172.21.15.31:0/3020636386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0104a30 0x7fedf0104e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 -- 172.21.15.31:0/3020636386 shutdown_connections 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 --2- 172.21.15.31:0/3020636386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf010de60 0x7fedf0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 --2- 172.21.15.31:0/3020636386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf0105380 0x7fedf010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 --2- 172.21.15.31:0/3020636386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0104a30 0x7fedf0104e40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 -- 172.21.15.31:0/3020636386 >> 172.21.15.31:0/3020636386 conn(0x7fedf00797b0 msgr2=0x7fedf0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 -- 172.21.15.31:0/3020636386 shutdown_connections 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.801+0000 7fedf80f3640 1 -- 172.21.15.31:0/3020636386 wait complete. 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 Processor -- start 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 -- start start 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf0104a30 0x7fedf019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf0105380 0x7fedf019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 0x7fedf019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0113aa0 con 0x7fedf010de60 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0113920 con 0x7fedf0104a30 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf80f3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0113c20 con 0x7fedf0105380 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf6690640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 0x7fedf019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf6690640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 0x7fedf019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49864/0 (socket says 172.21.15.31:49864) 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf6690640 1 -- 172.21.15.31:0/1297918214 learned_addr learned my addr 172.21.15.31:0/1297918214 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.802+0000 7fedf5e8f640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf0104a30 0x7fedf019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf568e640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf0105380 0x7fedf019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf6690640 1 -- 172.21.15.31:0/1297918214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf0105380 msgr2=0x7fedf019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf6690640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf0105380 0x7fedf019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf6690640 1 -- 172.21.15.31:0/1297918214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf0104a30 msgr2=0x7fedf019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf6690640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf0104a30 0x7fedf019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf6690640 1 -- 172.21.15.31:0/1297918214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf01a0400 con 0x7fedf010de60 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf568e640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf0105380 0x7fedf019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf6690640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 0x7fedf019fcd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fede800d750 tx=0x7fede800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8017070 con 0x7fedf010de60 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede80054d0 con 0x7fedf010de60 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8004c60 con 0x7fedf010de60 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf01a06f0 con 0x7fedf010de60 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.803+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01a6e50 con 0x7fedf010de60 2024-09-21T02:52:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.804+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0104e40 con 0x7fedf010de60 2024-09-21T02:52:13.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.806+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fede8009bd0 con 0x7fedf010de60 2024-09-21T02:52:13.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.807+0000 7fedeeffd640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fedd80b6670 0x7fedd80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.807+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fede80e7350 con 0x7fedf010de60 2024-09-21T02:52:13.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.807+0000 7fedf5e8f640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fedd80b6670 0x7fedd80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.807+0000 7fedf5e8f640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fedd80b6670 0x7fedd80b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fede0005e30 tx=0x7fede0009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:13.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:13.810+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fede80ada00 con 0x7fedf010de60 2024-09-21T02:52:14.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.036+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedf01a7180 con 0x7fedf010de60 2024-09-21T02:52:14.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.036+0000 7fedeeffd640 1 -- 172.21.15.31:0/1297918214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7fede80b1de0 con 0x7fedf010de60 2024-09-21T02:52:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:14.038 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:51:02.068424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24659,24655,24637,24632]},"id":1}]} 2024-09-21T02:52:14.038 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fedd80b6670 msgr2=0x7fedd80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fedd80b6670 0x7fedd80b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fede0005e30 tx=0x7fede0009360 comp rx=0 tx=0).stop 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 msgr2=0x7fedf019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 0x7fedf019fcd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fede800d750 tx=0x7fede800dc10 comp rx=0 tx=0).stop 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 shutdown_connections 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fedd80b6670 0x7fedd80b8b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedf0105380 0x7fedf019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf010de60 0x7fedf019fcd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 --2- 172.21.15.31:0/1297918214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf0104a30 0x7fedf019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 >> 172.21.15.31:0/1297918214 conn(0x7fedf00797b0 msgr2=0x7fedf0107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:14.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 shutdown_connections 2024-09-21T02:52:14.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:14.039+0000 7fedf80f3640 1 -- 172.21.15.31:0/1297918214 wait complete. 2024-09-21T02:52:14.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:52:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:14 smithi165 ceph-mon[34818]: pgmap v305: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:14 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1297918214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:14 smithi178 ceph-mon[34759]: pgmap v305: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:14 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1297918214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:15.040 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:14 smithi031 ceph-mon[31209]: pgmap v305: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:14 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1297918214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 -- 172.21.15.31:0/4164611847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501025b0 msgr2=0x7f64501029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 --2- 172.21.15.31:0/4164611847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501025b0 0x7f64501029c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6440009890 tx=0x7f644002a380 comp rx=0 tx=0).stop 2024-09-21T02:52:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 -- 172.21.15.31:0/4164611847 shutdown_connections 2024-09-21T02:52:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 --2- 172.21.15.31:0/4164611847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501079f0 0x7f6450107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 --2- 172.21.15.31:0/4164611847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 0x7f6450107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 --2- 172.21.15.31:0/4164611847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501025b0 0x7f64501029c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 -- 172.21.15.31:0/4164611847 >> 172.21.15.31:0/4164611847 conn(0x7f64500fe2a0 msgr2=0x7f64501006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.560+0000 7f644ffff640 1 -- 172.21.15.31:0/4164611847 shutdown_connections 2024-09-21T02:52:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.561+0000 7f644ffff640 1 -- 172.21.15.31:0/4164611847 wait complete. 2024-09-21T02:52:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.561+0000 7f644ffff640 1 Processor -- start 2024-09-21T02:52:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.561+0000 7f644ffff640 1 -- start start 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501025b0 0x7f64501a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 0x7f64501a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501079f0 0x7f64501a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644ffff640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6450119eb0 con 0x7f64501079f0 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644ffff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6450119d30 con 0x7f6450102f00 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644ffff640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f645011a030 con 0x7f64501025b0 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501025b0 0x7f64501a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f6447fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 0x7f64501a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f6447fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 0x7f64501a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:55926/0 (socket says 172.21.15.31:55926) 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f6447fff640 1 -- 172.21.15.31:0/2583146796 learned_addr learned my addr 172.21.15.31:0/2583146796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644effd640 1 -- 172.21.15.31:0/2583146796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 msgr2=0x7f64501a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644effd640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 0x7f64501a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644effd640 1 -- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501079f0 msgr2=0x7f64501a6140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644effd640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501079f0 0x7f64501a6140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.562+0000 7f644effd640 1 -- 172.21.15.31:0/2583146796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64501a6870 con 0x7f64501025b0 2024-09-21T02:52:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.563+0000 7f644effd640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501025b0 0x7f64501a0900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6440000c00 tx=0x7f6440033d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.563+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440037040 con 0x7f64501025b0 2024-09-21T02:52:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.563+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64400054d0 con 0x7f64501025b0 2024-09-21T02:52:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.563+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64501a6b00 con 0x7f64501025b0 2024-09-21T02:52:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.563+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645006cde0 con 0x7f64501025b0 2024-09-21T02:52:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.564+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440030e50 con 0x7f64501025b0 2024-09-21T02:52:15.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.565+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645006c020 con 0x7f64501025b0 2024-09-21T02:52:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.567+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f6440044060 con 0x7f64501025b0 2024-09-21T02:52:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.568+0000 7f644cff9640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64300b6670 0x7f64300b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.568+0000 7f6447fff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64300b6670 0x7f64300b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.568+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f64401070e0 con 0x7f64501025b0 2024-09-21T02:52:15.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.568+0000 7f6447fff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64300b6670 0x7f64300b8b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6450108130 tx=0x7f6438009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:15.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.572+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64400cd6b0 con 0x7f64501025b0 2024-09-21T02:52:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.797+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6450107430 con 0x7f64501025b0 2024-09-21T02:52:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.798+0000 7f644cff9640 1 -- 172.21.15.31:0/2583146796 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7f64400d1a90 con 0x7f64501025b0 2024-09-21T02:52:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:15.800 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:51:02.068424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24659,24655,24637,24632]},"id":1}]} 2024-09-21T02:52:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64300b6670 msgr2=0x7f64300b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64300b6670 0x7f64300b8b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6450108130 tx=0x7f6438009260 comp rx=0 tx=0).stop 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501025b0 msgr2=0x7f64501a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501025b0 0x7f64501a0900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6440000c00 tx=0x7f6440033d60 comp rx=0 tx=0).stop 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 shutdown_connections 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64300b6670 0x7f64300b8b80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64501079f0 0x7f64501a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6450102f00 0x7f64501a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 --2- 172.21.15.31:0/2583146796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64501025b0 0x7f64501a0900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 >> 172.21.15.31:0/2583146796 conn(0x7f64500fe2a0 msgr2=0x7f6450105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 shutdown_connections 2024-09-21T02:52:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:15.801+0000 7f644ffff640 1 -- 172.21.15.31:0/2583146796 wait complete. 2024-09-21T02:52:16.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:52:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:16 smithi031 ceph-mon[31209]: pgmap v306: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:16 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2583146796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:16 smithi165 ceph-mon[34818]: pgmap v306: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:16 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2583146796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:16 smithi178 ceph-mon[34759]: pgmap v306: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:16 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2583146796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.250+0000 7fb6e8f00640 1 -- 172.21.15.31:0/1197469782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e4108d00 msgr2=0x7fb6e410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.250+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/1197469782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e4108d00 0x7fb6e410f530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009ca0 tx=0x7fb6dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.250+0000 7fb6e8f00640 1 -- 172.21.15.31:0/1197469782 shutdown_connections 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.250+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/1197469782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e4108d00 0x7fb6e410f530 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/1197469782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e41083b0 0x7fb6e41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/1197469782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 0x7fb6e4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 -- 172.21.15.31:0/1197469782 >> 172.21.15.31:0/1197469782 conn(0x7fb6e4079710 msgr2=0x7fb6e4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 -- 172.21.15.31:0/1197469782 shutdown_connections 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 -- 172.21.15.31:0/1197469782 wait complete. 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 Processor -- start 2024-09-21T02:52:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.251+0000 7fb6e8f00640 1 -- start start 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e8f00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 0x7fb6e419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e8f00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 0x7fb6e419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e8f00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4108d00 0x7fb6e419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e8f00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41129c0 con 0x7fb6e41083b0 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e8f00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4112840 con 0x7fb6e4108d00 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e8f00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4112b40 con 0x7fb6e4102a70 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 0x7fb6e419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 0x7fb6e419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49896/0 (socket says 172.21.15.31:49896) 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 -- 172.21.15.31:0/3382626149 learned_addr learned my addr 172.21.15.31:0/3382626149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e259c640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 0x7fb6e419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e2d9d640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4108d00 0x7fb6e419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 -- 172.21.15.31:0/3382626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 msgr2=0x7fb6e419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 0x7fb6e419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 -- 172.21.15.31:0/3382626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4108d00 msgr2=0x7fb6e419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4108d00 0x7fb6e419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 -- 172.21.15.31:0/3382626149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41a0390 con 0x7fb6e41083b0 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6e259c640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 0x7fb6e419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.252+0000 7fb6dbfff640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 0x7fb6e419a9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00d950 tx=0x7fb6cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.253+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc017070 con 0x7fb6e41083b0 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.253+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6cc009c30 con 0x7fb6e41083b0 2024-09-21T02:52:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.253+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0047a0 con 0x7fb6e41083b0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.253+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e41a0680 con 0x7fb6e41083b0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.253+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41a6de0 con 0x7fb6e41083b0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.254+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e4103cf0 con 0x7fb6e41083b0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.255+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb6cc004020 con 0x7fb6e41083b0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.256+0000 7fb6db7fe640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6670 0x7fb6d00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.256+0000 7fb6e259c640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6670 0x7fb6d00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.256+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb6cc0e76e0 con 0x7fb6e41083b0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.257+0000 7fb6e259c640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6670 0x7fb6d00b8b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40058c0 tx=0x7fb6d4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.259+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6cc020020 con 0x7fb6e41083b0 2024-09-21T02:52:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.449+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e410bb30 con 0x7fb6e41083b0 2024-09-21T02:52:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.450+0000 7fb6db7fe640 1 -- 172.21.15.31:0/3382626149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+23145 (secure 0 0 0) 0x7fb6cc0add90 con 0x7fb6e41083b0 2024-09-21T02:52:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:17.452 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:51:16.567886+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:52:17.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6670 msgr2=0x7fb6d00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6670 0x7fb6d00b8b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40058c0 tx=0x7fb6d4005850 comp rx=0 tx=0).stop 2024-09-21T02:52:17.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 msgr2=0x7fb6e419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 0x7fb6e419a9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00d950 tx=0x7fb6cc00de10 comp rx=0 tx=0).stop 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 shutdown_connections 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6670 0x7fb6d00b8b80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4108d00 0x7fb6e419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41083b0 0x7fb6e419a9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 --2- 172.21.15.31:0/3382626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4102a70 0x7fb6e419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 >> 172.21.15.31:0/3382626149 conn(0x7fb6e4079710 msgr2=0x7fb6e41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 shutdown_connections 2024-09-21T02:52:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:17.453+0000 7fb6e8f00640 1 -- 172.21.15.31:0/3382626149 wait complete. 2024-09-21T02:52:17.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:17 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3382626149' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:17.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:17 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3382626149' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:17.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:17 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3382626149' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:18.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:52:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:18.752 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:18 smithi031 ceph-mon[31209]: pgmap v307: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T02:52:18.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:18 smithi031 ceph-mon[31209]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 -- 172.21.15.31:0/3932979335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380109df0 msgr2=0x7f5380110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 --2- 172.21.15.31:0/3932979335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380109df0 0x7f5380110620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5378009ca0 tx=0x7f537802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 -- 172.21.15.31:0/3932979335 shutdown_connections 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 --2- 172.21.15.31:0/3932979335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53801094a0 0x7f53801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 --2- 172.21.15.31:0/3932979335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380109df0 0x7f5380110620 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 --2- 172.21.15.31:0/3932979335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5380103b60 0x7f5380103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 -- 172.21.15.31:0/3932979335 >> 172.21.15.31:0/3932979335 conn(0x7f538007b850 msgr2=0x7f538007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 -- 172.21.15.31:0/3932979335 shutdown_connections 2024-09-21T02:52:18.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.843+0000 7f5384b94640 1 -- 172.21.15.31:0/3932979335 wait complete. 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.844+0000 7f5384b94640 1 Processor -- start 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.844+0000 7f5384b94640 1 -- start start 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f5384b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 0x7f538019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f5384b94640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53801094a0 0x7f538019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f5384b94640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5380109df0 0x7f538019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f5384b94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5380113ab0 con 0x7f5380103b60 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f5384b94640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5380113930 con 0x7f5380109df0 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f5384b94640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5380113c30 con 0x7f53801094a0 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 0x7f538019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 0x7f538019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49912/0 (socket says 172.21.15.31:49912) 2024-09-21T02:52:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 -- 172.21.15.31:0/3240120559 learned_addr learned my addr 172.21.15.31:0/3240120559 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537dd9b640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53801094a0 0x7f538019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537ed9d640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5380109df0 0x7f538019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 -- 172.21.15.31:0/3240120559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53801094a0 msgr2=0x7f538019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53801094a0 0x7f538019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 -- 172.21.15.31:0/3240120559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5380109df0 msgr2=0x7f538019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5380109df0 0x7f538019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 -- 172.21.15.31:0/3240120559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53801a03d0 con 0x7f5380103b60 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f537e59c640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 0x7f538019a460 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f537000e9f0 tx=0x7f537000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.845+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537000cd80 con 0x7f5380103b60 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.846+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53801a06c0 con 0x7f5380103b60 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.846+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53700108e0 con 0x7f5380103b60 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.846+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370018980 con 0x7f5380103b60 2024-09-21T02:52:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.846+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53801a6e20 con 0x7f5380103b60 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.847+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5380103f70 con 0x7f5380103b60 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.848+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f5370010430 con 0x7f5380103b60 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.849+0000 7f536f7fe640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53680b6560 0x7f53680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.849+0000 7f537dd9b640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53680b6560 0x7f53680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.849+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f53700e7c20 con 0x7f5380103b60 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.849+0000 7f537dd9b640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53680b6560 0x7f53680b8a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5380109040 tx=0x7f537400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:18.852+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53700b2790 con 0x7f5380103b60 2024-09-21T02:52:18.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:18 smithi165 ceph-mon[34818]: pgmap v307: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T02:52:18.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:18 smithi165 ceph-mon[34818]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T02:52:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:18 smithi178 ceph-mon[34759]: pgmap v307: 97 pgs: 97 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T02:52:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:18 smithi178 ceph-mon[34759]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T02:52:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.078+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f538019b590 con 0x7f5380103b60 2024-09-21T02:52:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.079+0000 7f536f7fe640 1 -- 172.21.15.31:0/3240120559 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7f53700b76d0 con 0x7f5380103b60 2024-09-21T02:52:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:19.081 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:51:02.068424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24659,24655,24637,24632]},"id":1}]} 2024-09-21T02:52:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:52:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53680b6560 msgr2=0x7f53680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53680b6560 0x7f53680b8a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5380109040 tx=0x7f537400b390 comp rx=0 tx=0).stop 2024-09-21T02:52:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 msgr2=0x7f538019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 0x7f538019a460 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f537000e9f0 tx=0x7f537000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 shutdown_connections 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53680b6560 0x7f53680b8a70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53801094a0 0x7f538019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5380109df0 0x7f538019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 --2- 172.21.15.31:0/3240120559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5380103b60 0x7f538019a460 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 >> 172.21.15.31:0/3240120559 conn(0x7f538007b850 msgr2=0x7f53801074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 shutdown_connections 2024-09-21T02:52:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:19.081+0000 7f5384b94640 1 -- 172.21.15.31:0/3240120559 wait complete. 2024-09-21T02:52:19.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:19 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3240120559' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:19.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T02:50:50.973503+0000', 'modified': '2024-09-21T02:51:02.068424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24695, 'mds_1': 24659, 'mds_2': 24655, 'mds_3': 24637, 'mds_4': 24632}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14838': {'gid': 14838, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6839/3653301851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6838', 'nonce': 3653301851}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 3653301851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14850': {'gid': 14850, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/2144690594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 2144690594}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 2144690594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24583': {'gid': 24583, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1642580204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1642580204}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1642580204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/3390770693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 3390770693}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 3390770693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24625': {'gid': 24625, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/178089911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 178089911}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 178089911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.165:6835/3816837605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3816837605}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3816837605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.178:6837/1985506653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1985506653}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1985506653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.178:6839/730341197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 730341197}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 730341197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24659': {'gid': 24659, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.165:6837/3116204029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 3116204029}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 3116204029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.165:6839/2689629692', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 2689629692}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 2689629692}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24695, 'qdb_cluster': [24695, 24659, 24655, 24637, 24632]} 2024-09-21T02:52:19.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T02:52:19.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T02:52:19.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:19 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3240120559' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:19.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:19 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3240120559' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:20.063 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2515719784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 msgr2=0x7f5f2c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2515719784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c1098b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009ca0 tx=0x7f5f1c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2515719784 shutdown_connections 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2515719784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c109df0 0x7f5f2c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2515719784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c1098b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2515719784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2c103b60 0x7f5f2c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2515719784 >> 172.21.15.31:0/2515719784 conn(0x7f5f2c07b850 msgr2=0x7f5f2c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2515719784 shutdown_connections 2024-09-21T02:52:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.536+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2515719784 wait complete. 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 Processor -- start 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 -- start start 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2c103b60 0x7f5f2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c109df0 0x7f5f2c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c113ab0 con 0x7f5f2c1094a0 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c113930 con 0x7f5f2c109df0 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f32e5b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c113c30 con 0x7f5f2c103b60 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36810/0 (socket says 172.21.15.31:36810) 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f2bfff640 1 -- 172.21.15.31:0/2975738410 learned_addr learned my addr 172.21.15.31:0/2975738410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.537+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2c103b60 0x7f5f2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f313f8640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c109df0 0x7f5f2c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f2bfff640 1 -- 172.21.15.31:0/2975738410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2c103b60 msgr2=0x7f5f2c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2c103b60 0x7f5f2c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f2bfff640 1 -- 172.21.15.31:0/2975738410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c109df0 msgr2=0x7f5f2c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c109df0 0x7f5f2c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f2bfff640 1 -- 172.21.15.31:0/2975738410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c1a87c0 con 0x7f5f2c1094a0 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c075c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0319f0 tx=0x7f5f1c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f29ffb640 1 -- 172.21.15.31:0/2975738410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c003bf0 con 0x7f5f2c1094a0 2024-09-21T02:52:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f29ffb640 1 -- 172.21.15.31:0/2975738410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c003d90 con 0x7f5f2c1094a0 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c1a8a50 con 0x7f5f2c1094a0 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f29ffb640 1 -- 172.21.15.31:0/2975738410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c02fb70 con 0x7f5f2c1094a0 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.538+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c1a8fe0 con 0x7f5f2c1094a0 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 Processor -- start 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 -- start start 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f29ffb640 1 -- 172.21.15.31:0/2975738410 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f5f1c02fd10 con 0x7f5f2c1094a0 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222742e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x56222742e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5622273a5c10 con 0x562227087f60 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562227395040 con 0x56222705f0a0 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f393cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562227398e50 con 0x562227067280 2024-09-21T02:52:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f313f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f313f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36818/0 (socket says 172.21.15.31:36818) 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.541+0000 7f5f313f8640 1 -- 172.21.15.31:0/1102453586 learned_addr learned my addr 172.21.15.31:0/1102453586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x56222742e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222742e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f313f8640 1 -- 172.21.15.31:0/1102453586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 msgr2=0x56222742e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f313f8640 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x56222742e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f313f8640 1 -- 172.21.15.31:0/1102453586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 msgr2=0x56222742e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f313f8640 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222742e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f313f8640 1 -- 172.21.15.31:0/1102453586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622270f3540 con 0x562227087f60 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f29ffb640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f180b65e0 0x7f5f180b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f313f8640 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742df00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400caa0 tx=0x7f5f2400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d7bed0e306296 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f180b65e0 0x7f5f180b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f22ffd640 1 -- 172.21.15.31:0/1102453586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24004020 con 0x562227087f60 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f22ffd640 1 -- 172.21.15.31:0/1102453586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f240041c0 con 0x562227087f60 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f29ffb640 1 -- 172.21.15.31:0/2975738410 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5f1c108760 con 0x7f5f2c1094a0 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.542+0000 7f5f22ffd640 1 -- 172.21.15.31:0/1102453586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24010630 con 0x562227087f60 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 -- 172.21.15.31:0/1102453586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 msgr2=0x56222742df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f180b65e0 0x7f5f180b8af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x56222742cb80 tx=0x7f5f1400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742df00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400caa0 tx=0x7f5f2400cf60 comp rx=0 tx=0).stop 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 -- 172.21.15.31:0/1102453586 shutdown_connections 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x56222742e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222742e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 --2- 172.21.15.31:0/1102453586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742df00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 -- 172.21.15.31:0/1102453586 >> 172.21.15.31:0/1102453586 conn(0x56222707c350 msgr2=0x56222742bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 -- 172.21.15.31:0/1102453586 shutdown_connections 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.543+0000 7f5f393cf740 1 -- 172.21.15.31:0/1102453586 wait complete. 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 -- start start 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 Processor -- start 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 0x5622274399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227439f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562227087f60 0x562227445a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5622271f65f0 con 0x56222705f0a0 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562227330010 con 0x562227087f60 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56222743c6e0 con 0x562227067280 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f393cf740 10 client.0 _ll_register_callbacks cb 0x56222738f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:20.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f30bf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 0x5622274399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f30bf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 0x5622274399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36820/0 (socket says 172.21.15.31:36820) 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.544+0000 7f5f30bf7640 1 -- 172.21.15.31:0/3169954869 learned_addr learned my addr 172.21.15.31:0/3169954869 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f313f8640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562227087f60 0x562227445a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227439f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f30bf7640 1 -- 172.21.15.31:0/3169954869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 msgr2=0x562227439f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227439f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f30bf7640 1 -- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562227087f60 msgr2=0x562227445a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562227087f60 0x562227445a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f30bf7640 1 -- 172.21.15.31:0/3169954869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222735da00 con 0x56222705f0a0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f313f8640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562227087f60 0x562227445a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 0x5622274399e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400bfd0 tx=0x7f5f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac943543ba41e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f0bfff640 1 -- 172.21.15.31:0/3169954869 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1401f620 con 0x56222705f0a0 2024-09-21T02:52:20.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f0bfff640 1 -- 172.21.15.31:0/3169954869 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5f1401f7c0 con 0x56222705f0a0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f240041c0 con 0x56222705f0a0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f0bfff640 1 -- 172.21.15.31:0/3169954869 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f1401f9c0 con 0x56222705f0a0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f393cf740 10 client.15237 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.545+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562227448640 con 0x56222705f0a0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f0bfff640 1 -- 172.21.15.31:0/3169954869 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f240041c0 con 0x56222705f0a0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f0bfff640 1 -- 172.21.15.31:0/3169954869 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5f1402e020 con 0x56222705f0a0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f393cf740 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f393cf740 10 client.15237 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f393cf740 20 client.15237 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f393cf740 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5622270468e0 0x56222744c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f393cf740 4 client.15237 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622273a7b60 con 0x5622270468e0 2024-09-21T02:52:20.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.546+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5622270468e0 0x56222744c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f0bfff640 10 client.15237 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5622270468e0 0x56222744c0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9da3908844400e33 server_cookie=90a09adb7dad75d8 in_seq=0 out_seq=0 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f0bfff640 1 -- 172.21.15.31:0/3169954869 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5622273a7b60 con 0x5622270468e0 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f0bfff640 10 client.15237 handle_command_reply: tid=0 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f393cf740 1 client.15237 shutdown 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 msgr2=0x5622274399e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.547+0000 7f5f393cf740 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 0x5622274399e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400bfd0 tx=0x7f5f140046c0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 shutdown_connections 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5622270468e0 0x56222744c0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227439f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562227087f60 0x562227445a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 --2- 172.21.15.31:0/3169954869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56222705f0a0 0x5622274399e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f0bfff640 0 client.15237 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:20.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 >> 172.21.15.31:0/3169954869 conn(0x56222707c350 msgr2=0x5622274310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:20.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 shutdown_connections 2024-09-21T02:52:20.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 1 -- 172.21.15.31:0/3169954869 wait complete. 2024-09-21T02:52:20.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.548+0000 7f5f393cf740 20 client.15237 trim_cache size 0 max 16384 2024-09-21T02:52:20.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 Processor -- start 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 -- start start 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x562227445970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227445eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56222744a3f0 con 0x562227087f60 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56222744aac0 con 0x56222705f0a0 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f393cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56222744b190 con 0x562227067280 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f313f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f313f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36830/0 (socket says 172.21.15.31:36830) 2024-09-21T02:52:20.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.566+0000 7f5f313f8640 1 -- 172.21.15.31:0/192450558 learned_addr learned my addr 172.21.15.31:0/192450558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:20.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x562227445970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227445eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f313f8640 1 -- 172.21.15.31:0/192450558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 msgr2=0x562227445eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f313f8640 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227445eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f313f8640 1 -- 172.21.15.31:0/192450558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 msgr2=0x562227445970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f313f8640 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x562227445970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f313f8640 1 -- 172.21.15.31:0/192450558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622274301f0 con 0x562227087f60 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x562227445970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f313f8640 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742b750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f24018000 tx=0x7f5f2400c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b7e6e2cc5263c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f227fc640 1 -- 172.21.15.31:0/192450558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240040d0 con 0x562227087f60 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f227fc640 1 -- 172.21.15.31:0/192450558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f240054f0 con 0x562227087f60 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.567+0000 7f5f227fc640 1 -- 172.21.15.31:0/192450558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003b40 con 0x562227087f60 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 -- 172.21.15.31:0/192450558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 msgr2=0x56222742b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742b750 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f24018000 tx=0x7f5f2400c370 comp rx=0 tx=0).stop 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 -- 172.21.15.31:0/192450558 shutdown_connections 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227067280 0x562227445eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227087f60 0x56222742b750 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 --2- 172.21.15.31:0/192450558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x562227445970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 -- 172.21.15.31:0/192450558 >> 172.21.15.31:0/192450558 conn(0x56222707c350 msgr2=0x562227438060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 -- 172.21.15.31:0/192450558 shutdown_connections 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 -- 172.21.15.31:0/192450558 wait complete. 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.568+0000 7f5f393cf740 1 -- start start 2024-09-21T02:52:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 Processor -- start 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222744e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 0x56222744ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227087f60 0x562227453ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56222744bf30 con 0x562227067280 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56222744b860 con 0x56222705f0a0 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56222744c600 con 0x562227087f60 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f393cf740 10 client.0 _ll_register_callbacks cb 0x7f5f14005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 0x56222744ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 0x56222744ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36842/0 (socket says 172.21.15.31:36842) 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 -- 172.21.15.31:0/131715362 learned_addr learned my addr 172.21.15.31:0/131715362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222744e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f313f8640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227087f60 0x562227453ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 -- 172.21.15.31:0/131715362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227087f60 msgr2=0x562227453ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227087f60 0x562227453ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 -- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 msgr2=0x56222744e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222744e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.569+0000 7f5f2bfff640 1 -- 172.21.15.31:0/131715362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562227454430 con 0x562227067280 2024-09-21T02:52:20.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f2bfff640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 0x56222744ed50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c02ade0 tx=0x7f5f1c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ac5e1421faca0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f0bfff640 1 -- 172.21.15.31:0/131715362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004d40 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f0bfff640 1 -- 172.21.15.31:0/131715362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5f1c0cda50 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f240054f0 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f0bfff640 1 -- 172.21.15.31:0/131715362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c0f3070 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f393cf740 10 client.15243 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622274b7bb0 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f0bfff640 1 -- 172.21.15.31:0/131715362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f240054f0 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f0bfff640 1 -- 172.21.15.31:0/131715362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5f1c0cd4c0 con 0x562227067280 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.570+0000 7f5f393cf740 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f393cf740 10 client.15243 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T02:52:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f393cf740 20 client.15243 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f393cf740 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x56222701bc90 0x5622274bb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f393cf740 4 client.15243 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "status"}) -- 0x5622273a3610 con 0x56222701bc90 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x56222701bc90 0x5622274bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f30bf7640 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x56222701bc90 0x5622274bb410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6689dd7830301e1c server_cookie=2e2181907b006d6 in_seq=0 out_seq=0 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.571+0000 7f5f0bfff640 10 client.15243 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f0bfff640 1 -- 172.21.15.31:0/131715362 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5622273a3610 con 0x56222701bc90 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f0bfff640 10 client.15243 handle_command_reply: tid=0 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 client.15243 shutdown 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 msgr2=0x56222744ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 0x56222744ed50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c02ade0 tx=0x7f5f1c030000 comp rx=0 tx=0).stop 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 shutdown_connections 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x56222701bc90 0x5622274bb410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562227087f60 0x562227453ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562227067280 0x56222744ed50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 --2- 172.21.15.31:0/131715362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56222705f0a0 0x56222744e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 >> 172.21.15.31:0/131715362 conn(0x56222707c350 msgr2=0x562227437c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f0bfff640 0 client.15243 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 shutdown_connections 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 1 -- 172.21.15.31:0/131715362 wait complete. 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.572+0000 7f5f393cf740 20 client.15243 trim_cache size 0 max 16384 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2024-09-21T02:52:20.580 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24632, 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 79.148934419, 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 120.08290706699999 2024-09-21T02:52:20.581 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f180b65e0 msgr2=0x7f5f180b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f180b65e0 0x7f5f180b8af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x56222742cb80 tx=0x7f5f1400b390 comp rx=0 tx=0).stop 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 msgr2=0x7f5f2c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c075c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0319f0 tx=0x7f5f1c031a20 comp rx=0 tx=0).stop 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 shutdown_connections 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5f180b65e0 0x7f5f180b8af0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c109df0 0x7f5f2c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1094a0 0x7f5f2c075c50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 --2- 172.21.15.31:0/2975738410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f2c103b60 0x7f5f2c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:20.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 >> 172.21.15.31:0/2975738410 conn(0x7f5f2c07b850 msgr2=0x7f5f2c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:20.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 shutdown_connections 2024-09-21T02:52:20.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:20.573+0000 7f5f32e5b640 1 -- 172.21.15.31:0/2975738410 wait complete. 2024-09-21T02:52:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:20 smithi031 ceph-mon[31209]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T02:52:20.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:20 smithi165 ceph-mon[34818]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T02:52:20.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:20 smithi178 ceph-mon[34759]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T02:52:21.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T02:52:21.551 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.010+0000 7f8888dc4640 1 -- 172.21.15.31:0/1072530200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884102290 msgr2=0x7f88841026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.010+0000 7f8888dc4640 1 --2- 172.21.15.31:0/1072530200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884102290 0x7f88841026a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f886c009900 tx=0x7f886c02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 -- 172.21.15.31:0/1072530200 shutdown_connections 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 --2- 172.21.15.31:0/1072530200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8884108520 0x7f888410ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 --2- 172.21.15.31:0/1072530200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8884107bd0 0x7f8884107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 --2- 172.21.15.31:0/1072530200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884102290 0x7f88841026a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 -- 172.21.15.31:0/1072530200 >> 172.21.15.31:0/1072530200 conn(0x7f8884079ea0 msgr2=0x7f888407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 -- 172.21.15.31:0/1072530200 shutdown_connections 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 -- 172.21.15.31:0/1072530200 wait complete. 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 Processor -- start 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.011+0000 7f8888dc4640 1 -- start start 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8888dc4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8884102290 0x7f888419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8888dc4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8884107bd0 0x7f888419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8888dc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 0x7f888419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8888dc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88841121f0 con 0x7f8884108520 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8888dc4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8884112070 con 0x7f8884107bd0 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8888dc4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8884112370 con 0x7f8884102290 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 0x7f888419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 0x7f888419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36860/0 (socket says 172.21.15.31:36860) 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 -- 172.21.15.31:0/2245551017 learned_addr learned my addr 172.21.15.31:0/2245551017 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f888259c640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8884102290 0x7f888419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8881d9b640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8884107bd0 0x7f888419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 -- 172.21.15.31:0/2245551017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8884102290 msgr2=0x7f888419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8884102290 0x7f888419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 -- 172.21.15.31:0/2245551017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8884107bd0 msgr2=0x7f888419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8884107bd0 0x7f888419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.012+0000 7f8882d9d640 1 -- 172.21.15.31:0/2245551017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88841a6570 con 0x7f8884108520 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.013+0000 7f8882d9d640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 0x7f888419bdf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f887c00eaa0 tx=0x7f887c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.013+0000 7f887b7fe640 1 -- 172.21.15.31:0/2245551017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00cdb0 con 0x7f8884108520 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.013+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88841a6860 con 0x7f8884108520 2024-09-21T02:52:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.013+0000 7f887b7fe640 1 -- 172.21.15.31:0/2245551017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f887c005020 con 0x7f8884108520 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.013+0000 7f887b7fe640 1 -- 172.21.15.31:0/2245551017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c0116d0 con 0x7f8884108520 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.013+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88841a6df0 con 0x7f8884108520 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.016+0000 7f888f338740 1 Processor -- start 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.016+0000 7f887b7fe640 1 -- 172.21.15.31:0/2245551017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f887c004b20 con 0x7f8884108520 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 -- start start 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 0x55ccfc01df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc01e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc57280 0x55ccfc01e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbf95c10 con 0x55ccfbc77f60 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbf85040 con 0x55ccfbc57280 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888f338740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbf88e50 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 0x55ccfc01df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 0x55ccfc01df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36876/0 (socket says 172.21.15.31:36876) 2024-09-21T02:52:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 -- 172.21.15.31:0/1690839776 learned_addr learned my addr 172.21.15.31:0/1690839776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc57280 0x55ccfc01e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f888259c640 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc01e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 -- 172.21.15.31:0/1690839776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 msgr2=0x55ccfc01e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc01e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 -- 172.21.15.31:0/1690839776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc57280 msgr2=0x55ccfc01e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc57280 0x55ccfc01e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f8881d9b640 1 -- 172.21.15.31:0/1690839776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccfbce3540 con 0x55ccfbc77f60 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.017+0000 7f887b7fe640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f88700b6590 0x7f88700b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.018+0000 7f8881d9b640 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 0x55ccfc01df00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f88740124f0 tx=0x7f887400c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4691bb3e78f26a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.018+0000 7f888259c640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f88700b6590 0x7f88700b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.018+0000 7f887b7fe640 1 -- 172.21.15.31:0/2245551017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f887c0e76b0 con 0x7f8884108520 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.018+0000 7f8878ff9640 1 -- 172.21.15.31:0/1690839776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874004260 con 0x55ccfbc77f60 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.018+0000 7f8878ff9640 1 -- 172.21.15.31:0/1690839776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8874004400 con 0x55ccfbc77f60 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.018+0000 7f8878ff9640 1 -- 172.21.15.31:0/1690839776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740035f0 con 0x55ccfbc77f60 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 -- 172.21.15.31:0/1690839776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 msgr2=0x55ccfc01df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888259c640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f88700b6590 0x7f88700b8aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f886c000c00 tx=0x7f886c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 0x55ccfc01df00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f88740124f0 tx=0x7f887400c5c0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 -- 172.21.15.31:0/1690839776 shutdown_connections 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc01e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc57280 0x55ccfc01e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 --2- 172.21.15.31:0/1690839776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc77f60 0x55ccfc01df00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 -- 172.21.15.31:0/1690839776 >> 172.21.15.31:0/1690839776 conn(0x55ccfbc6c350 msgr2=0x55ccfc01bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 -- 172.21.15.31:0/1690839776 shutdown_connections 2024-09-21T02:52:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 -- 172.21.15.31:0/1690839776 wait complete. 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.019+0000 7f888f338740 1 -- start start 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 Processor -- start 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc029a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 0x55ccfc029fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc0359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbde65f0 con 0x55ccfbc57280 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbf20010 con 0x55ccfbc77f60 2024-09-21T02:52:22.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc02c6e0 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888f338740 10 client.0 _ll_register_callbacks cb 0x55ccfbf7f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 0x55ccfc029fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 0x55ccfc029fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36878/0 (socket says 172.21.15.31:36878) 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 -- 172.21.15.31:0/886909176 learned_addr learned my addr 172.21.15.31:0/886909176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888259c640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc029a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 -- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 msgr2=0x55ccfc029a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8882d9d640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc0359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc029a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 -- 172.21.15.31:0/886909176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 msgr2=0x55ccfc0359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc0359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f8881d9b640 1 -- 172.21.15.31:0/886909176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccfbf4da00 con 0x55ccfbc57280 2024-09-21T02:52:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.020+0000 7f888259c640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc029a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8881d9b640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 0x55ccfc029fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f88740058c0 tx=0x7f8874005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc45dfc7adb8d620 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8869ffb640 1 -- 172.21.15.31:0/886909176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874004260 con 0x55ccfbc57280 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8869ffb640 1 -- 172.21.15.31:0/886909176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f88740037f0 con 0x55ccfbc57280 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8874004400 con 0x55ccfbc57280 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8869ffb640 1 -- 172.21.15.31:0/886909176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8874005650 con 0x55ccfbc57280 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccfc0386c0 con 0x55ccfbc57280 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8869ffb640 1 -- 172.21.15.31:0/886909176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8874004400 con 0x55ccfbc57280 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f8869ffb640 1 -- 172.21.15.31:0/886909176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f887402e070 con 0x55ccfbc57280 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 20 client.15255 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc368e0 0x55ccfc03bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 4 client.15255 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.021+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccfbf97b60 con 0x55ccfbc368e0 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.022+0000 7f888259c640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc368e0 0x55ccfc03bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.022+0000 7f8869ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.022+0000 7f888259c640 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc368e0 0x55ccfc03bea0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fdc9ecefb0c33f2 server_cookie=5fc07bf92d15eb70 in_seq=0 out_seq=0 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f8869ffb640 1 -- 172.21.15.31:0/886909176 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccfbf97b60 con 0x55ccfbc368e0 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f8869ffb640 10 client.15255 handle_command_reply: tid=0 2024-09-21T02:52:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 client.15255 shutdown 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 msgr2=0x55ccfc029fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 0x55ccfc029fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f88740058c0 tx=0x7f8874005250 comp rx=0 tx=0).stop 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 shutdown_connections 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc368e0 0x55ccfc03bea0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc0359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc57280 0x55ccfc029fc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f8869ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 --2- 172.21.15.31:0/886909176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc029a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 >> 172.21.15.31:0/886909176 conn(0x55ccfbc6c350 msgr2=0x55ccfc0210e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 shutdown_connections 2024-09-21T02:52:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.023+0000 7f888f338740 1 -- 172.21.15.31:0/886909176 wait complete. 2024-09-21T02:52:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.024+0000 7f888f338740 20 client.15255 trim_cache size 0 max 16384 2024-09-21T02:52:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 Processor -- start 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 -- start start 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc03bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc035940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc035e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc03a1e0 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc03a8b0 con 0x55ccfbc77f60 2024-09-21T02:52:22.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888f338740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc03af80 con 0x55ccfbc57280 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc03bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc03bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36892/0 (socket says 172.21.15.31:36892) 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888259c640 1 -- 172.21.15.31:0/334104436 learned_addr learned my addr 172.21.15.31:0/334104436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f8881d9b640 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc035940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888259c640 1 -- 172.21.15.31:0/334104436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 msgr2=0x55ccfc035940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.042+0000 7f888259c640 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc035940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f888259c640 1 -- 172.21.15.31:0/334104436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 msgr2=0x55ccfc035e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f888259c640 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc035e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f888259c640 1 -- 172.21.15.31:0/334104436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccfc02a7a0 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f888259c640 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc03bea0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f886c009900 tx=0x7f886c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20a4c691c1684c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f886bfff640 1 -- 172.21.15.31:0/334104436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c056070 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f886bfff640 1 -- 172.21.15.31:0/334104436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886c0054f0 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.043+0000 7f886bfff640 1 -- 172.21.15.31:0/334104436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c035050 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 -- 172.21.15.31:0/334104436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 msgr2=0x55ccfc03bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc03bea0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f886c009900 tx=0x7f886c03d000 comp rx=0 tx=0).stop 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 -- 172.21.15.31:0/334104436 shutdown_connections 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc035940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc035e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 --2- 172.21.15.31:0/334104436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc03bea0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 -- 172.21.15.31:0/334104436 >> 172.21.15.31:0/334104436 conn(0x55ccfbc6c350 msgr2=0x55ccfc0249c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 -- 172.21.15.31:0/334104436 shutdown_connections 2024-09-21T02:52:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 -- 172.21.15.31:0/334104436 wait complete. 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 -- start start 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.044+0000 7f888f338740 1 Processor -- start 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc044880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc03dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc03e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc03bd20 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc03b650 con 0x55ccfbc77f60 2024-09-21T02:52:22.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc03c3f0 con 0x55ccfbc57280 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 10 client.0 _ll_register_callbacks cb 0x7f887400cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc044880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc044880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36896/0 (socket says 172.21.15.31:36896) 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 -- 172.21.15.31:0/2598717132 learned_addr learned my addr 172.21.15.31:0/2598717132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f8882d9d640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc03e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f8881d9b640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc03dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 -- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 msgr2=0x55ccfc03dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc03dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 -- 172.21.15.31:0/2598717132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 msgr2=0x55ccfc03e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc03e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 -- 172.21.15.31:0/2598717132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccfc03e960 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888259c640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc044880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f886c006010 tx=0x7f886c0318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7729754ea9051850 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f8869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f8869ffb640 1 -- 172.21.15.31:0/2598717132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c056070 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886c0054f0 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.045+0000 7f8869ffb640 1 -- 172.21.15.31:0/2598717132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f886c005040 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f8869ffb640 1 -- 172.21.15.31:0/2598717132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f886c035040 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 10 client.15261 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccfc0a7370 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f8869ffb640 1 -- 172.21.15.31:0/2598717132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886c0054f0 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f8869ffb640 1 -- 172.21.15.31:0/2598717132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f886c061020 con 0x55ccfbc4f0a0 2024-09-21T02:52:22.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 20 client.15261 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc0bc90 0x55ccfc0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 4 client.15261 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.046+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "status"}) -- 0x55ccfbf93610 con 0x55ccfbc0bc90 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f8881d9b640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc0bc90 0x55ccfc0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f8869ffb640 10 client.15261 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:22.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f8881d9b640 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc0bc90 0x55ccfc0aa5b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b2db6443f7d216d server_cookie=f69855a33bdf3ad8 in_seq=0 out_seq=0 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f8869ffb640 1 -- 172.21.15.31:0/2598717132 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ccfbf93610 con 0x55ccfbc0bc90 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f8869ffb640 10 client.15261 handle_command_reply: tid=0 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 client.15261 shutdown 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 msgr2=0x55ccfc044880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc044880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f886c006010 tx=0x7f886c0318c0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 shutdown_connections 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ccfbc0bc90 0x55ccfc0aa5b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ccfbc57280 0x55ccfc03dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ccfbc77f60 0x55ccfc03e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 --2- 172.21.15.31:0/2598717132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccfbc4f0a0 0x55ccfc044880 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 >> 172.21.15.31:0/2598717132 conn(0x55ccfbc6c350 msgr2=0x55ccfc021000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.047+0000 7f8869ffb640 0 client.15261 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 shutdown_connections 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f888f338740 1 -- 172.21.15.31:0/2598717132 wait complete. 2024-09-21T02:52:22.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f888f338740 20 client.15261 trim_cache size 0 max 16384 2024-09-21T02:52:22.063 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2024-09-21T02:52:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24637, 2024-09-21T02:52:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 81.636467046999996, 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 116.50156794599999 2024-09-21T02:52:22.064 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f88700b6590 msgr2=0x7f88700b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f88700b6590 0x7f88700b8aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f886c000c00 tx=0x7f886c009680 comp rx=0 tx=0).stop 2024-09-21T02:52:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 msgr2=0x7f888419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 0x7f888419bdf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f887c00eaa0 tx=0x7f887c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:52:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 shutdown_connections 2024-09-21T02:52:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f88700b6590 0x7f88700b8aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8884108520 0x7f888419bdf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8884107bd0 0x7f888419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 --2- 172.21.15.31:0/2245551017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8884102290 0x7f888419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 >> 172.21.15.31:0/2245551017 conn(0x7f8884079ea0 msgr2=0x7f8884100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 shutdown_connections 2024-09-21T02:52:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:22.048+0000 7f8888dc4640 1 -- 172.21.15.31:0/2245551017 wait complete. 2024-09-21T02:52:22.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T02:52:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:22 smithi165 ceph-mon[34818]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T02:52:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:22 smithi178 ceph-mon[34759]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T02:52:23.064 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:22 smithi031 ceph-mon[31209]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T02:52:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 -- 172.21.15.31:0/4068889430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0102ae0 msgr2=0x7f13d0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 --2- 172.21.15.31:0/4068889430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0102ae0 0x7f13d0102ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009900 tx=0x7f13c002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 -- 172.21.15.31:0/4068889430 shutdown_connections 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 --2- 172.21.15.31:0/4068889430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d0108d20 0x7f13d010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 --2- 172.21.15.31:0/4068889430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 0x7f13d01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 --2- 172.21.15.31:0/4068889430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0102ae0 0x7f13d0102ef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 -- 172.21.15.31:0/4068889430 >> 172.21.15.31:0/4068889430 conn(0x7f13d0079760 msgr2=0x7f13d0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 -- 172.21.15.31:0/4068889430 shutdown_connections 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.520+0000 7f13d4e95640 1 -- 172.21.15.31:0/4068889430 wait complete. 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 Processor -- start 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 -- start start 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d0102ae0 0x7f13d019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 0x7f13d019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 0x7f13d019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0112a10 con 0x7f13d0108d20 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0112890 con 0x7f13d01083d0 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13d4e95640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0112b90 con 0x7f13d0102ae0 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 0x7f13d019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 0x7f13d019be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36912/0 (socket says 172.21.15.31:36912) 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 -- 172.21.15.31:0/900776094 learned_addr learned my addr 172.21.15.31:0/900776094 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ce59c640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d0102ae0 0x7f13d019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 0x7f13d019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 -- 172.21.15.31:0/900776094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d0102ae0 msgr2=0x7f13d019b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d0102ae0 0x7f13d019b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 -- 172.21.15.31:0/900776094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 msgr2=0x7f13d019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.521+0000 7f13ced9d640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 0x7f13d019b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13ced9d640 1 -- 172.21.15.31:0/900776094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d01a6670 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 0x7f13d019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13ced9d640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 0x7f13d019be60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f13c800cad0 tx=0x7f13c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13bf7fe640 1 -- 172.21.15.31:0/900776094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c800ae20 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d01a6960 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13bf7fe640 1 -- 172.21.15.31:0/900776094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c8005020 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13bf7fe640 1 -- 172.21.15.31:0/900776094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c800f6b0 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.522+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d01a6ef0 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.524+0000 7f13db409740 1 Processor -- start 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13bf7fe640 1 -- 172.21.15.31:0/900776094 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f13c80051d0 con 0x7f13d0108d20 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 -- start start 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abf9eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abf9f0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abf9f0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558abf967c10 con 0x558abf629280 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558abf957040 con 0x558abf649f60 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13db409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558abf95ae50 con 0x558abf6210a0 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abf9f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abf9f0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36924/0 (socket says 172.21.15.31:36924) 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.525+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2611472795 learned_addr learned my addr 172.21.15.31:0/2611472795 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13ce59c640 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abf9f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abf9eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13bf7fe640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f13b80b6590 0x7f13b80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2611472795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 msgr2=0x558abf9f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abf9f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2611472795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 msgr2=0x558abf9eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abf9eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2611472795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558abf6b5540 con 0x558abf629280 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13ce59c640 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abf9f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13ced9d640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f13b80b6590 0x7f13b80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abf9f0980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13c400cf60 tx=0x7f13c400c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4318064cb25d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13bf7fe640 1 -- 172.21.15.31:0/900776094 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f13c80e74f0 con 0x7f13d0108d20 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13bcff9640 1 -- 172.21.15.31:0/2611472795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c401b070 con 0x558abf629280 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13bcff9640 1 -- 172.21.15.31:0/2611472795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c40168a0 con 0x558abf629280 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.526+0000 7f13bcff9640 1 -- 172.21.15.31:0/2611472795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c400f040 con 0x558abf629280 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 -- 172.21.15.31:0/2611472795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 msgr2=0x558abf9f0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13ced9d640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f13b80b6590 0x7f13b80b8aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13c80ad280 tx=0x7f13c807a860 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abf9f0980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13c400cf60 tx=0x7f13c400c3d0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 -- 172.21.15.31:0/2611472795 shutdown_connections 2024-09-21T02:52:23.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abf9f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abf9f0980 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 --2- 172.21.15.31:0/2611472795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abf9eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 -- 172.21.15.31:0/2611472795 >> 172.21.15.31:0/2611472795 conn(0x558abf63e350 msgr2=0x558abf9eddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 -- 172.21.15.31:0/2611472795 shutdown_connections 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.527+0000 7f13db409740 1 -- 172.21.15.31:0/2611472795 wait complete. 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 -- start start 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 Processor -- start 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abf9fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf629280 0x558abf9fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf649f60 0x558abfa079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558abf7b85f0 con 0x558abf6210a0 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558abf8f2010 con 0x558abf629280 2024-09-21T02:52:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558abf9fe6e0 con 0x558abf649f60 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13db409740 10 client.0 _ll_register_callbacks cb 0x558abf951340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abf9fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abf9fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36932/0 (socket says 172.21.15.31:36932) 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2666130477 learned_addr learned my addr 172.21.15.31:0/2666130477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.528+0000 7f13ce59c640 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf629280 0x558abf9fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf649f60 msgr2=0x558abfa079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf649f60 0x558abfa079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2666130477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf629280 msgr2=0x558abf9fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf629280 0x558abf9fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13cdd9b640 1 -- 172.21.15.31:0/2666130477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558abf91fa00 con 0x558abf6210a0 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abf9fb9e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f13c400cfd0 tx=0x7f13c400c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f61f322b1bc998e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13b5ffb640 1 -- 172.21.15.31:0/2666130477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c401b070 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13b5ffb640 1 -- 172.21.15.31:0/2666130477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f13c4015530 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c40168a0 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13b5ffb640 1 -- 172.21.15.31:0/2666130477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13c400c040 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13db409740 10 client.15273 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.529+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558abfa0a620 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13b5ffb640 1 -- 172.21.15.31:0/2666130477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c40168a0 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13b5ffb640 1 -- 172.21.15.31:0/2666130477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f13c4028070 con 0x558abf6210a0 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13db409740 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13db409740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13db409740 20 client.15273 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13db409740 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf6088e0 0x558abfa0de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13db409740 4 client.15273 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:23.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558abf969b60 con 0x558abf6088e0 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.530+0000 7f13ce59c640 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf6088e0 0x558abfa0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.532+0000 7f13ce59c640 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf6088e0 0x558abfa0de40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78e709472c8a096e server_cookie=cb84bf52c249e698 in_seq=0 out_seq=0 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.532+0000 7f13b5ffb640 10 client.15273 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13b5ffb640 1 -- 172.21.15.31:0/2666130477 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558abf969b60 con 0x558abf6088e0 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13b5ffb640 10 client.15273 handle_command_reply: tid=0 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 client.15273 shutdown 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 msgr2=0x558abf9fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abf9fb9e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f13c400cfd0 tx=0x7f13c400c500 comp rx=0 tx=0).stop 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 shutdown_connections 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf6088e0 0x558abfa0de40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf649f60 0x558abfa079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13b5ffb640 0 client.15273 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf629280 0x558abf9fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 --2- 172.21.15.31:0/2666130477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abf9fb9e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 >> 172.21.15.31:0/2666130477 conn(0x558abf63e350 msgr2=0x558abf9f30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 shutdown_connections 2024-09-21T02:52:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 1 -- 172.21.15.31:0/2666130477 wait complete. 2024-09-21T02:52:23.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.534+0000 7f13db409740 20 client.15273 trim_cache size 0 max 16384 2024-09-21T02:52:23.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 Processor -- start 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 -- start start 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abfa07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abfa07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa16510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558abfa0c180 con 0x558abf629280 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558abfa0c850 con 0x558abf649f60 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13db409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558abfa0cf20 con 0x558abf6210a0 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13ce59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abfa07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13ce59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abfa07e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36936/0 (socket says 172.21.15.31:36936) 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13ce59c640 1 -- 172.21.15.31:0/3734253983 learned_addr learned my addr 172.21.15.31:0/3734253983 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa16510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.553+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abfa07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13ce59c640 1 -- 172.21.15.31:0/3734253983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 msgr2=0x558abfa07940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13ce59c640 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abfa07940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13ce59c640 1 -- 172.21.15.31:0/3734253983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 msgr2=0x558abfa16510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13ce59c640 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa16510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13ce59c640 1 -- 172.21.15.31:0/3734253983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558abf9ee9f0 con 0x558abf629280 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa16510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13ce59c640 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abfa07e80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f13c00096c0 tx=0x7f13c0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d4626c55c459fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13b7fff640 1 -- 172.21.15.31:0/3734253983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0048070 con 0x558abf629280 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13b7fff640 1 -- 172.21.15.31:0/3734253983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c0031950 con 0x558abf629280 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.554+0000 7f13b7fff640 1 -- 172.21.15.31:0/3734253983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0035040 con 0x558abf629280 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 -- 172.21.15.31:0/3734253983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 msgr2=0x558abfa07e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abfa07e80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f13c00096c0 tx=0x7f13c0031400 comp rx=0 tx=0).stop 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 -- 172.21.15.31:0/3734253983 shutdown_connections 2024-09-21T02:52:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa16510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf629280 0x558abfa07e80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 --2- 172.21.15.31:0/3734253983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf6210a0 0x558abfa07940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 -- 172.21.15.31:0/3734253983 >> 172.21.15.31:0/3734253983 conn(0x558abf63e350 msgr2=0x558abf9f69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 -- 172.21.15.31:0/3734253983 shutdown_connections 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 -- 172.21.15.31:0/3734253983 wait complete. 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 -- start start 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.555+0000 7f13db409740 1 Processor -- start 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abfa0f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf629280 0x558abfa0f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558abfa0dcc0 con 0x558abf6210a0 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558abfa0d5f0 con 0x558abf649f60 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558abfa0e390 con 0x558abf629280 2024-09-21T02:52:23.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13db409740 10 client.0 _ll_register_callbacks cb 0x7f13c4019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abfa0f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abfa0f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36946/0 (socket says 172.21.15.31:36946) 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 -- 172.21.15.31:0/4209883669 learned_addr learned my addr 172.21.15.31:0/4209883669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13ce59c640 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf629280 0x558abfa0f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 -- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf629280 msgr2=0x558abfa0f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf629280 0x558abfa0f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 -- 172.21.15.31:0/4209883669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 msgr2=0x558abfa7c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 -- 172.21.15.31:0/4209883669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558abfa7c880 con 0x558abf6210a0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13cdd9b640 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abfa0f040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f13c400ea90 tx=0x7f13c40214b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153ffb75e22d2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.556+0000 7f13b5ffb640 1 -- 172.21.15.31:0/4209883669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c402e070 con 0x558abf6210a0 2024-09-21T02:52:23.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13b5ffb640 1 -- 172.21.15.31:0/4209883669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f13c4021970 con 0x558abf6210a0 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558abf965610 con 0x558abf6210a0 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13b5ffb640 1 -- 172.21.15.31:0/4209883669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13c403a050 con 0x558abf6210a0 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558abfa7d970 con 0x558abf6210a0 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13b5ffb640 1 -- 172.21.15.31:0/4209883669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558abf965610 con 0x558abf6210a0 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13b5ffb640 1 -- 172.21.15.31:0/4209883669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f13c4028040 con 0x558abf6210a0 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 20 client.15279 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf5ddc90 0x558abfa810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 4 client.15279 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.557+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "status"}) -- 0x558abf962a40 con 0x558abf5ddc90 2024-09-21T02:52:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13ce59c640 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf5ddc90 0x558abfa810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13ce59c640 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf5ddc90 0x558abfa810c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=384e8c3d7878bf55 server_cookie=e56915441645af1c in_seq=0 out_seq=0 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13b5ffb640 10 client.15279 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13b5ffb640 1 -- 172.21.15.31:0/4209883669 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558abf962a40 con 0x558abf5ddc90 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13b5ffb640 10 client.15279 handle_command_reply: tid=0 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 client.15279 shutdown 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 msgr2=0x558abfa0f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abfa0f040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f13c400ea90 tx=0x7f13c40214b0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 shutdown_connections 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558abf5ddc90 0x558abfa810c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558abf629280 0x558abfa0f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558abf649f60 0x558abfa7c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 --2- 172.21.15.31:0/4209883669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558abf6210a0 0x558abfa0f040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13b5ffb640 0 client.15279 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.558+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 >> 172.21.15.31:0/4209883669 conn(0x558abf63e350 msgr2=0x558abf9ed600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 shutdown_connections 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13db409740 1 -- 172.21.15.31:0/4209883669 wait complete. 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13db409740 20 client.15279 trim_cache size 0 max 16384 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24655, 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:23.568 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 84.156508939000005, 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 112.809696378 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:23.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f13b80b6590 msgr2=0x7f13b80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f13b80b6590 0x7f13b80b8aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13c80ad280 tx=0x7f13c807a860 comp rx=0 tx=0).stop 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 msgr2=0x7f13d019be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 0x7f13d019be60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f13c800cad0 tx=0x7f13c800cf90 comp rx=0 tx=0).stop 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 shutdown_connections 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f13b80b6590 0x7f13b80b8aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13d0108d20 0x7f13d019be60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d01083d0 0x7f13d019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 --2- 172.21.15.31:0/900776094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d0102ae0 0x7f13d019b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 >> 172.21.15.31:0/900776094 conn(0x7f13d0079760 msgr2=0x7f13d0101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 shutdown_connections 2024-09-21T02:52:23.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:23.559+0000 7f13d4e95640 1 -- 172.21.15.31:0/900776094 wait complete. 2024-09-21T02:52:24.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T02:52:24.500 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:24.921 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:24 smithi031 ceph-mon[31209]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T02:52:24.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:24 smithi165 ceph-mon[34818]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T02:52:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:24 smithi178 ceph-mon[34759]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T02:52:25.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 -- 172.21.15.31:0/3309046119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58102aa0 msgr2=0x7feb58102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3309046119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58102aa0 0x7feb58102eb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb4802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:25.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 -- 172.21.15.31:0/3309046119 shutdown_connections 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3309046119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb581083e0 0x7feb581087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3309046119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb58108d30 0x7feb5810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3309046119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58102aa0 0x7feb58102eb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 -- 172.21.15.31:0/3309046119 >> 172.21.15.31:0/3309046119 conn(0x7feb58079750 msgr2=0x7feb58079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 -- 172.21.15.31:0/3309046119 shutdown_connections 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.012+0000 7feb5d04e640 1 -- 172.21.15.31:0/3309046119 wait complete. 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.013+0000 7feb5d04e640 1 Processor -- start 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.013+0000 7feb5d04e640 1 -- start start 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.013+0000 7feb5d04e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb58102aa0 0x7feb5819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.013+0000 7feb5d04e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb581083e0 0x7feb5819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.013+0000 7feb5d04e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 0x7feb5819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5d04e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb581129f0 con 0x7feb58108d30 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5d04e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb58112870 con 0x7feb581083e0 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5d04e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb58112b70 con 0x7feb58102aa0 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 0x7feb5819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 0x7feb5819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36962/0 (socket says 172.21.15.31:36962) 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 -- 172.21.15.31:0/3357066566 learned_addr learned my addr 172.21.15.31:0/3357066566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5659c640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb581083e0 0x7feb5819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb56d9d640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb58102aa0 0x7feb5819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 -- 172.21.15.31:0/3357066566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb58102aa0 msgr2=0x7feb5819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb58102aa0 0x7feb5819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 -- 172.21.15.31:0/3357066566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb581083e0 msgr2=0x7feb5819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb581083e0 0x7feb5819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 -- 172.21.15.31:0/3357066566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb581a6550 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5759e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 0x7feb5819bd40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feb5000eaa0 tx=0x7feb5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb3ffff640 1 -- 172.21.15.31:0/3357066566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5000cdb0 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb3ffff640 1 -- 172.21.15.31:0/3357066566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb50005020 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb3ffff640 1 -- 172.21.15.31:0/3357066566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb500116d0 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.014+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb581a6840 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.015+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb581a6d30 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.017+0000 7feb635c2740 1 Processor -- start 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.017+0000 7feb3ffff640 1 -- 172.21.15.31:0/3357066566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7feb500051d0 con 0x7feb58108d30 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.017+0000 7feb635c2740 1 -- start start 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.017+0000 7feb635c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x5565273f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.017+0000 7feb635c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x5565273f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb635c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x5565273f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb635c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55652736ac10 con 0x55652702c280 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb635c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55652735a040 con 0x5565270240a0 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb635c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55652735de50 con 0x55652704cf60 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x5565273f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x5565273f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36966/0 (socket says 172.21.15.31:36966) 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 -- 172.21.15.31:0/1572603368 learned_addr learned my addr 172.21.15.31:0/1572603368 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb56d9d640 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x5565273f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x5565273f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb3ffff640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feb400b6590 0x7feb400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 -- 172.21.15.31:0/1572603368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 msgr2=0x5565273f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x5565273f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 -- 172.21.15.31:0/1572603368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 msgr2=0x5565273f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x5565273f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 -- 172.21.15.31:0/1572603368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565270b8540 con 0x55652702c280 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5759e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feb400b6590 0x7feb400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb5659c640 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x5565273f3980 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009f30 tx=0x7feb4c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13915f490bcb915 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb3ffff640 1 -- 172.21.15.31:0/3357066566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7feb500e7550 con 0x7feb58108d30 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.018+0000 7feb3d7fa640 1 -- 172.21.15.31:0/1572603368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c004050 con 0x55652702c280 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb3d7fa640 1 -- 172.21.15.31:0/1572603368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c0054d0 con 0x55652702c280 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb3d7fa640 1 -- 172.21.15.31:0/1572603368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c004a90 con 0x55652702c280 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 -- 172.21.15.31:0/1572603368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 msgr2=0x5565273f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x5565273f3980 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009f30 tx=0x7feb4c005850 comp rx=0 tx=0).stop 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb5759e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feb400b6590 0x7feb400b8aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb500ad390 tx=0x7feb500de080 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 -- 172.21.15.31:0/1572603368 shutdown_connections 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x5565273f3980 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x5565273f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 --2- 172.21.15.31:0/1572603368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x5565273f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.019+0000 7feb635c2740 1 -- 172.21.15.31:0/1572603368 >> 172.21.15.31:0/1572603368 conn(0x556527041350 msgr2=0x5565273f0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 -- 172.21.15.31:0/1572603368 shutdown_connections 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 -- 172.21.15.31:0/1572603368 wait complete. 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 -- start start 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 Processor -- start 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x556527402dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x556527403310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5565271bb5f0 con 0x55652702c280 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5565272f5010 con 0x5565270240a0 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb635c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556527405a40 con 0x55652704cf60 2024-09-21T02:52:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.020+0000 7feb56d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x556527403310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x556527403310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36970/0 (socket says 172.21.15.31:36970) 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 -- 172.21.15.31:0/2544901831 learned_addr learned my addr 172.21.15.31:0/2544901831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb635c2740 10 client.0 _ll_register_callbacks cb 0x556527354340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb5659c640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x556527402dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb5659c640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 -- 172.21.15.31:0/2544901831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 msgr2=0x55652740ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 -- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 msgr2=0x556527402dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x556527402dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 -- 172.21.15.31:0/2544901831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556527322a00 con 0x55652702c280 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb5659c640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb56d9d640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x556527403310 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb48005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db65aa8ef1adebdf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:25.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb3aff5640 1 -- 172.21.15.31:0/2544901831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48035040 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb3aff5640 1 -- 172.21.15.31:0/2544901831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7feb480317a0 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.021+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb4c0054d0 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb3aff5640 1 -- 172.21.15.31:0/2544901831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb48004040 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 10 client.15291 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556527411970 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb3aff5640 1 -- 172.21.15.31:0/2544901831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c0054d0 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb3aff5640 1 -- 172.21.15.31:0/2544901831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7feb4804d020 con 0x55652702c280 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 10 client.15291 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 10 client.15291 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 20 client.15291 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55652700b8e0 0x5565274153e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 4 client.15291 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55652736cb60 con 0x55652700b8e0 2024-09-21T02:52:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.022+0000 7feb5659c640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55652700b8e0 0x5565274153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.023+0000 7feb5659c640 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55652700b8e0 0x5565274153e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3733ca67758c3457 server_cookie=25321de365c87611 in_seq=0 out_seq=0 2024-09-21T02:52:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.023+0000 7feb3aff5640 10 client.15291 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb3aff5640 1 -- 172.21.15.31:0/2544901831 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55652736cb60 con 0x55652700b8e0 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb3aff5640 10 client.15291 handle_command_reply: tid=0 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 client.15291 shutdown 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 msgr2=0x556527403310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x556527403310 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb48005f00 comp rx=0 tx=0).stop 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 shutdown_connections 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55652700b8e0 0x5565274153e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652702c280 0x556527403310 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 --2- 172.21.15.31:0/2544901831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x556527402dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 >> 172.21.15.31:0/2544901831 conn(0x556527041350 msgr2=0x5565273ef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb3aff5640 0 client.15291 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 shutdown_connections 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 1 -- 172.21.15.31:0/2544901831 wait complete. 2024-09-21T02:52:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.024+0000 7feb635c2740 20 client.15291 trim_cache size 0 max 16384 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.042+0000 7feb635c2740 1 Processor -- start 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.042+0000 7feb635c2740 1 -- start start 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb635c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 0x5565273f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb635c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55652702c280 0x55652740ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb635c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb635c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556527413720 con 0x5565270240a0 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb635c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556527413df0 con 0x55652702c280 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb635c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5565274144c0 con 0x55652704cf60 2024-09-21T02:52:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 0x5565273f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 0x5565273f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36982/0 (socket says 172.21.15.31:36982) 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 -- 172.21.15.31:0/3031026663 learned_addr learned my addr 172.21.15.31:0/3031026663 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb56d9d640 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55652702c280 0x55652740ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 -- 172.21.15.31:0/3031026663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 msgr2=0x55652740f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 -- 172.21.15.31:0/3031026663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55652702c280 msgr2=0x55652740ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55652702c280 0x55652740ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 -- 172.21.15.31:0/3031026663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565273f57f0 con 0x5565270240a0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.043+0000 7feb5659c640 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 0x5565273f00d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009fd0 tx=0x7feb4c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc865cf1408013a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb3cff9640 1 -- 172.21.15.31:0/3031026663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0055b0 con 0x5565270240a0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb3cff9640 1 -- 172.21.15.31:0/3031026663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c003aa0 con 0x5565270240a0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb3cff9640 1 -- 172.21.15.31:0/3031026663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c02cac0 con 0x5565270240a0 2024-09-21T02:52:25.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 -- 172.21.15.31:0/3031026663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 msgr2=0x5565273f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 0x5565273f00d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009fd0 tx=0x7feb4c009310 comp rx=0 tx=0).stop 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 -- 172.21.15.31:0/3031026663 shutdown_connections 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652704cf60 0x55652740f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55652702c280 0x55652740ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 --2- 172.21.15.31:0/3031026663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5565270240a0 0x5565273f00d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 -- 172.21.15.31:0/3031026663 >> 172.21.15.31:0/3031026663 conn(0x556527041350 msgr2=0x5565273f0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 -- 172.21.15.31:0/3031026663 shutdown_connections 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.044+0000 7feb635c2740 1 -- 172.21.15.31:0/3031026663 wait complete. 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 -- start start 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 Processor -- start 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x55652741d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652702c280 0x55652741dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 0x556527416fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556527415260 con 0x55652704cf60 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556527414b90 con 0x5565270240a0 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556527415930 con 0x55652702c280 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb635c2740 10 client.0 _ll_register_callbacks cb 0x7feb48009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb5659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 0x556527416fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb5659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 0x556527416fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36996/0 (socket says 172.21.15.31:36996) 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb5659c640 1 -- 172.21.15.31:0/2171182066 learned_addr learned my addr 172.21.15.31:0/2171182066 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.045+0000 7feb56d9d640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652702c280 0x55652741dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x55652741d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 -- 172.21.15.31:0/2171182066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652702c280 msgr2=0x55652741dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652702c280 0x55652741dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 -- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 msgr2=0x55652741d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x55652741d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 -- 172.21.15.31:0/2171182066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556527417740 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb5659c640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 0x556527416fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009de0 tx=0x7feb4c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc45b32f96db30c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb3aff5640 1 -- 172.21.15.31:0/2171182066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0053b0 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb4c003aa0 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb3aff5640 1 -- 172.21.15.31:0/2171182066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7feb4c005550 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb3aff5640 1 -- 172.21.15.31:0/2171182066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb4c0035f0 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb635c2740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.046+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55652741d0f0 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb3aff5640 1 -- 172.21.15.31:0/2171182066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c003aa0 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb3aff5640 1 -- 172.21.15.31:0/2171182066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7feb4c030070 con 0x55652704cf60 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb635c2740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:25.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb635c2740 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T02:52:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb635c2740 20 client.15297 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb635c2740 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556526fe0c90 0x556527483540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb635c2740 4 client.15297 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:25.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "status"}) -- 0x556527368610 con 0x556526fe0c90 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.047+0000 7feb56d9d640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556526fe0c90 0x556527483540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb3aff5640 10 client.15297 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb56d9d640 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556526fe0c90 0x556527483540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a3148b3ca737c8d server_cookie=ada77c8ae727e07f in_seq=0 out_seq=0 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb3aff5640 1 -- 172.21.15.31:0/2171182066 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556527368610 con 0x556526fe0c90 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb3aff5640 10 client.15297 handle_command_reply: tid=0 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 client.15297 shutdown 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 msgr2=0x556527416fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 0x556527416fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009de0 tx=0x7feb4c005970 comp rx=0 tx=0).stop 2024-09-21T02:52:25.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 shutdown_connections 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556526fe0c90 0x556527483540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55652702c280 0x55652741dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55652704cf60 0x556527416fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 --2- 172.21.15.31:0/2171182066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565270240a0 0x55652741d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 >> 172.21.15.31:0/2171182066 conn(0x556527041350 msgr2=0x5565273fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb3aff5640 0 client.15297 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 shutdown_connections 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 1 -- 172.21.15.31:0/2171182066 wait complete. 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.048+0000 7feb635c2740 20 client.15297 trim_cache size 0 max 16384 2024-09-21T02:52:25.055 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24659, 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 86.658100560999998, 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 116.838448222 2024-09-21T02:52:25.056 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feb400b6590 msgr2=0x7feb400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feb400b6590 0x7feb400b8aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb500ad390 tx=0x7feb500de080 comp rx=0 tx=0).stop 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 msgr2=0x7feb5819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 0x7feb5819bd40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feb5000eaa0 tx=0x7feb5000ef60 comp rx=0 tx=0).stop 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 shutdown_connections 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feb400b6590 0x7feb400b8aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb58108d30 0x7feb5819bd40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb581083e0 0x7feb5819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 --2- 172.21.15.31:0/3357066566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb58102aa0 0x7feb5819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 >> 172.21.15.31:0/3357066566 conn(0x7feb58079750 msgr2=0x7feb58106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 shutdown_connections 2024-09-21T02:52:25.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:25.049+0000 7feb5d04e640 1 -- 172.21.15.31:0/3357066566 wait complete. 2024-09-21T02:52:25.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T02:52:26.148 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:26.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 -- 172.21.15.31:0/603262297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f788010de30 msgr2=0x7f7880110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 --2- 172.21.15.31:0/603262297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f788010de30 0x7f7880110300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7878009ca0 tx=0x7f787802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 -- 172.21.15.31:0/603262297 shutdown_connections 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 --2- 172.21.15.31:0/603262297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78801052b0 0x7f788010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 --2- 172.21.15.31:0/603262297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f788010de30 0x7f7880110300 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 --2- 172.21.15.31:0/603262297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7880104880 0x7f7880104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 -- 172.21.15.31:0/603262297 >> 172.21.15.31:0/603262297 conn(0x7f7880079730 msgr2=0x7f7880079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 -- 172.21.15.31:0/603262297 shutdown_connections 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 -- 172.21.15.31:0/603262297 wait complete. 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.658+0000 7f78869da640 1 Processor -- start 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 -- start start 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 0x7f7880102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78801052b0 0x7f7880102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788010de30 0x7f7880103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7880113a70 con 0x7f7880104880 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78801138f0 con 0x7f78801052b0 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f78869da640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7880113bf0 con 0x7f788010de30 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 0x7f7880102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 0x7f7880102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37006/0 (socket says 172.21.15.31:37006) 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 -- 172.21.15.31:0/3681487735 learned_addr learned my addr 172.21.15.31:0/3681487735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f7884f77640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788010de30 0x7f7880103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787f7fe640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78801052b0 0x7f7880102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 -- 172.21.15.31:0/3681487735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788010de30 msgr2=0x7f7880103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788010de30 0x7f7880103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 -- 172.21.15.31:0/3681487735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78801052b0 msgr2=0x7f7880102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78801052b0 0x7f7880102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.659+0000 7f787ffff640 1 -- 172.21.15.31:0/3681487735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78801a86a0 con 0x7f7880104880 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f787f7fe640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78801052b0 0x7f7880102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f787ffff640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 0x7f7880102790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f787000d950 tx=0x7f787000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f787d7fa640 1 -- 172.21.15.31:0/3681487735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870017020 con 0x7f7880104880 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78801a8990 con 0x7f7880104880 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f787d7fa640 1 -- 172.21.15.31:0/3681487735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7870009c30 con 0x7f7880104880 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f787d7fa640 1 -- 172.21.15.31:0/3681487735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000f690 con 0x7f7880104880 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.660+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78801a8ed0 con 0x7f7880104880 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.663+0000 7f788cf4e740 1 Processor -- start 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.663+0000 7f788cf4e740 1 -- start start 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d268af60 0x5602d2a30f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a31440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a31980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f788cf4e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5602d29a8c10 con 0x5602d26620a0 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f788cf4e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602d2998040 con 0x5602d266a280 2024-09-21T02:52:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f788cf4e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602d299be50 con 0x5602d268af60 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787d7fa640 1 -- 172.21.15.31:0/3681487735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f787001a020 con 0x7f7880104880 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a31980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a31980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:60796/0 (socket says 172.21.15.31:60796) 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 -- 172.21.15.31:0/1952028665 learned_addr learned my addr 172.21.15.31:0/1952028665 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f7884f77640 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d268af60 0x5602d2a30f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787ffff640 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a31440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 -- 172.21.15.31:0/1952028665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d268af60 msgr2=0x5602d2a30f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d268af60 0x5602d2a30f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 -- 172.21.15.31:0/1952028665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 msgr2=0x5602d2a31440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a31440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.664+0000 7f787f7fe640 1 -- 172.21.15.31:0/1952028665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602d26f6540 con 0x5602d266a280 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f7884f77640 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d268af60 0x5602d2a30f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f787f7fe640 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a31980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f787400dc00 tx=0x7f787400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe3c77b46ed4528 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f787d7fa640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78680b6560 0x7f78680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f7884f77640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78680b6560 0x7f78680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f78667fc640 1 -- 172.21.15.31:0/1952028665 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874005020 con 0x5602d266a280 2024-09-21T02:52:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f787d7fa640 1 -- 172.21.15.31:0/3681487735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f78700e7400 con 0x7f7880104880 2024-09-21T02:52:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f78667fc640 1 -- 172.21.15.31:0/1952028665 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78740047a0 con 0x5602d266a280 2024-09-21T02:52:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.665+0000 7f78667fc640 1 -- 172.21.15.31:0/1952028665 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740116a0 con 0x5602d266a280 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 -- 172.21.15.31:0/1952028665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 msgr2=0x5602d2a31980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a31980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f787400dc00 tx=0x7f787400f730 comp rx=0 tx=0).stop 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f7884f77640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78680b6560 0x7f78680b8a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f787802f500 tx=0x7f7878009820 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 -- 172.21.15.31:0/1952028665 shutdown_connections 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a31980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a31440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1952028665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d268af60 0x5602d2a30f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 -- 172.21.15.31:0/1952028665 >> 172.21.15.31:0/1952028665 conn(0x5602d267f350 msgr2=0x5602d2a2edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 -- 172.21.15.31:0/1952028665 shutdown_connections 2024-09-21T02:52:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 -- 172.21.15.31:0/1952028665 wait complete. 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.666+0000 7f788cf4e740 1 -- start start 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 Processor -- start 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a3c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a3cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a48a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5602d27f95f0 con 0x5602d268af60 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602d2933010 con 0x5602d266a280 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a3f6e0 con 0x5602d26620a0 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f788cf4e740 10 client.0 _ll_register_callbacks cb 0x5602d2992340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f7884f77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a48a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f7884f77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a48a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37030/0 (socket says 172.21.15.31:37030) 2024-09-21T02:52:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f7884f77640 1 -- 172.21.15.31:0/1707705546 learned_addr learned my addr 172.21.15.31:0/1707705546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f787ffff640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a3c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.667+0000 7f787f7fe640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a3cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f7884f77640 1 -- 172.21.15.31:0/1707705546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 msgr2=0x5602d2a3c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f7884f77640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a3c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f7884f77640 1 -- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 msgr2=0x5602d2a3cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f7884f77640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a3cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f7884f77640 1 -- 172.21.15.31:0/1707705546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602d2960a00 con 0x5602d268af60 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f7884f77640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a48a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7878035000 tx=0x7f787802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ae6533ca3556a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f785f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f785f7fe640 1 -- 172.21.15.31:0/1707705546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878003cc0 con 0x5602d268af60 2024-09-21T02:52:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78740047a0 con 0x5602d268af60 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f785f7fe640 1 -- 172.21.15.31:0/1707705546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7878004040 con 0x5602d268af60 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f785f7fe640 1 -- 172.21.15.31:0/1707705546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7878003e60 con 0x5602d268af60 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f788cf4e740 10 client.15309 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602d2a4b580 con 0x5602d268af60 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.668+0000 7f785f7fe640 1 -- 172.21.15.31:0/1707705546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78740047a0 con 0x5602d268af60 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f785f7fe640 1 -- 172.21.15.31:0/1707705546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f787805a070 con 0x5602d268af60 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f788cf4e740 10 client.15309 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f788cf4e740 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f788cf4e740 20 client.15309 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d26498e0 0x5602d2a4f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f788cf4e740 4 client.15309 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602d29aab60 con 0x5602d26498e0 2024-09-21T02:52:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.669+0000 7f787ffff640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d26498e0 0x5602d2a4f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.670+0000 7f785f7fe640 10 client.15309 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.670+0000 7f787ffff640 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d26498e0 0x5602d2a4f0b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1356db3e094c6f69 server_cookie=b3e8a51b8dbf922e in_seq=0 out_seq=0 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.670+0000 7f785f7fe640 1 -- 172.21.15.31:0/1707705546 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602d29aab60 con 0x5602d26498e0 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.670+0000 7f785f7fe640 10 client.15309 handle_command_reply: tid=0 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 client.15309 shutdown 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 msgr2=0x5602d2a48a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a48a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7878035000 tx=0x7f787802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 shutdown_connections 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d26498e0 0x5602d2a4f0b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a48a30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a3cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f785f7fe640 0 client.15309 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 --2- 172.21.15.31:0/1707705546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a3c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 >> 172.21.15.31:0/1707705546 conn(0x5602d267f350 msgr2=0x5602d2a34090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 shutdown_connections 2024-09-21T02:52:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 1 -- 172.21.15.31:0/1707705546 wait complete. 2024-09-21T02:52:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.671+0000 7f788cf4e740 20 client.15309 trim_cache size 0 max 16384 2024-09-21T02:52:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.689+0000 7f788cf4e740 1 Processor -- start 2024-09-21T02:52:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.689+0000 7f788cf4e740 1 -- start start 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a37020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d266a280 0x5602d2a489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d268af60 0x5602d2a48ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f788cf4e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a4d3f0 con 0x5602d26620a0 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f788cf4e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a4dac0 con 0x5602d268af60 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f788cf4e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a4e190 con 0x5602d266a280 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a37020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a37020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37044/0 (socket says 172.21.15.31:37044) 2024-09-21T02:52:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 -- 172.21.15.31:0/4269018384 learned_addr learned my addr 172.21.15.31:0/4269018384 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f7884f77640 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d268af60 0x5602d2a48ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787f7fe640 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d266a280 0x5602d2a489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 -- 172.21.15.31:0/4269018384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d266a280 msgr2=0x5602d2a489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d266a280 0x5602d2a489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 -- 172.21.15.31:0/4269018384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d268af60 msgr2=0x5602d2a48ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d268af60 0x5602d2a48ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 -- 172.21.15.31:0/4269018384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602d2a33d60 con 0x5602d26620a0 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f7884f77640 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d268af60 0x5602d2a48ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.690+0000 7f787ffff640 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a37020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f78700ad420 tx=0x7f7870010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10418ae9bd07f652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f7865ffb640 1 -- 172.21.15.31:0/4269018384 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700acfa0 con 0x5602d26620a0 2024-09-21T02:52:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f7865ffb640 1 -- 172.21.15.31:0/4269018384 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7870009de0 con 0x5602d26620a0 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f7865ffb640 1 -- 172.21.15.31:0/4269018384 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700a7b90 con 0x5602d26620a0 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 -- 172.21.15.31:0/4269018384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 msgr2=0x5602d2a37020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a37020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f78700ad420 tx=0x7f7870010d70 comp rx=0 tx=0).stop 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 -- 172.21.15.31:0/4269018384 shutdown_connections 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d266a280 0x5602d2a489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d268af60 0x5602d2a48ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 --2- 172.21.15.31:0/4269018384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d26620a0 0x5602d2a37020 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.691+0000 7f788cf4e740 1 -- 172.21.15.31:0/4269018384 >> 172.21.15.31:0/4269018384 conn(0x5602d267f350 msgr2=0x5602d2a4c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 -- 172.21.15.31:0/4269018384 shutdown_connections 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 -- 172.21.15.31:0/4269018384 wait complete. 2024-09-21T02:52:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 -- start start 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 Processor -- start 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a57b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a50ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a51530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a4ef30 con 0x5602d268af60 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a4e860 con 0x5602d266a280 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.692+0000 7f788cf4e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602d2a4f600 con 0x5602d26620a0 2024-09-21T02:52:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f788cf4e740 10 client.0 _ll_register_callbacks cb 0x7f78780099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f7884f77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a51530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a57b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f7884f77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a51530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37054/0 (socket says 172.21.15.31:37054) 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f7884f77640 1 -- 172.21.15.31:0/3074286524 learned_addr learned my addr 172.21.15.31:0/3074286524 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787f7fe640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a50ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 -- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 msgr2=0x5602d2a50ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a50ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 -- 172.21.15.31:0/3074286524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 msgr2=0x5602d2a51530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a51530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 -- 172.21.15.31:0/3074286524 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602d2a51cc0 con 0x5602d26620a0 2024-09-21T02:52:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f7884f77640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a51530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f787ffff640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a57b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5602d29aacb0 tx=0x7f7870010b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a08b00098935911d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.693+0000 7f785f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:52:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.697+0000 7f785f7fe640 1 -- 172.21.15.31:0/3074286524 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700acfa0 con 0x5602d26620a0 2024-09-21T02:52:26.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.697+0000 7f785f7fe640 1 -- 172.21.15.31:0/3074286524 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7870094dd0 con 0x5602d26620a0 2024-09-21T02:52:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.697+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7870009de0 con 0x5602d26620a0 2024-09-21T02:52:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.698+0000 7f785f7fe640 1 -- 172.21.15.31:0/3074286524 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f78700b1f70 con 0x5602d26620a0 2024-09-21T02:52:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.698+0000 7f788cf4e740 10 client.25051 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:26.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.698+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602d2aba580 con 0x5602d26620a0 2024-09-21T02:52:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f785f7fe640 1 -- 172.21.15.31:0/3074286524 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7870009de0 con 0x5602d26620a0 2024-09-21T02:52:26.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f785f7fe640 1 -- 172.21.15.31:0/3074286524 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7878031d20 con 0x5602d26620a0 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f788cf4e740 10 client.25051 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f788cf4e740 10 client.25051 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f788cf4e740 20 client.25051 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f788cf4e740 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d261ec90 0x5602d2abd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f788cf4e740 4 client.25051 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "status"}) -- 0x5602d29a6610 con 0x5602d261ec90 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.699+0000 7f787f7fe640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d261ec90 0x5602d2abd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f785f7fe640 10 client.25051 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f787f7fe640 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d261ec90 0x5602d2abd850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5475a2389c07fd4 server_cookie=f09a47127efacbc8 in_seq=0 out_seq=0 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f785f7fe640 1 -- 172.21.15.31:0/3074286524 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5602d29a6610 con 0x5602d261ec90 2024-09-21T02:52:26.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f785f7fe640 10 client.25051 handle_command_reply: tid=0 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 client.25051 shutdown 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 msgr2=0x5602d2a57b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a57b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5602d29aacb0 tx=0x7f7870010b10 comp rx=0 tx=0).stop 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 shutdown_connections 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5602d261ec90 0x5602d2abd850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5602d268af60 0x5602d2a51530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602d266a280 0x5602d2a50ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f785f7fe640 0 client.25051 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 --2- 172.21.15.31:0/3074286524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602d26620a0 0x5602d2a57b40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 >> 172.21.15.31:0/3074286524 conn(0x5602d267f350 msgr2=0x5602d2a3c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:26.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 shutdown_connections 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.700+0000 7f788cf4e740 1 -- 172.21.15.31:0/3074286524 wait complete. 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f788cf4e740 20 client.25051 trim_cache size 0 max 16384 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24695, 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:26.706 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 95.344947640000001, 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 106.61103843799999 2024-09-21T02:52:26.707 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:26.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78680b6560 msgr2=0x7f78680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78680b6560 0x7f78680b8a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f787802f500 tx=0x7f7878009820 comp rx=0 tx=0).stop 2024-09-21T02:52:26.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 msgr2=0x7f7880102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:26.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 0x7f7880102790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f787000d950 tx=0x7f787000de10 comp rx=0 tx=0).stop 2024-09-21T02:52:26.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 shutdown_connections 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78680b6560 0x7f78680b8a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788010de30 0x7f7880103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78801052b0 0x7f7880102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 --2- 172.21.15.31:0/3681487735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7880104880 0x7f7880102790 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 >> 172.21.15.31:0/3681487735 conn(0x7f7880079730 msgr2=0x7f7880107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 shutdown_connections 2024-09-21T02:52:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:26.701+0000 7f78869da640 1 -- 172.21.15.31:0/3681487735 wait complete. 2024-09-21T02:52:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:26 smithi031 ceph-mon[31209]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T02:52:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:26 smithi178 ceph-mon[34759]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T02:52:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:26 smithi165 ceph-mon[34818]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T02:52:27.663 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T02:52:27.663 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T02:52:27.672 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T02:52:27.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T02:52:27.937 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 -- 172.21.15.31:0/1355828962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c10de10 msgr2=0x7f950c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 --2- 172.21.15.31:0/1355828962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c10de10 0x7f950c110300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009900 tx=0x7f94fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 -- 172.21.15.31:0/1355828962 shutdown_connections 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 --2- 172.21.15.31:0/1355828962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c105300 0x7f950c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 --2- 172.21.15.31:0/1355828962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c10de10 0x7f950c110300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 --2- 172.21.15.31:0/1355828962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f950c1049b0 0x7f950c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 -- 172.21.15.31:0/1355828962 >> 172.21.15.31:0/1355828962 conn(0x7f950c079730 msgr2=0x7f950c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 -- 172.21.15.31:0/1355828962 shutdown_connections 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.475+0000 7f951449a640 1 -- 172.21.15.31:0/1355828962 wait complete. 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 Processor -- start 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 -- start start 2024-09-21T02:52:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f950c1049b0 0x7f950c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 0x7f950c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c10de10 0x7f950c19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f950c113b10 con 0x7f950c105300 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f950c113990 con 0x7f950c1049b0 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f951449a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f950c113c90 con 0x7f950c10de10 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f9511a35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 0x7f950c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f9511a35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 0x7f950c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37068/0 (socket says 172.21.15.31:37068) 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f9511a35640 1 -- 172.21.15.31:0/3245170918 learned_addr learned my addr 172.21.15.31:0/3245170918 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.476+0000 7f9512a37640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c10de10 0x7f950c19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9512236640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f950c1049b0 0x7f950c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9511a35640 1 -- 172.21.15.31:0/3245170918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c10de10 msgr2=0x7f950c19fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9511a35640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c10de10 0x7f950c19fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9511a35640 1 -- 172.21.15.31:0/3245170918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f950c1049b0 msgr2=0x7f950c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9511a35640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f950c1049b0 0x7f950c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9511a35640 1 -- 172.21.15.31:0/3245170918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950c1a04d0 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9512a37640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c10de10 0x7f950c19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f9511a35640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 0x7f950c19abc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f950000c8c0 tx=0x7f950000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500007bb0 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950c1a07c0 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9500007d50 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000187c0 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.477+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950c1a6fe0 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.478+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950c104dc0 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.480+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f950000fa20 con 0x7f950c105300 2024-09-21T02:52:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.480+0000 7f950b7fe640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94f40b6590 0x7f94f40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.480+0000 7f9512236640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94f40b6590 0x7f94f40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.480+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f95000e7840 con 0x7f950c105300 2024-09-21T02:52:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.481+0000 7f9512236640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94f40b6590 0x7f94f40b8aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f950c06bc60 tx=0x7f95040072f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:28.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.484+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9500014030 con 0x7f950c105300 2024-09-21T02:52:28.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.691+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f950c063800 con 0x7f94f40b6590 2024-09-21T02:52:28.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.692+0000 7f950b7fe640 1 -- 172.21.15.31:0/3245170918 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f950c063800 con 0x7f94f40b6590 2024-09-21T02:52:28.693 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-09-21T02:52:28.693 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-09-21T02:52:28.693 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-09-21T02:52:28.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94f40b6590 msgr2=0x7f94f40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94f40b6590 0x7f94f40b8aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f950c06bc60 tx=0x7f95040072f0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 msgr2=0x7f950c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 0x7f950c19abc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f950000c8c0 tx=0x7f950000cd80 comp rx=0 tx=0).stop 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 shutdown_connections 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94f40b6590 0x7f94f40b8aa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f950c10de10 0x7f950c19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f950c105300 0x7f950c19abc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 --2- 172.21.15.31:0/3245170918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f950c1049b0 0x7f950c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 >> 172.21.15.31:0/3245170918 conn(0x7f950c079730 msgr2=0x7f950c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 shutdown_connections 2024-09-21T02:52:28.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:28.694+0000 7f951449a640 1 -- 172.21.15.31:0/3245170918 wait complete. 2024-09-21T02:52:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:28 smithi178 ceph-mon[34759]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T02:52:28.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:28 smithi031 ceph-mon[31209]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T02:52:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:28 smithi165 ceph-mon[34818]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T02:52:29.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T02:52:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:29.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:29 smithi031 ceph-mon[31209]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:29.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:29 smithi178 ceph-mon[34759]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:30.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:29 smithi165 ceph-mon[34818]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:30.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.261+0000 7fbc108b6640 1 -- 172.21.15.31:0/1902913552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c108d10 msgr2=0x7fbc0c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.261+0000 7fbc108b6640 1 --2- 172.21.15.31:0/1902913552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c108d10 0x7fbc0c10f540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009ca0 tx=0x7fbc0402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 -- 172.21.15.31:0/1902913552 shutdown_connections 2024-09-21T02:52:30.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 --2- 172.21.15.31:0/1902913552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c108d10 0x7fbc0c10f540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 --2- 172.21.15.31:0/1902913552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 --2- 172.21.15.31:0/1902913552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc0c102a80 0x7fbc0c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 -- 172.21.15.31:0/1902913552 >> 172.21.15.31:0/1902913552 conn(0x7fbc0c079740 msgr2=0x7fbc0c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:30.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 -- 172.21.15.31:0/1902913552 shutdown_connections 2024-09-21T02:52:30.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 -- 172.21.15.31:0/1902913552 wait complete. 2024-09-21T02:52:30.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.262+0000 7fbc108b6640 1 Processor -- start 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 -- start start 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc0c102a80 0x7fbc0c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0c108d10 0x7fbc0c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c112980 con 0x7fbc0c1083c0 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c112800 con 0x7fbc0c108d10 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc108b6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c112b00 con 0x7fbc0c102a80 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36066/0 (socket says 172.21.15.31:36066) 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 -- 172.21.15.31:0/2746669941 learned_addr learned my addr 172.21.15.31:0/2746669941 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc0a59c640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc0c102a80 0x7fbc0c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:30.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 -- 172.21.15.31:0/2746669941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc0c102a80 msgr2=0x7fbc0c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc0c102a80 0x7fbc0c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 -- 172.21.15.31:0/2746669941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0c108d10 msgr2=0x7fbc0c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc0ad9d640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0c108d10 0x7fbc0c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0c108d10 0x7fbc0c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.263+0000 7fbc09d9b640 1 -- 172.21.15.31:0/2746669941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c1a0330 con 0x7fbc0c1083c0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.264+0000 7fbc09d9b640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c19aa20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000c970 tx=0x7fbc0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.264+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00014070 con 0x7fbc0c1083c0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.264+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc00004440 con 0x7fbc0c1083c0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.264+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c1a0620 con 0x7fbc0c1083c0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.264+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00003e50 con 0x7fbc0c1083c0 2024-09-21T02:52:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.264+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1a6e40 con 0x7fbc0c1083c0 2024-09-21T02:52:30.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.265+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c103d00 con 0x7fbc0c1083c0 2024-09-21T02:52:30.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.266+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fbc00017070 con 0x7fbc0c1083c0 2024-09-21T02:52:30.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.267+0000 7fbbfb7fe640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbf40b6730 0x7fbbf40b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.267+0000 7fbc0a59c640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbf40b6730 0x7fbbf40b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.267+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbc000e7740 con 0x7fbc0c1083c0 2024-09-21T02:52:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.268+0000 7fbc0a59c640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbf40b6730 0x7fbbf40b8c40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0058c0 tx=0x7fbbfc00a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:30.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.270+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbc0001e030 con 0x7fbc0c1083c0 2024-09-21T02:52:30.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.468+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbc0c10bb60 con 0x7fbbf40b6730 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.480+0000 7fbbfb7fe640 1 -- 172.21.15.31:0/2746669941 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fbbfc00fb60 con 0x7fbbf40b6730 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (113s) 22s ago 112s 87.0M - 19.3.0-5120-g18432218 ddce307e1f32 f3a818248bed 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi165 running (110s) 45s ago 110s 94.2M - 19.3.0-5120-g18432218 ddce307e1f32 49ce0d5da4b8 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi178 running (2m) 45s ago 2m 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 abf086330aad 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (2m) 22s ago 2m 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 5f616260a6d9 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi165 running (2m) 45s ago 2m 85.8M - 19.3.0-5120-g18432218 ddce307e1f32 34ee203c2021 2024-09-21T02:52:30.482 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi178 running (2m) 45s ago 2m 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 3de6889f53ff 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (116s) 22s ago 116s 84.1M - 19.3.0-5120-g18432218 ddce307e1f32 49d3dbaf1837 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi165 running (2m) 45s ago 2m 88.3M - 19.3.0-5120-g18432218 ddce307e1f32 4d0682e3266a 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi178 running (2m) 45s ago 119s 88.0M - 19.3.0-5120-g18432218 ddce307e1f32 17789cc7aef8 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (2m) 22s ago 2m 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 1ad677833a18 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi165 running (2m) 45s ago 2m 84.8M - 19.3.0-5120-g18432218 ddce307e1f32 392f53cb3143 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi178 running (2m) 45s ago 2m 85.1M - 19.3.0-5120-g18432218 ddce307e1f32 4031152f76c1 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (10m) 22s ago 10m 586M - 19.3.0-5120-g18432218 ddce307e1f32 1b306d23b166 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi165 *:8443,8765 running (8m) 45s ago 8m 524M - 19.3.0-5120-g18432218 ddce307e1f32 41c6c7592df2 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi178 *:8443,8765 running (8m) 45s ago 8m 526M - 19.3.0-5120-g18432218 ddce307e1f32 9fc30afc7849 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (10m) 22s ago 10m 123M 2048M 19.3.0-5120-g18432218 ddce307e1f32 ecef9b21124f 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi165 running (8m) 45s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 b9f87f32b7ce 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi178 running (8m) 45s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 cd588d01e868 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 22s ago 7m 115M 4273M 19.3.0-5120-g18432218 ddce307e1f32 df18da783d6d 2024-09-21T02:52:30.483 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi165 running (6m) 45s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3bef73e04cb3 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi178 running (6m) 45s ago 6m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 824a66cda9e2 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (6m) 22s ago 6m 121M 4273M 19.3.0-5120-g18432218 ddce307e1f32 e2fa7add2ccb 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi165 running (5m) 45s ago 5m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 e61c6596732a 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi178 running (5m) 45s ago 5m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1c3f733fc592 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (4m) 22s ago 4m 117M 4273M 19.3.0-5120-g18432218 ddce307e1f32 df30cc4ea2ef 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi165 running (4m) 45s ago 4m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 98aeeb769708 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi178 running (3m) 45s ago 3m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ff40fd1a44d7 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 22s ago 3m 121M 4273M 19.3.0-5120-g18432218 ddce307e1f32 c87a663ca623 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi165 running (3m) 45s ago 3m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 2260c719d9d7 2024-09-21T02:52:30.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi178 running (2m) 45s ago 2m 107M 4272M 19.3.0-5120-g18432218 ddce307e1f32 032ecd819b44 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.482+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbf40b6730 msgr2=0x7fbbf40b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.482+0000 7fbc108b6640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbf40b6730 0x7fbbf40b8c40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0058c0 tx=0x7fbbfc00a250 comp rx=0 tx=0).stop 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.482+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 msgr2=0x7fbc0c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.482+0000 7fbc108b6640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c19aa20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000c970 tx=0x7fbc0000ce30 comp rx=0 tx=0).stop 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 shutdown_connections 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbf40b6730 0x7fbbf40b8c40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0c108d10 0x7fbc0c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc0c1083c0 0x7fbc0c19aa20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 --2- 172.21.15.31:0/2746669941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc0c102a80 0x7fbc0c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 >> 172.21.15.31:0/2746669941 conn(0x7fbc0c079740 msgr2=0x7fbc0c06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 shutdown_connections 2024-09-21T02:52:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:30.483+0000 7fbc108b6640 1 -- 172.21.15.31:0/2746669941 wait complete. 2024-09-21T02:52:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:30 smithi178 ceph-mon[34759]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T02:52:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:30 smithi031 ceph-mon[31209]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T02:52:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:30 smithi165 ceph-mon[34818]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T02:52:31.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T02:52:31.456 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:31.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:31 smithi031 ceph-mon[31209]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:31.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:31 smithi178 ceph-mon[34759]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:31.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 -- 172.21.15.31:0/674087248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114104840 msgr2=0x7ff114104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:31.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 --2- 172.21.15.31:0/674087248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114104840 0x7ff114104c50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff104009900 tx=0x7ff10402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:31.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 -- 172.21.15.31:0/674087248 shutdown_connections 2024-09-21T02:52:31.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 --2- 172.21.15.31:0/674087248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff11410ddf0 0x7ff114110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:31.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 --2- 172.21.15.31:0/674087248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114105270 0x7ff11410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:31.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 --2- 172.21.15.31:0/674087248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114104840 0x7ff114104c50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:31.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 -- 172.21.15.31:0/674087248 >> 172.21.15.31:0/674087248 conn(0x7ff114079730 msgr2=0x7ff114079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:31.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 -- 172.21.15.31:0/674087248 shutdown_connections 2024-09-21T02:52:31.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.980+0000 7ff11986b640 1 -- 172.21.15.31:0/674087248 wait complete. 2024-09-21T02:52:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 Processor -- start 2024-09-21T02:52:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 -- start start 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114104840 0x7ff11419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff114105270 0x7ff11419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 0x7ff11419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff114113a70 con 0x7ff11410ddf0 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141138f0 con 0x7ff114104840 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff11986b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff114113bf0 con 0x7ff114105270 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 0x7ff11419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 0x7ff11419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36078/0 (socket says 172.21.15.31:36078) 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff1137fe640 1 -- 172.21.15.31:0/3557414144 learned_addr learned my addr 172.21.15.31:0/3557414144 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff1127fc640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff114105270 0x7ff11419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff112ffd640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114104840 0x7ff11419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff1137fe640 1 -- 172.21.15.31:0/3557414144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff114105270 msgr2=0x7ff11419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.981+0000 7ff1137fe640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff114105270 0x7ff11419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff1137fe640 1 -- 172.21.15.31:0/3557414144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114104840 msgr2=0x7ff11419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff1137fe640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114104840 0x7ff11419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff1137fe640 1 -- 172.21.15.31:0/3557414144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1141a0410 con 0x7ff11410ddf0 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff112ffd640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114104840 0x7ff11419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff1137fe640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 0x7ff11419fda0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00eaa0 tx=0x7ff10c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00cdb0 con 0x7ff11410ddf0 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1141a0700 con 0x7ff11410ddf0 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff10c005020 con 0x7ff11410ddf0 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.982+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0116d0 con 0x7ff11410ddf0 2024-09-21T02:52:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.983+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a6f20 con 0x7ff11410ddf0 2024-09-21T02:52:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.984+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11406b500 con 0x7ff11410ddf0 2024-09-21T02:52:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.985+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7ff10c0054f0 con 0x7ff11410ddf0 2024-09-21T02:52:31.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.986+0000 7ff118869640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff0fc0b6590 0x7ff0fc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:31.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.986+0000 7ff112ffd640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff0fc0b6590 0x7ff0fc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:31.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.986+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff10c0e72d0 con 0x7ff11410ddf0 2024-09-21T02:52:31.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.986+0000 7ff112ffd640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff0fc0b6590 0x7ff0fc0b8aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1040096c0 tx=0x7ff104031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:31.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:31.990+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff10c016030 con 0x7ff11410ddf0 2024-09-21T02:52:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.187+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff114063800 con 0x7ff0fc0b6590 2024-09-21T02:52:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:31 smithi165 ceph-mon[34818]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:32.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.193+0000 7ff118869640 1 -- 172.21.15.31:0/3557414144 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff114063800 con 0x7ff0fc0b6590 2024-09-21T02:52:32.194 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T02:52:32.194 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 46s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-21T02:52:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 46s ago 8m smithi031=x;smithi165=y;smithi178=z;count:3 2024-09-21T02:52:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 46s ago 8m smithi031:172.21.15.31=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-21T02:52:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 46s ago - 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff0fc0b6590 msgr2=0x7ff0fc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff0fc0b6590 0x7ff0fc0b8aa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1040096c0 tx=0x7ff104031400 comp rx=0 tx=0).stop 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 msgr2=0x7ff11419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 0x7ff11419fda0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00eaa0 tx=0x7ff10c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 shutdown_connections 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff0fc0b6590 0x7ff0fc0b8aa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:32.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff114105270 0x7ff11419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11410ddf0 0x7ff11419fda0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 --2- 172.21.15.31:0/3557414144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff114104840 0x7ff11419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.195+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 >> 172.21.15.31:0/3557414144 conn(0x7ff114079730 msgr2=0x7ff114100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.196+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 shutdown_connections 2024-09-21T02:52:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:32.196+0000 7ff11986b640 1 -- 172.21.15.31:0/3557414144 wait complete. 2024-09-21T02:52:32.826 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:32 smithi031 ceph-mon[31209]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:52:32.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T02:52:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:32 smithi178 ceph-mon[34759]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:52:33.116 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:32 smithi165 ceph-mon[34818]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 -- 172.21.15.31:0/3337263293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c108cf0 msgr2=0x7f964c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 --2- 172.21.15.31:0/3337263293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c108cf0 0x7f964c10f520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9644009ca0 tx=0x7f964402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 -- 172.21.15.31:0/3337263293 shutdown_connections 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 --2- 172.21.15.31:0/3337263293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 0x7f964c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 --2- 172.21.15.31:0/3337263293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c108cf0 0x7f964c10f520 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 --2- 172.21.15.31:0/3337263293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c102a60 0x7f964c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 -- 172.21.15.31:0/3337263293 >> 172.21.15.31:0/3337263293 conn(0x7f964c079730 msgr2=0x7f964c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 -- 172.21.15.31:0/3337263293 shutdown_connections 2024-09-21T02:52:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.589+0000 7f9653904640 1 -- 172.21.15.31:0/3337263293 wait complete. 2024-09-21T02:52:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 Processor -- start 2024-09-21T02:52:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 -- start start 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 0x7f964c113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 0x7f964c113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c108cf0 0x7f964c113c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f964c112960 con 0x7f964c102a60 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1127e0 con 0x7f964c108cf0 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f9653904640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f964c112ae0 con 0x7f964c1083a0 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f96516a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 0x7f964c113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f96516a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 0x7f964c113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36098/0 (socket says 172.21.15.31:36098) 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.590+0000 7f96516a0640 1 -- 172.21.15.31:0/3580365107 learned_addr learned my addr 172.21.15.31:0/3580365107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f9650e9f640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 0x7f964c113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f9651ea1640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c108cf0 0x7f964c113c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96516a0640 1 -- 172.21.15.31:0/3580365107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 msgr2=0x7f964c113750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96516a0640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 0x7f964c113750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96516a0640 1 -- 172.21.15.31:0/3580365107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c108cf0 msgr2=0x7f964c113c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96516a0640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c108cf0 0x7f964c113c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96516a0640 1 -- 172.21.15.31:0/3580365107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c1a87b0 con 0x7f964c102a60 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f9651ea1640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c108cf0 0x7f964c113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f9650e9f640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 0x7f964c113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96516a0640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 0x7f964c113210 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f964800c890 tx=0x7f964800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004220 con 0x7f964c102a60 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c1a8aa0 con 0x7f964c102a60 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96480043c0 con 0x7f964c102a60 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648004a00 con 0x7f964c102a60 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.591+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c1a8fe0 con 0x7f964c102a60 2024-09-21T02:52:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.592+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964c103ce0 con 0x7f964c102a60 2024-09-21T02:52:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.594+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f96480079b0 con 0x7f964c102a60 2024-09-21T02:52:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.595+0000 7f96427fc640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96380b6560 0x7f96380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.595+0000 7f9650e9f640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96380b6560 0x7f96380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.595+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f96480e7700 con 0x7f964c102a60 2024-09-21T02:52:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.595+0000 7f9650e9f640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96380b6560 0x7f96380b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f964c105df0 tx=0x7f963c005cb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.598+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9648014030 con 0x7f964c102a60 2024-09-21T02:52:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.790+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f964c101560 con 0x7f96380b6560 2024-09-21T02:52:33.792 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:33 smithi031 ceph-mon[31209]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.792+0000 7f96427fc640 1 -- 172.21.15.31:0/3580365107 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f964c101560 con 0x7f96380b6560 2024-09-21T02:52:33.793 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-09-21T02:52:33.793 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-09-21T02:52:33.794 INFO:teuthology.orchestra.run.smithi031.stdout:smithi165 172.21.15.165 2024-09-21T02:52:33.794 INFO:teuthology.orchestra.run.smithi031.stdout:smithi178 172.21.15.178 2024-09-21T02:52:33.794 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-09-21T02:52:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96380b6560 msgr2=0x7f96380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96380b6560 0x7f96380b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f964c105df0 tx=0x7f963c005cb0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 msgr2=0x7f964c113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 0x7f964c113210 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f964800c890 tx=0x7f964800cd50 comp rx=0 tx=0).stop 2024-09-21T02:52:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 shutdown_connections 2024-09-21T02:52:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96380b6560 0x7f96380b8a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f964c1083a0 0x7f964c113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f964c108cf0 0x7f964c113c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 --2- 172.21.15.31:0/3580365107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f964c102a60 0x7f964c113210 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 >> 172.21.15.31:0/3580365107 conn(0x7f964c079730 msgr2=0x7f964c06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.794+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 shutdown_connections 2024-09-21T02:52:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:33.795+0000 7f9653904640 1 -- 172.21.15.31:0/3580365107 wait complete. 2024-09-21T02:52:33.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:33 smithi178 ceph-mon[34759]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:34.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:33 smithi165 ceph-mon[34818]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:34.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T02:52:34.716 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:34 smithi178 ceph-mon[34759]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T02:52:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:34 smithi178 ceph-mon[34759]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:34 smithi031 ceph-mon[31209]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T02:52:34.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:34 smithi031 ceph-mon[31209]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:34 smithi165 ceph-mon[34818]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T02:52:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:34 smithi165 ceph-mon[34818]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:35.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 -- 172.21.15.31:0/3169007529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 msgr2=0x7ff6781098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:35.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 --2- 172.21.15.31:0/3169007529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff6781098d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009900 tx=0x7ff66c02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:35.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 -- 172.21.15.31:0/3169007529 shutdown_connections 2024-09-21T02:52:35.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 --2- 172.21.15.31:0/3169007529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 0x7ff678110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 --2- 172.21.15.31:0/3169007529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff6781098d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 --2- 172.21.15.31:0/3169007529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff678103b80 0x7ff678103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 -- 172.21.15.31:0/3169007529 >> 172.21.15.31:0/3169007529 conn(0x7ff6780ff610 msgr2=0x7ff678101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:35.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 -- 172.21.15.31:0/3169007529 shutdown_connections 2024-09-21T02:52:35.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.227+0000 7ff67d152640 1 -- 172.21.15.31:0/3169007529 wait complete. 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 Processor -- start 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 -- start start 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff678103b80 0x7ff67819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff67819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 0x7ff67819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff678113b60 con 0x7ff6781094c0 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff6781139e0 con 0x7ff678109e10 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.228+0000 7ff67d152640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff678113ce0 con 0x7ff678103b80 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff67819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff67819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36122/0 (socket says 172.21.15.31:36122) 2024-09-21T02:52:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 -- 172.21.15.31:0/1405423402 learned_addr learned my addr 172.21.15.31:0/1405423402 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff676d9d640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff678103b80 0x7ff67819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67759e640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 0x7ff67819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 -- 172.21.15.31:0/1405423402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff678103b80 msgr2=0x7ff67819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff678103b80 0x7ff67819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 -- 172.21.15.31:0/1405423402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 msgr2=0x7ff67819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 0x7ff67819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 -- 172.21.15.31:0/1405423402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6781a0520 con 0x7ff6781094c0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67759e640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 0x7ff67819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67659c640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff67819ab50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff66c031440 tx=0x7ff66c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c03f070 con 0x7ff6781094c0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.229+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6781a07b0 con 0x7ff6781094c0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.230+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff66c031d60 con 0x7ff6781094c0 2024-09-21T02:52:35.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.230+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c035030 con 0x7ff6781094c0 2024-09-21T02:52:35.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.230+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6781a6f70 con 0x7ff6781094c0 2024-09-21T02:52:35.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.231+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff678103f90 con 0x7ff6781094c0 2024-09-21T02:52:35.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.232+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7ff66c042070 con 0x7ff6781094c0 2024-09-21T02:52:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.233+0000 7ff65ffff640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff6600b6560 0x7ff6600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.233+0000 7ff676d9d640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff6600b6560 0x7ff6600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.233+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff66c107d90 con 0x7ff6781094c0 2024-09-21T02:52:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.234+0000 7ff676d9d640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff6600b6560 0x7ff6600b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6680058c0 tx=0x7ff66800a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:35.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.236+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff66c0ce520 con 0x7ff6781094c0 2024-09-21T02:52:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.433+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff678063800 con 0x7ff6600b6560 2024-09-21T02:52:35.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.435+0000 7ff65ffff640 1 -- 172.21.15.31:0/1405423402 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff678063800 con 0x7ff6600b6560 2024-09-21T02:52:35.436 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T02:52:35.436 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T02:52:35.436 INFO:teuthology.orchestra.run.smithi031.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T02:52:35.436 INFO:teuthology.orchestra.run.smithi031.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-21T02:52:35.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff6600b6560 msgr2=0x7ff6600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff6600b6560 0x7ff6600b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff6680058c0 tx=0x7ff66800a250 comp rx=0 tx=0).stop 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 msgr2=0x7ff67819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff67819ab50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff66c031440 tx=0x7ff66c003d50 comp rx=0 tx=0).stop 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 shutdown_connections 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff6600b6560 0x7ff6600b8a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678109e10 0x7ff67819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6781094c0 0x7ff67819ab50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 --2- 172.21.15.31:0/1405423402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff678103b80 0x7ff67819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 >> 172.21.15.31:0/1405423402 conn(0x7ff6780ff610 msgr2=0x7ff678106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 shutdown_connections 2024-09-21T02:52:35.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:35.437+0000 7ff67d152640 1 -- 172.21.15.31:0/1405423402 wait complete. 2024-09-21T02:52:36.372 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T02:52:36.381 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T02:52:36.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T02:52:36.646 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:36 smithi178 ceph-mon[34759]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:36 smithi178 ceph-mon[34759]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:37.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:36 smithi031 ceph-mon[31209]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:37.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:36 smithi031 ceph-mon[31209]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:37.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.166+0000 7fdef8a1a640 1 -- 172.21.15.31:0/497073489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef41083d0 msgr2=0x7fdef41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:37.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.166+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/497073489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef41083d0 0x7fdef41087e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009ca0 tx=0x7fdee802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 -- 172.21.15.31:0/497073489 shutdown_connections 2024-09-21T02:52:37.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/497073489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef4108d20 0x7fdef410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/497073489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef41083d0 0x7fdef41087e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/497073489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4102ae0 0x7fdef4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 -- 172.21.15.31:0/497073489 >> 172.21.15.31:0/497073489 conn(0x7fdef4079760 msgr2=0x7fdef4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 -- 172.21.15.31:0/497073489 shutdown_connections 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.167+0000 7fdef8a1a640 1 -- 172.21.15.31:0/497073489 wait complete. 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 Processor -- start 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 -- start start 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 0x7fdef419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef41083d0 0x7fdef419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4108d20 0x7fdef419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4112990 con 0x7fdef4102ae0 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4112810 con 0x7fdef41083d0 2024-09-21T02:52:37.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef8a1a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdef4112b10 con 0x7fdef4108d20 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 0x7fdef419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 0x7fdef419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36142/0 (socket says 172.21.15.31:36142) 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 -- 172.21.15.31:0/2759897692 learned_addr learned my addr 172.21.15.31:0/2759897692 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef1d9b640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef41083d0 0x7fdef419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef2d9d640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4108d20 0x7fdef419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 -- 172.21.15.31:0/2759897692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4108d20 msgr2=0x7fdef419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4108d20 0x7fdef419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 -- 172.21.15.31:0/2759897692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef41083d0 msgr2=0x7fdef419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.168+0000 7fdef259c640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef41083d0 0x7fdef419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdef259c640 1 -- 172.21.15.31:0/2759897692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef41a0500 con 0x7fdef4102ae0 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdef1d9b640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef41083d0 0x7fdef419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdef2d9d640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4108d20 0x7fdef419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:37.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdef259c640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 0x7fdef419a590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdee400d770 tx=0x7fdee400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4017070 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee40054d0 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef41a07f0 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4004c30 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.169+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef41a6f50 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.170+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef4103d10 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.172+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fdee4004e10 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.172+0000 7fdee37fe640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdedc0b6560 0x7fdedc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.172+0000 7fdef1d9b640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdedc0b6560 0x7fdedc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.172+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdee40e7030 con 0x7fdef4102ae0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.173+0000 7fdef1d9b640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdedc0b6560 0x7fdedc0b8a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdee8006010 tx=0x7fdee8000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.177+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdee40ad7c0 con 0x7fdef4102ae0 2024-09-21T02:52:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:36 smithi165 ceph-mon[34818]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:36 smithi165 ceph-mon[34818]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:37.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.407+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fdef410bb70 con 0x7fdef4102ae0 2024-09-21T02:52:37.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.407+0000 7fdee37fe640 1 -- 172.21.15.31:0/2759897692 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3134 (secure 0 0 0) 0x7fdef410bb70 con 0x7fdef4102ae0 2024-09-21T02:52:37.409 INFO:teuthology.orchestra.run.smithi031.stdout:e26 2024-09-21T02:52:37.409 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-21T02:51:02:519865+0000 2024-09-21T02:52:37.409 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 26 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-21T02:50:50.973503+0000 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-21T02:51:02.068424+0000 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-09-21T02:52:37.410 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 5 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1,2,3,4 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24695,1=24659,2=24655,3=24637,4=24632} 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-21T02:52:37.411 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24695 members: 24695,24659,24655,24637,24632 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24695} state up:active seq 4 addr [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{0:24625} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/178089911,v1:172.21.15.178:6835/178089911] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{1:24659} state up:active seq 9 addr [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{1:24602} state up:standby-replay seq 1 addr [v2:172.21.15.165:6832/3390770693,v1:172.21.15.165:6833/3390770693] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{2:24655} state up:active seq 9 addr [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{2:24583} state up:standby-replay seq 1 addr [v2:172.21.15.178:6832/1642580204,v1:172.21.15.178:6833/1642580204] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{3:24637} state up:active seq 10 addr [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{3:14850} state up:standby-replay seq 1 addr [v2:172.21.15.31:6840/2144690594,v1:172.21.15.31:6841/2144690594] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{4:24632} state up:active seq 12 addr [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{4:14838} state up:standby-replay seq 1 addr [v2:172.21.15.31:6838/3653301851,v1:172.21.15.31:6839/3653301851] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14775} state up:standby seq 1 addr [v2:172.21.15.31:6834/3355921594,v1:172.21.15.31:6835/3355921594] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.413 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14787} state up:standby seq 1 addr [v2:172.21.15.31:6836/3179894266,v1:172.21.15.31:6837/3179894266] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:52:37.414 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:52:37.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdedc0b6560 msgr2=0x7fdedc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:37.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdedc0b6560 0x7fdedc0b8a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdee8006010 tx=0x7fdee8000f40 comp rx=0 tx=0).stop 2024-09-21T02:52:37.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 msgr2=0x7fdef419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:37.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 0x7fdef419a590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdee400d770 tx=0x7fdee400dc30 comp rx=0 tx=0).stop 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 shutdown_connections 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdedc0b6560 0x7fdedc0b8a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdef4108d20 0x7fdef419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef41083d0 0x7fdef419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 --2- 172.21.15.31:0/2759897692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4102ae0 0x7fdef419a590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 >> 172.21.15.31:0/2759897692 conn(0x7fdef4079760 msgr2=0x7fdef4101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 shutdown_connections 2024-09-21T02:52:37.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:37.411+0000 7fdef8a1a640 1 -- 172.21.15.31:0/2759897692 wait complete. 2024-09-21T02:52:37.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:37 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2759897692' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:52:38.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:37 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2759897692' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:52:38.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:37 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2759897692' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:52:38.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T02:52:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:38.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:38 smithi031 ceph-mon[31209]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 -- 172.21.15.31:0/1757867218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 msgr2=0x7f74b4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 --2- 172.21.15.31:0/1757867218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b4110620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009ca0 tx=0x7f74ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 -- 172.21.15.31:0/1757867218 shutdown_connections 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 --2- 172.21.15.31:0/1757867218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74b41094a0 0x7f74b41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 --2- 172.21.15.31:0/1757867218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b4110620 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 --2- 172.21.15.31:0/1757867218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74b4103b60 0x7f74b4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 -- 172.21.15.31:0/1757867218 >> 172.21.15.31:0/1757867218 conn(0x7f74b407b850 msgr2=0x7f74b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 -- 172.21.15.31:0/1757867218 shutdown_connections 2024-09-21T02:52:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.042+0000 7f74baaa7640 1 -- 172.21.15.31:0/1757867218 wait complete. 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.043+0000 7f74baaa7640 1 Processor -- start 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.043+0000 7f74baaa7640 1 -- start start 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74baaa7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74b4103b60 0x7f74b419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74baaa7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74b41094a0 0x7f74b419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74baaa7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74baaa7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f74b4113ab0 con 0x7f74b4109df0 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74baaa7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f74b4113930 con 0x7f74b4103b60 2024-09-21T02:52:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74baaa7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74b4113c30 con 0x7f74b41094a0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36156/0 (socket says 172.21.15.31:36156) 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 -- 172.21.15.31:0/690025526 learned_addr learned my addr 172.21.15.31:0/690025526 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b8843640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74b4103b60 0x7f74b419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b3fff640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74b41094a0 0x7f74b419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 -- 172.21.15.31:0/690025526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74b41094a0 msgr2=0x7f74b419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74b41094a0 0x7f74b419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 -- 172.21.15.31:0/690025526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74b4103b60 msgr2=0x7f74b419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74b4103b60 0x7f74b419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 -- 172.21.15.31:0/690025526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b41a03b0 con 0x7f74b4109df0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.044+0000 7f74b9044640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b419fc80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0098d0 tx=0x7f74ac0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.045+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac003b70 con 0x7f74b4109df0 2024-09-21T02:52:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.045+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac003d10 con 0x7f74b4109df0 2024-09-21T02:52:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.045+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b41a0640 con 0x7f74b4109df0 2024-09-21T02:52:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.045+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac02fa20 con 0x7f74b4109df0 2024-09-21T02:52:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.045+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b41a6e00 con 0x7f74b4109df0 2024-09-21T02:52:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.046+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b4103f70 con 0x7f74b4109df0 2024-09-21T02:52:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.048+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f74ac02fc00 con 0x7f74b4109df0 2024-09-21T02:52:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.049+0000 7f74b1ffb640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f749c0b6560 0x7f749c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.049+0000 7f74b8843640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f749c0b6560 0x7f749c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.049+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f74ac108740 con 0x7f74b4109df0 2024-09-21T02:52:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.049+0000 7f74b8843640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f749c0b6560 0x7f749c0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74a400bbe0 tx=0x7f74a400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.053+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74ac0cee70 con 0x7f74b4109df0 2024-09-21T02:52:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:38 smithi165 ceph-mon[34818]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:39.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:38 smithi178 ceph-mon[34759]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.243+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f74b402e640 con 0x7f74b4109df0 2024-09-21T02:52:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.244+0000 7f74b1ffb640 1 -- 172.21.15.31:0/690025526 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v78) ==== 56+0+5724 (secure 0 0 0) 0x7f74ac0d3250 con 0x7f74b4109df0 2024-09-21T02:52:39.246 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 78 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:fsid fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-21T02:42:18.754603+0000 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-21T02:51:16.567886+0000 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-21T02:52:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 9 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/3594547707,v1:172.21.15.31:6803/3594547707] [v2:172.21.15.31:6804/3594547707,v1:172.21.15.31:6805/3594547707] exists,up 23d2c70a-c096-4c33-acfd-b18efa64387a 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 14 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/3056017088,v1:172.21.15.165:6801/3056017088] [v2:172.21.15.165:6802/3056017088,v1:172.21.15.165:6803/3056017088] exists,up c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 19 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/730728159,v1:172.21.15.178:6801/730728159] [v2:172.21.15.178:6802/730728159,v1:172.21.15.178:6803/730728159] exists,up eca4fd08-95dd-4326-a6e7-59be134cad7a 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 26 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/3748731073,v1:172.21.15.31:6811/3748731073] [v2:172.21.15.31:6812/3748731073,v1:172.21.15.31:6813/3748731073] exists,up 68acf89d-75fa-43bf-8181-73e5043cc7a5 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 32 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/3903622433,v1:172.21.15.165:6809/3903622433] [v2:172.21.15.165:6810/3903622433,v1:172.21.15.165:6811/3903622433] exists,up edf3d333-8a5b-4a48-81ac-50917b7593c6 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 37 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/1160605172,v1:172.21.15.178:6809/1160605172] [v2:172.21.15.178:6810/1160605172,v1:172.21.15.178:6811/1160605172] exists,up d68d7120-069f-428b-823a-5fbd7f74f03b 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 43 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/787463273,v1:172.21.15.31:6819/787463273] [v2:172.21.15.31:6820/787463273,v1:172.21.15.31:6821/787463273] exists,up 3a7e5212-2bd5-4861-8192-1ab5dcb592c0 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 48 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/2423292259,v1:172.21.15.165:6817/2423292259] [v2:172.21.15.165:6818/2423292259,v1:172.21.15.165:6819/2423292259] exists,up a58534f6-f366-4d71-a057-272ba5874b0d 2024-09-21T02:52:39.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 54 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/3837162203,v1:172.21.15.178:6817/3837162203] [v2:172.21.15.178:6818/3837162203,v1:172.21.15.178:6819/3837162203] exists,up 5831064f-dc15-4eab-8239-3648a77f4b4e 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 60 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/1204611917,v1:172.21.15.31:6827/1204611917] [v2:172.21.15.31:6828/1204611917,v1:172.21.15.31:6829/1204611917] exists,up 2d67c10d-2a66-4175-8ec4-f6178369fff1 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 65 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/3486886499,v1:172.21.15.165:6825/3486886499] [v2:172.21.15.165:6826/3486886499,v1:172.21.15.165:6827/3486886499] exists,up d0ae22f5-028f-4723-9374-c88933908b49 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 70 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/3803011013,v1:172.21.15.178:6825/3803011013] [v2:172.21.15.178:6826/3803011013,v1:172.21.15.178:6827/3803011013] exists,up 0fb36d0d-9be8-49f1-b950-2e61cfcef7c5 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.5 [8,11] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.b [2,1] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1b [2,4] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1d [8,11] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.10 [5,9] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.12 [3,9] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.23 [4,7] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.28 [5,6] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.34 [4,9] 2024-09-21T02:52:39.249 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.3a [4,8] 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/944835008 expires 2024-09-22T02:44:15.049623+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/4246092214 expires 2024-09-22T02:44:15.049623+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/262827684 expires 2024-09-22T02:43:13.322815+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/4029989165 expires 2024-09-22T02:43:13.322815+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/1175221317 expires 2024-09-22T02:42:44.994232+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/1876173055 expires 2024-09-22T02:43:13.322815+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/1175221317 expires 2024-09-22T02:42:44.994232+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/814395253 expires 2024-09-22T02:42:44.994232+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3796123698 expires 2024-09-22T02:43:13.322815+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1104840323 expires 2024-09-22T02:42:44.994232+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/633109246 expires 2024-09-22T02:42:44.994232+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/1876173055 expires 2024-09-22T02:43:13.322815+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3425970992 expires 2024-09-22T02:44:15.049623+0000 2024-09-21T02:52:39.250 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/4246092214 expires 2024-09-22T02:44:15.049623+0000 2024-09-21T02:52:39.251 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1565999986 expires 2024-09-22T02:44:15.049623+0000 2024-09-21T02:52:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f749c0b6560 msgr2=0x7f749c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f749c0b6560 0x7f749c0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f74a400bbe0 tx=0x7f74a400b390 comp rx=0 tx=0).stop 2024-09-21T02:52:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 msgr2=0x7f74b419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b419fc80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0098d0 tx=0x7f74ac0317b0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 shutdown_connections 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f749c0b6560 0x7f749c0b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74b41094a0 0x7f74b419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b4109df0 0x7f74b419fc80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 --2- 172.21.15.31:0/690025526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74b4103b60 0x7f74b419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 >> 172.21.15.31:0/690025526 conn(0x7f74b407b850 msgr2=0x7f74b4105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 shutdown_connections 2024-09-21T02:52:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:39.246+0000 7f74baaa7640 1 -- 172.21.15.31:0/690025526 wait complete. 2024-09-21T02:52:39.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:39 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/690025526' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:52:40.035 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T02:52:40.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:52:40.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:39 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/690025526' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:52:40.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:39 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/690025526' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:52:40.315 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.821+0000 7f3893570640 1 -- 172.21.15.31:0/2295628169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c1094a0 msgr2=0x7f388c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.821+0000 7f3893570640 1 --2- 172.21.15.31:0/2295628169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c1094a0 0x7f388c1098b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f387c009900 tx=0x7f387c02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 -- 172.21.15.31:0/2295628169 shutdown_connections 2024-09-21T02:52:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 --2- 172.21.15.31:0/2295628169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f388c109df0 0x7f388c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 --2- 172.21.15.31:0/2295628169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c1094a0 0x7f388c1098b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 --2- 172.21.15.31:0/2295628169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f388c103b60 0x7f388c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 -- 172.21.15.31:0/2295628169 >> 172.21.15.31:0/2295628169 conn(0x7f388c07b850 msgr2=0x7f388c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 -- 172.21.15.31:0/2295628169 shutdown_connections 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 -- 172.21.15.31:0/2295628169 wait complete. 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 Processor -- start 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.822+0000 7f3893570640 1 -- start start 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3893570640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f388c103b60 0x7f388c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3893570640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f388c1094a0 0x7f388c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3893570640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 0x7f388c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3893570640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113ad0 con 0x7f388c109df0 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3893570640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113950 con 0x7f388c1094a0 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3893570640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f388c113c50 con 0x7f388c103b60 2024-09-21T02:52:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 0x7f388c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 0x7f388c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35926/0 (socket says 172.21.15.31:35926) 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 -- 172.21.15.31:0/22383712 learned_addr learned my addr 172.21.15.31:0/22383712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f389130c640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f388c103b60 0x7f388c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3890b0b640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f388c1094a0 0x7f388c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 -- 172.21.15.31:0/22383712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f388c103b60 msgr2=0x7f388c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f388c103b60 0x7f388c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 -- 172.21.15.31:0/22383712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f388c1094a0 msgr2=0x7f388c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f388c1094a0 0x7f388c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 -- 172.21.15.31:0/22383712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c1a8780 con 0x7f388c109df0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f3891b0d640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 0x7f388c076190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f388400b970 tx=0x7f388400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.823+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884019070 con 0x7f388c109df0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.824+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1a8a70 con 0x7f388c109df0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.824+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3884014440 con 0x7f388c109df0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.824+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840075d0 con 0x7f388c109df0 2024-09-21T02:52:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.824+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a9000 con 0x7f388c109df0 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.825+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c103f70 con 0x7f388c109df0 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.827+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f388401c070 con 0x7f388c109df0 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.828+0000 7f38827fc640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f38780b6560 0x7f38780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.828+0000 7f389130c640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f38780b6560 0x7f38780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.828+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f38840e6ed0 con 0x7f388c109df0 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.828+0000 7f389130c640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f38780b6560 0x7f38780b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3888005fd0 tx=0x7f3888005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:40.831+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38840ad600 con 0x7f388c109df0 2024-09-21T02:52:41.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.057+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f388c076a10 con 0x7f388c109df0 2024-09-21T02:52:41.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:40 smithi031 ceph-mon[31209]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.058+0000 7f38827fc640 1 -- 172.21.15.31:0/22383712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7f388c076a10 con 0x7f388c109df0 2024-09-21T02:52:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:41.061 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:51:02.068424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24659,24655,24637,24632]},"id":1}]} 2024-09-21T02:52:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f38780b6560 msgr2=0x7f38780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f38780b6560 0x7f38780b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3888005fd0 tx=0x7f3888005850 comp rx=0 tx=0).stop 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 msgr2=0x7f388c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 0x7f388c076190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f388400b970 tx=0x7f388400be30 comp rx=0 tx=0).stop 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 shutdown_connections 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f38780b6560 0x7f38780b8a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c109df0 0x7f388c076190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f388c1094a0 0x7f388c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 --2- 172.21.15.31:0/22383712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f388c103b60 0x7f388c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 >> 172.21.15.31:0/22383712 conn(0x7f388c07b850 msgr2=0x7f388c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.060+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 shutdown_connections 2024-09-21T02:52:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:41.061+0000 7f3893570640 1 -- 172.21.15.31:0/22383712 wait complete. 2024-09-21T02:52:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:40 smithi165 ceph-mon[34818]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:40 smithi178 ceph-mon[34759]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:41.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:52:42.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:41 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/22383712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:42.085 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:42.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:41 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/22383712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:42.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:41 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/22383712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:42.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 -- 172.21.15.31:0/2043153967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4109e10 msgr2=0x7fe1f4110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/2043153967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4109e10 0x7fe1f4110640 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009ca0 tx=0x7fe1ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 -- 172.21.15.31:0/2043153967 shutdown_connections 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/2043153967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4109e10 0x7fe1f4110640 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/2043153967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 0x7fe1f41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/2043153967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4101fe0 0x7fe1f41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 -- 172.21.15.31:0/2043153967 >> 172.21.15.31:0/2043153967 conn(0x7fe1f4077bc0 msgr2=0x7fe1f4078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 -- 172.21.15.31:0/2043153967 shutdown_connections 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.555+0000 7fe1f92a7640 1 -- 172.21.15.31:0/2043153967 wait complete. 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 Processor -- start 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 -- start start 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 0x7fe1f419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:42.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 0x7fe1f419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4109e10 0x7fe1f419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4113a80 con 0x7fe1f4101fe0 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4113900 con 0x7fe1f41094c0 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f92a7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4113c00 con 0x7fe1f4109e10 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 0x7fe1f419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 0x7fe1f419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35940/0 (socket says 172.21.15.31:35940) 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.556+0000 7fe1f2ffd640 1 -- 172.21.15.31:0/3027309712 learned_addr learned my addr 172.21.15.31:0/3027309712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f27fc640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 0x7fe1f419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f37fe640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4109e10 0x7fe1f419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f2ffd640 1 -- 172.21.15.31:0/3027309712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4109e10 msgr2=0x7fe1f419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:42.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f2ffd640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4109e10 0x7fe1f419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f2ffd640 1 -- 172.21.15.31:0/3027309712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 msgr2=0x7fe1f419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f2ffd640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 0x7fe1f419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f2ffd640 1 -- 172.21.15.31:0/3027309712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f41a0360 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f37fe640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4109e10 0x7fe1f419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f27fc640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 0x7fe1f419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f2ffd640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 0x7fe1f419a3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c970 tx=0x7fe1e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4007bb0 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f41a0650 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e4007d50 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400f450 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.557+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f4102e60 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.558+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f41023f0 con 0x7fe1f4101fe0 2024-09-21T02:52:42.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.560+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fe1e4004f00 con 0x7fe1f4101fe0 2024-09-21T02:52:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.560+0000 7fe1dbfff640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe1dc0ba970 0x7fe1dc0bce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.561+0000 7fe1f27fc640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe1dc0ba970 0x7fe1dc0bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.561+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe1e40e7640 con 0x7fe1f4101fe0 2024-09-21T02:52:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.561+0000 7fe1f27fc640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe1dc0ba970 0x7fe1dc0bce80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4107010 tx=0x7fe1e800a560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:42.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.564+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1e4014030 con 0x7fe1f4101fe0 2024-09-21T02:52:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.787+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f419b520 con 0x7fe1f4101fe0 2024-09-21T02:52:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.788+0000 7fe1dbfff640 1 -- 172.21.15.31:0/3027309712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11982 (secure 0 0 0) 0x7fe1e40addd0 con 0x7fe1f4101fe0 2024-09-21T02:52:42.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:42.795 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-21T02:51:02:519865+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:51:02.068424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24695,24659,24655,24637,24632]},"id":1}]} 2024-09-21T02:52:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-21T02:52:42.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe1dc0ba970 msgr2=0x7fe1dc0bce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe1dc0ba970 0x7fe1dc0bce80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4107010 tx=0x7fe1e800a560 comp rx=0 tx=0).stop 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 msgr2=0x7fe1f419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 0x7fe1f419a3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c970 tx=0x7fe1e400ce30 comp rx=0 tx=0).stop 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 shutdown_connections 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe1dc0ba970 0x7fe1dc0bce80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1f4109e10 0x7fe1f419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1f41094c0 0x7fe1f419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 --2- 172.21.15.31:0/3027309712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f4101fe0 0x7fe1f419a3f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 >> 172.21.15.31:0/3027309712 conn(0x7fe1f4077bc0 msgr2=0x7fe1f41053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 shutdown_connections 2024-09-21T02:52:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:42.791+0000 7fe1f92a7640 1 -- 172.21.15.31:0/3027309712 wait complete. 2024-09-21T02:52:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:42 smithi031 ceph-mon[31209]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:42 smithi165 ceph-mon[34818]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:42 smithi178 ceph-mon[34759]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:43.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:52:43.784 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:44.039 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:43 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3027309712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:43 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3027309712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:43 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3027309712' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:52:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 -- 172.21.15.31:0/978543998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 msgr2=0x7f2ce41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/978543998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce41087e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009900 tx=0x7f2cd802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 -- 172.21.15.31:0/978543998 shutdown_connections 2024-09-21T02:52:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/978543998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4108d20 0x7f2ce410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/978543998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce41087e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/978543998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ce4102ae0 0x7f2ce4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 -- 172.21.15.31:0/978543998 >> 172.21.15.31:0/978543998 conn(0x7f2ce4079760 msgr2=0x7f2ce4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 -- 172.21.15.31:0/978543998 shutdown_connections 2024-09-21T02:52:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.271+0000 7f2ce93c0640 1 -- 172.21.15.31:0/978543998 wait complete. 2024-09-21T02:52:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 Processor -- start 2024-09-21T02:52:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 -- start start 2024-09-21T02:52:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 0x7f2ce41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ce4108d20 0x7f2ce419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4112990 con 0x7f2ce41083d0 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4112810 con 0x7f2ce4108d20 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.272+0000 7f2ce93c0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4112b10 con 0x7f2ce4102ae0 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 0x7f2ce41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 0x7f2ce41993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:55432/0 (socket says 172.21.15.31:55432) 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 -- 172.21.15.31:0/2818210315 learned_addr learned my addr 172.21.15.31:0/2818210315 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce37fe640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ce4108d20 0x7f2ce419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 -- 172.21.15.31:0/2818210315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ce4108d20 msgr2=0x7f2ce419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ce4108d20 0x7f2ce419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 -- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 msgr2=0x7f2ce4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 -- 172.21.15.31:0/2818210315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce419f310 con 0x7f2ce4102ae0 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce27fc640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ce2ffd640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 0x7f2ce41993a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400d950 tx=0x7f2cd400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.273+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4017070 con 0x7f2ce4102ae0 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.274+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce419f600 con 0x7f2ce4102ae0 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.274+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd4009c30 con 0x7f2ce4102ae0 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.274+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400f620 con 0x7f2ce4102ae0 2024-09-21T02:52:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.274+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce406bb40 con 0x7f2ce4102ae0 2024-09-21T02:52:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.275+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce4103d10 con 0x7f2ce4102ae0 2024-09-21T02:52:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.276+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f2cd4010460 con 0x7f2ce4102ae0 2024-09-21T02:52:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.277+0000 7f2ccbfff640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ccc0b6560 0x7f2ccc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.277+0000 7f2ce27fc640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ccc0b6560 0x7f2ccc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.277+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2cd40e7130 con 0x7f2ce4102ae0 2024-09-21T02:52:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.278+0000 7f2ce27fc640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ccc0b6560 0x7f2ccc0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8000c00 tx=0x7f2cd803a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:44.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.281+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2cd40ad860 con 0x7f2ce4102ae0 2024-09-21T02:52:44.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.473+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ce419a4d0 con 0x7f2ce4102ae0 2024-09-21T02:52:44.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.474+0000 7f2ccbfff640 1 -- 172.21.15.31:0/2818210315 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+23145 (secure 0 0 0) 0x7f2cd4009de0 con 0x7f2ce4102ae0 2024-09-21T02:52:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:44.477 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:51:16.567886+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ccc0b6560 msgr2=0x7f2ccc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ccc0b6560 0x7f2ccc0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8000c00 tx=0x7f2cd803a040 comp rx=0 tx=0).stop 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 msgr2=0x7f2ce41993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 0x7f2ce41993a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400d950 tx=0x7f2cd400de10 comp rx=0 tx=0).stop 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 shutdown_connections 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2ccc0b6560 0x7f2ccc0b8a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ce4108d20 0x7f2ce419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce41083d0 0x7f2ce4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 --2- 172.21.15.31:0/2818210315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ce4102ae0 0x7f2ce41993a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:44.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 >> 172.21.15.31:0/2818210315 conn(0x7f2ce4079760 msgr2=0x7f2ce4104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:44.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 shutdown_connections 2024-09-21T02:52:44.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:44.477+0000 7f2ce93c0640 1 -- 172.21.15.31:0/2818210315 wait complete. 2024-09-21T02:52:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:44 smithi031 ceph-mon[31209]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:44 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2818210315' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:44 smithi165 ceph-mon[34818]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:44 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2818210315' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:44 smithi178 ceph-mon[34759]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:44 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2818210315' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:45.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T02:50:50.973503+0000', 'modified': '2024-09-21T02:51:02.068424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24695, 'mds_1': 24659, 'mds_2': 24655, 'mds_3': 24637, 'mds_4': 24632}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14838': {'gid': 14838, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6839/3653301851', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6838', 'nonce': 3653301851}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 3653301851}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14850': {'gid': 14850, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6841/2144690594', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6840', 'nonce': 2144690594}, {'type': 'v1', 'addr': '172.21.15.31:6841', 'nonce': 2144690594}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24583': {'gid': 24583, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1642580204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1642580204}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1642580204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/3390770693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 3390770693}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 3390770693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24625': {'gid': 24625, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/178089911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 178089911}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 178089911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24632': {'gid': 24632, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.165:6835/3816837605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3816837605}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3816837605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.178:6837/1985506653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 1985506653}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 1985506653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.178:6839/730341197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 730341197}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 730341197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24659': {'gid': 24659, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.165:6837/3116204029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 3116204029}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 3116204029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.165:6839/2689629692', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 2689629692}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 2689629692}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24695, 'qdb_cluster': [24695, 24659, 24655, 24637, 24632]} 2024-09-21T02:52:45.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T02:52:45.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.e status 2024-09-21T02:52:45.526 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:46.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 -- 172.21.15.31:0/849626997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 msgr2=0x7f5200102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 --2- 172.21.15.31:0/849626997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f5200102ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009900 tx=0x7f51f002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 -- 172.21.15.31:0/849626997 shutdown_connections 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 --2- 172.21.15.31:0/849626997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200108d20 0x7f520010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 --2- 172.21.15.31:0/849626997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001083d0 0x7f52001087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 --2- 172.21.15.31:0/849626997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f5200102ef0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.041+0000 7f52056e3640 1 -- 172.21.15.31:0/849626997 >> 172.21.15.31:0/849626997 conn(0x7f5200079760 msgr2=0x7f5200079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.042+0000 7f52056e3640 1 -- 172.21.15.31:0/849626997 shutdown_connections 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.042+0000 7f52056e3640 1 -- 172.21.15.31:0/849626997 wait complete. 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.042+0000 7f52056e3640 1 Processor -- start 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.042+0000 7f52056e3640 1 -- start start 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f52056e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f520019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f52056e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001083d0 0x7f520019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f52056e3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200108d20 0x7f520019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f52056e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5200112990 con 0x7f5200102ae0 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f52056e3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5200112810 con 0x7f52001083d0 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f52056e3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5200112b10 con 0x7f5200108d20 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f520019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f520019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35970/0 (socket says 172.21.15.31:35970) 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 -- 172.21.15.31:0/2749104633 learned_addr learned my addr 172.21.15.31:0/2749104633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51f7fff640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001083d0 0x7f520019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200108d20 0x7f520019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 -- 172.21.15.31:0/2749104633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200108d20 msgr2=0x7f520019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200108d20 0x7f520019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 -- 172.21.15.31:0/2749104633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001083d0 msgr2=0x7f520019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001083d0 0x7f520019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 -- 172.21.15.31:0/2749104633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001a6550 con 0x7f5200102ae0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.043+0000 7f51feffd640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f520019b240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005940 tx=0x7f51f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.044+0000 7f51fcff9640 1 -- 172.21.15.31:0/2749104633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f003f030 con 0x7f5200102ae0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.044+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001a67e0 con 0x7f5200102ae0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.044+0000 7f51fcff9640 1 -- 172.21.15.31:0/2749104633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f0003cb0 con 0x7f5200102ae0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.044+0000 7f51fcff9640 1 -- 172.21.15.31:0/2749104633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0035070 con 0x7f5200102ae0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.044+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a6d10 con 0x7f5200102ae0 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.046+0000 7f520bc57740 1 Processor -- start 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.046+0000 7f520bc57740 1 -- start start 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f520bc57740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640741f60 0x55d640ae7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f520bc57740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6407190a0 0x55d640ae8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f520bc57740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640ae8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f520bc57740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d640a5fc10 con 0x55d640721280 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f520bc57740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d640a4f040 con 0x55d6407190a0 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f520bc57740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d640a52e50 con 0x55d640741f60 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51fcff9640 1 -- 172.21.15.31:0/2749104633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f51f0042030 con 0x7f5200102ae0 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640ae8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640ae8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35984/0 (socket says 172.21.15.31:35984) 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 -- 172.21.15.31:0/1532339573 learned_addr learned my addr 172.21.15.31:0/1532339573 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640741f60 0x55d640ae7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 -- 172.21.15.31:0/1532339573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640741f60 msgr2=0x55d640ae7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640741f60 0x55d640ae7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 -- 172.21.15.31:0/1532339573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6407190a0 msgr2=0x55d640ae8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6407190a0 0x55d640ae8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 -- 172.21.15.31:0/1532339573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6407ad540 con 0x55d640721280 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f7fff640 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640ae8980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bca0 tx=0x7f51e80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62fc5119baaba38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f57fa640 1 -- 172.21.15.31:0/1532339573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8005220 con 0x55d640721280 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f57fa640 1 -- 172.21.15.31:0/1532339573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e800be50 con 0x55d640721280 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.047+0000 7f51f57fa640 1 -- 172.21.15.31:0/1532339573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80116a0 con 0x55d640721280 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f51fcff9640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f51ec0b6560 0x7f51ec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f51ec0b6560 0x7f51ec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f51fcff9640 1 -- 172.21.15.31:0/2749104633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f51f0107900 con 0x7f5200102ae0 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 -- 172.21.15.31:0/1532339573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 msgr2=0x55d640ae8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640ae8980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bca0 tx=0x7f51e80107a0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 -- 172.21.15.31:0/1532339573 shutdown_connections 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640ae8980 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6407190a0 0x55d640ae8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 --2- 172.21.15.31:0/1532339573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640741f60 0x55d640ae7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.048+0000 7f520bc57740 1 -- 172.21.15.31:0/1532339573 >> 172.21.15.31:0/1532339573 conn(0x55d640736350 msgr2=0x55d640ae5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 -- 172.21.15.31:0/1532339573 shutdown_connections 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f51ec0b6560 0x7f51ec0b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f520006cc20 tx=0x7f51f8009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 -- 172.21.15.31:0/1532339573 wait complete. 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 -- start start 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 Processor -- start 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b03d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d6408b05f0 con 0x55d6407190a0 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d6409ea010 con 0x55d640741f60 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d640afaa10 con 0x55d640721280 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f520bc57740 10 client.0 _ll_register_callbacks cb 0x55d640a49340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:46.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.049+0000 7f51feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36000/0 (socket says 172.21.15.31:36000) 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 -- 172.21.15.31:0/3081176908 learned_addr learned my addr 172.21.15.31:0/3081176908 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51f7fff640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b03d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 -- 172.21.15.31:0/3081176908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 msgr2=0x55d640af8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 -- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 msgr2=0x55d640b03d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b03d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 -- 172.21.15.31:0/3081176908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d640a17a00 con 0x55d6407190a0 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51feffd640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7e20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f51f0003fd0 tx=0x7f51f00c99a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07ad1651caa9c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:46.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51e67fc640 1 -- 172.21.15.31:0/3081176908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0049040 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51e67fc640 1 -- 172.21.15.31:0/3081176908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f51f003bbf0 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e800be50 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f51e67fc640 1 -- 172.21.15.31:0/3081176908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51f00cd1b0 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f520bc57740 10 client.15387 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.050+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d640b069e0 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f51e67fc640 1 -- 172.21.15.31:0/3081176908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e800be50 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f51e67fc640 1 -- 172.21.15.31:0/3081176908 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f51f00d6f20 con 0x55d6407190a0 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f520bc57740 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f520bc57740 10 client.15387 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f520bc57740 20 client.15387 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f520bc57740 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6407008e0 0x55d640b0a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f520bc57740 4 client.15387 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d640a61b60 con 0x55d6407008e0 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.051+0000 7f51f7fff640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6407008e0 0x55d640b0a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.052+0000 7f51f7fff640 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6407008e0 0x55d640b0a420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b99321bdeed2e0e server_cookie=e63fc4ccb02746ed in_seq=0 out_seq=0 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.052+0000 7f51e67fc640 10 client.15387 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.052+0000 7f51e67fc640 1 -- 172.21.15.31:0/3081176908 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d640a61b60 con 0x55d6407008e0 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.052+0000 7f51e67fc640 10 client.15387 handle_command_reply: tid=0 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.052+0000 7f520bc57740 1 client.15387 shutdown 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 msgr2=0x55d640af7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7e20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f51f0003fd0 tx=0x7f51f00c99a0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 shutdown_connections 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6407008e0 0x55d640b0a420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b03d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 --2- 172.21.15.31:0/3081176908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7e20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 >> 172.21.15.31:0/3081176908 conn(0x55d640736350 msgr2=0x55d640aeb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:46.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f51e67fc640 0 client.15387 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:46.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 shutdown_connections 2024-09-21T02:52:46.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 1 -- 172.21.15.31:0/3081176908 wait complete. 2024-09-21T02:52:46.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.053+0000 7f520bc57740 20 client.15387 trim_cache size 0 max 16384 2024-09-21T02:52:46.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.071+0000 7f520bc57740 1 Processor -- start 2024-09-21T02:52:46.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.071+0000 7f520bc57740 1 -- start start 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f520bc57740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f520bc57740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f520bc57740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640af3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f520bc57740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d640b08760 con 0x55d6407190a0 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f520bc57740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d640b08e30 con 0x55d640741f60 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f520bc57740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d640b09500 con 0x55d640721280 2024-09-21T02:52:46.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36010/0 (socket says 172.21.15.31:36010) 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 -- 172.21.15.31:0/3861230596 learned_addr learned my addr 172.21.15.31:0/3861230596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640af3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51f7fff640 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 -- 172.21.15.31:0/3861230596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 msgr2=0x55d640af2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 -- 172.21.15.31:0/3861230596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 msgr2=0x55d640af3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640af3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 -- 172.21.15.31:0/3861230596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d640ae9e40 con 0x55d6407190a0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51f7fff640 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.072+0000 7f51feffd640 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cd9d0 tx=0x7f51f00045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6469e6b537acfe90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.073+0000 7f51f4ff9640 1 -- 172.21.15.31:0/3861230596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0049070 con 0x55d6407190a0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.073+0000 7f51f4ff9640 1 -- 172.21.15.31:0/3861230596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f00b6020 con 0x55d6407190a0 2024-09-21T02:52:46.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.073+0000 7f51f4ff9640 1 -- 172.21.15.31:0/3861230596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00c91b0 con 0x55d6407190a0 2024-09-21T02:52:46.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- 172.21.15.31:0/3861230596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 msgr2=0x55d640af7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cd9d0 tx=0x7f51f00045a0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- 172.21.15.31:0/3861230596 shutdown_connections 2024-09-21T02:52:46.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d640721280 0x55d640af2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640af3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- 172.21.15.31:0/3861230596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d6407190a0 0x55d640af7110 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- 172.21.15.31:0/3861230596 >> 172.21.15.31:0/3861230596 conn(0x55d640736350 msgr2=0x55d640b07d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- 172.21.15.31:0/3861230596 shutdown_connections 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- 172.21.15.31:0/3861230596 wait complete. 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- start start 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 Processor -- start 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6407190a0 0x55d640b0cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640b0d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b11c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d640b0a2a0 con 0x55d640721280 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d640b09bd0 con 0x55d640741f60 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.074+0000 7f520bc57740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d640b0a970 con 0x55d6407190a0 2024-09-21T02:52:46.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f520bc57740 10 client.0 _ll_register_callbacks cb 0x7f51f0005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640b0d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640b0d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36024/0 (socket says 172.21.15.31:36024) 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 -- 172.21.15.31:0/581244198 learned_addr learned my addr 172.21.15.31:0/581244198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51ff7fe640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b11c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51feffd640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6407190a0 0x55d640b0cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 -- 172.21.15.31:0/581244198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6407190a0 msgr2=0x55d640b0cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6407190a0 0x55d640b0cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 -- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 msgr2=0x55d640b11c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b11c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 -- 172.21.15.31:0/581244198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d640b12150 con 0x55d640721280 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51feffd640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6407190a0 0x55d640b0cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51f7fff640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640b0d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51e8000c00 tx=0x7f51e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfd88a5a83a48a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51e67fc640 1 -- 172.21.15.31:0/581244198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80037f0 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f00b6020 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.075+0000 7f51e67fc640 1 -- 172.21.15.31:0/581244198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f51e8003990 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f51e67fc640 1 -- 172.21.15.31:0/581244198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51e800be50 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 10 client.15393 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d640b13340 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f51e67fc640 1 -- 172.21.15.31:0/581244198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f00b6020 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f51e67fc640 1 -- 172.21.15.31:0/581244198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f51e8034030 con 0x55d640721280 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 10 client.15393 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 10 client.15393 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 20 client.15393 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6406d5c90 0x55d640b78b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 4 client.15393 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.076+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "status"}) -- 0x55d640a5d610 con 0x55d6406d5c90 2024-09-21T02:52:46.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f51feffd640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6406d5c90 0x55d640b78b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f51feffd640 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6406d5c90 0x55d640b78b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cdd3df8f6aa87199 server_cookie=f0f15fafa9c04083 in_seq=0 out_seq=0 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f51e67fc640 10 client.15393 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f51e67fc640 1 -- 172.21.15.31:0/581244198 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d640a5d610 con 0x55d6406d5c90 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f51e67fc640 10 client.15393 handle_command_reply: tid=0 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f520bc57740 1 client.15393 shutdown 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 msgr2=0x55d640b0d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f520bc57740 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640b0d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51e8000c00 tx=0x7f51e80058a0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 shutdown_connections 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.077+0000 7f520bc57740 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d640741f60 0x55d640b11c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d640721280 0x55d640b0d300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55d6406d5c90 0x55d640b78b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 1 --2- 172.21.15.31:0/581244198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6407190a0 0x55d640b0cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 >> 172.21.15.31:0/581244198 conn(0x55d640736350 msgr2=0x55d640af2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:46.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f51e67fc640 0 client.15393 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 shutdown_connections 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 1 -- 172.21.15.31:0/581244198 wait complete. 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f520bc57740 20 client.15393 trim_cache size 0 max 16384 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 4, 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24632, 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:46.085 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 104.65564568000001, 2024-09-21T02:52:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:46.086 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 145.58961909000001 2024-09-21T02:52:46.086 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:46.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f51ec0b6560 msgr2=0x7f51ec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f52056e3640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f51ec0b6560 0x7f51ec0b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f520006cc20 tx=0x7f51f8009600 comp rx=0 tx=0).stop 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 msgr2=0x7f520019b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.078+0000 7f52056e3640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f520019b240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005940 tx=0x7f51f00042f0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 shutdown_connections 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f51ec0b6560 0x7f51ec0b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200108d20 0x7f520019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001083d0 0x7f520019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 --2- 172.21.15.31:0/2749104633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5200102ae0 0x7f520019b240 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 >> 172.21.15.31:0/2749104633 conn(0x7f5200079760 msgr2=0x7f5200101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 shutdown_connections 2024-09-21T02:52:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:46.079+0000 7f52056e3640 1 -- 172.21.15.31:0/2749104633 wait complete. 2024-09-21T02:52:46.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T02:52:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:46 smithi031 ceph-mon[31209]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:47.173 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:46 smithi165 ceph-mon[34818]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:46 smithi178 ceph-mon[34759]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 -- 172.21.15.31:0/2596690765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 msgr2=0x7ff2841087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 --2- 172.21.15.31:0/2596690765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff2841087e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff274009900 tx=0x7ff27402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 -- 172.21.15.31:0/2596690765 shutdown_connections 2024-09-21T02:52:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 --2- 172.21.15.31:0/2596690765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff284108d20 0x7ff28410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 --2- 172.21.15.31:0/2596690765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff2841087e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 --2- 172.21.15.31:0/2596690765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff284102a90 0x7ff284102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 -- 172.21.15.31:0/2596690765 >> 172.21.15.31:0/2596690765 conn(0x7ff284079730 msgr2=0x7ff284079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 -- 172.21.15.31:0/2596690765 shutdown_connections 2024-09-21T02:52:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.661+0000 7ff28c575640 1 -- 172.21.15.31:0/2596690765 wait complete. 2024-09-21T02:52:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.662+0000 7ff28c575640 1 Processor -- start 2024-09-21T02:52:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.662+0000 7ff28c575640 1 -- start start 2024-09-21T02:52:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28c575640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff284102a90 0x7ff284199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28c575640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff284199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28c575640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff284108d20 0x7ff28419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28c575640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841129e0 con 0x7ff2841083d0 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28c575640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff284112860 con 0x7ff284102a90 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28c575640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff284112b60 con 0x7ff284108d20 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff284199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff284199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36048/0 (socket says 172.21.15.31:36048) 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 -- 172.21.15.31:0/3473515437 learned_addr learned my addr 172.21.15.31:0/3473515437 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28a311640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff284102a90 0x7ff284199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff284108d20 0x7ff28419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 -- 172.21.15.31:0/3473515437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff284108d20 msgr2=0x7ff28419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff284108d20 0x7ff28419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 -- 172.21.15.31:0/3473515437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff284102a90 msgr2=0x7ff284199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff284102a90 0x7ff284199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.663+0000 7ff289b10640 1 -- 172.21.15.31:0/3473515437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28419f240 con 0x7ff2841083d0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.664+0000 7ff289b10640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff284199930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff27402a930 tx=0x7ff274004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.664+0000 7ff27b7fe640 1 -- 172.21.15.31:0/3473515437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274046070 con 0x7ff2841083d0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.664+0000 7ff27b7fe640 1 -- 172.21.15.31:0/3473515437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274003aa0 con 0x7ff2841083d0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.664+0000 7ff27b7fe640 1 -- 172.21.15.31:0/3473515437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274035050 con 0x7ff2841083d0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.664+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28419f4d0 con 0x7ff2841083d0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.664+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28419fa00 con 0x7ff2841083d0 2024-09-21T02:52:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.666+0000 7ff292ae9740 1 Processor -- start 2024-09-21T02:52:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 -- start start 2024-09-21T02:52:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 0x5637e1d52f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d53440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d53980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637e1ccac10 con 0x5637e19acf60 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637e1cba040 con 0x5637e198c280 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff292ae9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637e1cbde50 con 0x5637e19840a0 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff27b7fe640 1 -- 172.21.15.31:0/3473515437 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7ff274049070 con 0x7ff2841083d0 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28ab12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 0x5637e1d52f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28ab12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 0x5637e1d52f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36056/0 (socket says 172.21.15.31:36056) 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28ab12640 1 -- 172.21.15.31:0/3137557688 learned_addr learned my addr 172.21.15.31:0/3137557688 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28a311640 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d53440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28a311640 1 -- 172.21.15.31:0/3137557688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 msgr2=0x5637e1d53980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff289b10640 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d53980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28a311640 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d53980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28a311640 1 -- 172.21.15.31:0/3137557688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 msgr2=0x5637e1d52f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28a311640 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 0x5637e1d52f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.667+0000 7ff28a311640 1 -- 172.21.15.31:0/3137557688 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e1a18540 con 0x5637e19840a0 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 0x5637e1d52f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff28a311640 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d53440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff280004630 tx=0x7ff28000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220cb9b05ab36319 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff27b7fe640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff2700b6610 0x7ff2700b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff278ff9640 1 -- 172.21.15.31:0/3137557688 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280003bd0 con 0x5637e19840a0 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff278ff9640 1 -- 172.21.15.31:0/3137557688 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280015070 con 0x5637e19840a0 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff278ff9640 1 -- 172.21.15.31:0/3137557688 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280004c80 con 0x5637e19840a0 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff2700b6610 0x7ff2700b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.668+0000 7ff27b7fe640 1 -- 172.21.15.31:0/3473515437 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff274108f00 con 0x7ff2841083d0 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 -- 172.21.15.31:0/3137557688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 msgr2=0x5637e1d53440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff2700b6610 0x7ff2700b8b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff27c010680 tx=0x7ff27c007520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d53440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff280004630 tx=0x7ff28000bb20 comp rx=0 tx=0).stop 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 -- 172.21.15.31:0/3137557688 shutdown_connections 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d53440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d53980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3137557688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19acf60 0x5637e1d52f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 -- 172.21.15.31:0/3137557688 >> 172.21.15.31:0/3137557688 conn(0x5637e19a1350 msgr2=0x5637e1d50df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 -- 172.21.15.31:0/3137557688 shutdown_connections 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.669+0000 7ff292ae9740 1 -- 172.21.15.31:0/3137557688 wait complete. 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 -- start start 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 Processor -- start 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d5ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19acf60 0x5637e1d6a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637e1b1b5f0 con 0x5637e19840a0 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637e1c55010 con 0x5637e198c280 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d616e0 con 0x5637e19acf60 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff292ae9740 10 client.0 _ll_register_callbacks cb 0x5637e1cb4340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff28a311640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d5ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff28a311640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d5ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36064/0 (socket says 172.21.15.31:36064) 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff28a311640 1 -- 172.21.15.31:0/3178120267 learned_addr learned my addr 172.21.15.31:0/3178120267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19acf60 0x5637e1d6a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.670+0000 7ff289b10640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff28a311640 1 -- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19acf60 msgr2=0x5637e1d6a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff28a311640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19acf60 0x5637e1d6a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff28a311640 1 -- 172.21.15.31:0/3178120267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 msgr2=0x5637e1d5ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff28a311640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d5ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff28a311640 1 -- 172.21.15.31:0/3178120267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e1c82a00 con 0x5637e19840a0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff28a311640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d5ea20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff280005d80 tx=0x7ff28000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6399f0ca90b8e872 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff26dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff26dffb640 1 -- 172.21.15.31:0/3178120267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280028060 con 0x5637e19840a0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff26dffb640 1 -- 172.21.15.31:0/3178120267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff280011640 con 0x5637e19840a0 2024-09-21T02:52:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff280015070 con 0x5637e19840a0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff26dffb640 1 -- 172.21.15.31:0/3178120267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff280011840 con 0x5637e19840a0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff292ae9740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637e1d6d650 con 0x5637e19840a0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.671+0000 7ff26dffb640 1 -- 172.21.15.31:0/3178120267 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280015070 con 0x5637e19840a0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff26dffb640 1 -- 172.21.15.31:0/3178120267 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff280022070 con 0x5637e19840a0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff292ae9740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff292ae9740 10 client.15405 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff292ae9740 20 client.15405 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e196b8e0 0x5637e1d70cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff292ae9740 4 client.15405 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e1cccb60 con 0x5637e196b8e0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.672+0000 7ff289b10640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e196b8e0 0x5637e1d70cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.673+0000 7ff26dffb640 10 client.15405 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.673+0000 7ff289b10640 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e196b8e0 0x5637e1d70cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9e69c7ddec11603 server_cookie=ab5104287a8eaa20 in_seq=0 out_seq=0 2024-09-21T02:52:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.673+0000 7ff26dffb640 1 -- 172.21.15.31:0/3178120267 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637e1cccb60 con 0x5637e196b8e0 2024-09-21T02:52:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.673+0000 7ff26dffb640 10 client.15405 handle_command_reply: tid=0 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.673+0000 7ff292ae9740 1 client.15405 shutdown 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 msgr2=0x5637e1d5ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d5ea20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff280005d80 tx=0x7ff28000bfa0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 shutdown_connections 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e196b8e0 0x5637e1d70cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19acf60 0x5637e1d6a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e198c280 0x5637e1d5ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3178120267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d5ea20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff26dffb640 0 client.15405 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 >> 172.21.15.31:0/3178120267 conn(0x5637e19a1350 msgr2=0x5637e1d560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 shutdown_connections 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 1 -- 172.21.15.31:0/3178120267 wait complete. 2024-09-21T02:52:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.674+0000 7ff292ae9740 20 client.15405 trim_cache size 0 max 16384 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 Processor -- start 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 -- start start 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d4bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e198c280 0x5637e1d59020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d6a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d6f000 con 0x5637e19840a0 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d6f6d0 con 0x5637e19acf60 2024-09-21T02:52:47.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff292ae9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d6fda0 con 0x5637e198c280 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d4bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d4bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36068/0 (socket says 172.21.15.31:36068) 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 -- 172.21.15.31:0/3790409156 learned_addr learned my addr 172.21.15.31:0/3790409156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff289b10640 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e198c280 0x5637e1d59020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d6a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 -- 172.21.15.31:0/3790409156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e198c280 msgr2=0x5637e1d59020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e198c280 0x5637e1d59020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 -- 172.21.15.31:0/3790409156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 msgr2=0x5637e1d6a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d6a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.693+0000 7ff28a311640 1 -- 172.21.15.31:0/3790409156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e1d4b080 con 0x5637e19840a0 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.694+0000 7ff28a311640 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d4bad0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b620 tx=0x7ff280004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8defb445b08c09b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.694+0000 7ff26ffff640 1 -- 172.21.15.31:0/3790409156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280028040 con 0x5637e19840a0 2024-09-21T02:52:47.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.694+0000 7ff26ffff640 1 -- 172.21.15.31:0/3790409156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280015070 con 0x5637e19840a0 2024-09-21T02:52:47.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.694+0000 7ff26ffff640 1 -- 172.21.15.31:0/3790409156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280019840 con 0x5637e19840a0 2024-09-21T02:52:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 -- 172.21.15.31:0/3790409156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 msgr2=0x5637e1d4bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d4bad0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b620 tx=0x7ff280004530 comp rx=0 tx=0).stop 2024-09-21T02:52:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 -- 172.21.15.31:0/3790409156 shutdown_connections 2024-09-21T02:52:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e198c280 0x5637e1d59020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d6a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3790409156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e19840a0 0x5637e1d4bad0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 -- 172.21.15.31:0/3790409156 >> 172.21.15.31:0/3790409156 conn(0x5637e19a1350 msgr2=0x5637e1d599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 -- 172.21.15.31:0/3790409156 shutdown_connections 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 -- 172.21.15.31:0/3790409156 wait complete. 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 -- start start 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.695+0000 7ff292ae9740 1 Processor -- start 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 0x5637e1d735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d70b40 con 0x5637e198c280 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d70470 con 0x5637e19acf60 2024-09-21T02:52:47.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637e1d71210 con 0x5637e19840a0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff292ae9740 10 client.0 _ll_register_callbacks cb 0x7ff27c007960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 0x5637e1d735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 0x5637e1d735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36080/0 (socket says 172.21.15.31:36080) 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 -- 172.21.15.31:0/3096186232 learned_addr learned my addr 172.21.15.31:0/3096186232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff28a311640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff28ab12640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 -- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 msgr2=0x5637e1d73080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d73080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 -- 172.21.15.31:0/3096186232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 msgr2=0x5637e1d73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 -- 172.21.15.31:0/3096186232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e1d78970 con 0x5637e198c280 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff289b10640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 0x5637e1d735c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff2740cc2b0 tx=0x7ff2740045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627b960db583f5aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff26dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff26dffb640 1 -- 172.21.15.31:0/3096186232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2740d3a70 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.696+0000 7ff26dffb640 1 -- 172.21.15.31:0/3096186232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff274003c50 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff280015070 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff26dffb640 1 -- 172.21.15.31:0/3096186232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2740d89b0 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 10 client.15411 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637e1ddc540 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff26dffb640 1 -- 172.21.15.31:0/3096186232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff280015070 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff26dffb640 1 -- 172.21.15.31:0/3096186232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff2740ce3c0 con 0x5637e198c280 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 10 client.15411 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 10 client.15411 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 20 client.15411 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e1940c90 0x5637e1ddfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 4 client.15411 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "status"}) -- 0x5637e1cc8610 con 0x5637e1940c90 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.697+0000 7ff28a311640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e1940c90 0x5637e1ddfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff26dffb640 10 client.15411 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff28a311640 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e1940c90 0x5637e1ddfea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97950c4a1661d26b server_cookie=e499f1105ade9b3f in_seq=0 out_seq=0 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff26dffb640 1 -- 172.21.15.31:0/3096186232 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5637e1cc8610 con 0x5637e1940c90 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff26dffb640 10 client.15411 handle_command_reply: tid=0 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 client.15411 shutdown 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 msgr2=0x5637e1d735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 0x5637e1d735c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff2740cc2b0 tx=0x7ff2740045f0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 shutdown_connections 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5637e1940c90 0x5637e1ddfea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637e19acf60 0x5637e1d73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637e198c280 0x5637e1d735c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff26dffb640 0 client.15411 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 --2- 172.21.15.31:0/3096186232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637e19840a0 0x5637e1d73080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 >> 172.21.15.31:0/3096186232 conn(0x5637e19a1350 msgr2=0x5637e1d50600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 shutdown_connections 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 1 -- 172.21.15.31:0/3096186232 wait complete. 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.698+0000 7ff292ae9740 20 client.15411 trim_cache size 0 max 16384 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:47.705 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 3, 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24637, 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 107.287327234, 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 142.15242790100001 2024-09-21T02:52:47.706 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff2700b6610 msgr2=0x7ff2700b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff2700b6610 0x7ff2700b8b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff27c010680 tx=0x7ff27c007520 comp rx=0 tx=0).stop 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 msgr2=0x7ff284199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff284199930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff27402a930 tx=0x7ff274004270 comp rx=0 tx=0).stop 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 shutdown_connections 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff2700b6610 0x7ff2700b8b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff284108d20 0x7ff28419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841083d0 0x7ff284199930 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 --2- 172.21.15.31:0/3473515437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff284102a90 0x7ff284199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 >> 172.21.15.31:0/3473515437 conn(0x7ff284079730 msgr2=0x7ff2841043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 shutdown_connections 2024-09-21T02:52:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:47.699+0000 7ff28c575640 1 -- 172.21.15.31:0/3473515437 wait complete. 2024-09-21T02:52:48.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T02:52:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:48 smithi031 ceph-mon[31209]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 -- 172.21.15.31:0/1804879461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 msgr2=0x7f93fc10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 --2- 172.21.15.31:0/1804879461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc10d850 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009900 tx=0x7f93e402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 -- 172.21.15.31:0/1804879461 shutdown_connections 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 --2- 172.21.15.31:0/1804879461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93fc10de10 0x7f93fc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 --2- 172.21.15.31:0/1804879461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc10d850 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 --2- 172.21.15.31:0/1804879461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93fc1049b0 0x7f93fc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 -- 172.21.15.31:0/1804879461 >> 172.21.15.31:0/1804879461 conn(0x7f93fc079730 msgr2=0x7f93fc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 -- 172.21.15.31:0/1804879461 shutdown_connections 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.095+0000 7f9401348640 1 -- 172.21.15.31:0/1804879461 wait complete. 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 Processor -- start 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 -- start start 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93fc1049b0 0x7f93fc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93fc10de10 0x7f93fc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc113a70 con 0x7f93fc105300 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1138f0 con 0x7f93fc1049b0 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f9401348640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc113bf0 con 0x7f93fc10de10 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f93fa7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f93fa7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45270/0 (socket says 172.21.15.31:45270) 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.096+0000 7f93fa7fc640 1 -- 172.21.15.31:0/2353748779 learned_addr learned my addr 172.21.15.31:0/2353748779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93fc10de10 0x7f93fc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93faffd640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93fc1049b0 0x7f93fc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fa7fc640 1 -- 172.21.15.31:0/2353748779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93fc10de10 msgr2=0x7f93fc19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93fc10de10 0x7f93fc19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fa7fc640 1 -- 172.21.15.31:0/2353748779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93fc1049b0 msgr2=0x7f93fc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93fc1049b0 0x7f93fc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fa7fc640 1 -- 172.21.15.31:0/2353748779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc1a0410 con 0x7f93fc105300 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc19ab00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f93e402a930 tx=0x7f93e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93f3fff640 1 -- 172.21.15.31:0/2353748779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e403f030 con 0x7f93fc105300 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93f3fff640 1 -- 172.21.15.31:0/2353748779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e4003cb0 con 0x7f93fc105300 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f93f3fff640 1 -- 172.21.15.31:0/2353748779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4035070 con 0x7f93fc105300 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc1a06a0 con 0x7f93fc105300 2024-09-21T02:52:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.097+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc1a0bd0 con 0x7f93fc105300 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 Processor -- start 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 -- start start 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f93f3fff640 1 -- 172.21.15.31:0/2353748779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f93e4042030 con 0x7f93fc105300 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34f8f60 0x5589c389ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 0x5589c389f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c389f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589c3816c10 con 0x5589c34d00a0 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5589c3806040 con 0x5589c34f8f60 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f94078bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c3809e50 con 0x5589c34d8280 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f93faffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 0x5589c389f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f93faffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 0x5589c389f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45286/0 (socket says 172.21.15.31:45286) 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.100+0000 7f93faffd640 1 -- 172.21.15.31:0/1083219297 learned_addr learned my addr 172.21.15.31:0/1083219297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34f8f60 0x5589c389ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c389f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93faffd640 1 -- 172.21.15.31:0/1083219297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 msgr2=0x5589c389f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93faffd640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c389f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93faffd640 1 -- 172.21.15.31:0/1083219297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34f8f60 msgr2=0x5589c389ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93faffd640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34f8f60 0x5589c389ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93faffd640 1 -- 172.21.15.31:0/1083219297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c3564540 con 0x5589c34d00a0 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34f8f60 0x5589c389ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c389f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93faffd640 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 0x5589c389f440 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f93f400cbb0 tx=0x7f93f400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cc66683d7d679e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93f3fff640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93e80b6610 0x7f93e80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93f17fa640 1 -- 172.21.15.31:0/1083219297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4005220 con 0x5589c34d00a0 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93f17fa640 1 -- 172.21.15.31:0/1083219297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f4004c50 con 0x5589c34d00a0 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93e80b6610 0x7f93e80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93f3fff640 1 -- 172.21.15.31:0/2353748779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f93e4107ae0 con 0x7f93fc105300 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.101+0000 7f93f17fa640 1 -- 172.21.15.31:0/1083219297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4010750 con 0x5589c34d00a0 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 -- 172.21.15.31:0/1083219297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 msgr2=0x5589c389f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93e80b6610 0x7f93e80b8b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00ae10 tx=0x7f93ec00a3c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 0x5589c389f440 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f93f400cbb0 tx=0x7f93f400e720 comp rx=0 tx=0).stop 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 -- 172.21.15.31:0/1083219297 shutdown_connections 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c389f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d00a0 0x5589c389f440 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 --2- 172.21.15.31:0/1083219297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34f8f60 0x5589c389ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 -- 172.21.15.31:0/1083219297 >> 172.21.15.31:0/1083219297 conn(0x5589c34ed350 msgr2=0x5589c389cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:49.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 -- 172.21.15.31:0/1083219297 shutdown_connections 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.102+0000 7f94078bc740 1 -- 172.21.15.31:0/1083219297 wait complete. 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 -- start start 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 Processor -- start 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 0x5589c38af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34f8f60 0x5589c38badf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589c36675f0 con 0x5589c34d8280 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5589c37a1010 con 0x5589c34d00a0 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c38b1a90 con 0x5589c34f8f60 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f94078bc740 10 client.0 _ll_register_callbacks cb 0x5589c3800340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f93fa7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 0x5589c38af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f93fa7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 0x5589c38af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45288/0 (socket says 172.21.15.31:45288) 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.103+0000 7f93fa7fc640 1 -- 172.21.15.31:0/203352105 learned_addr learned my addr 172.21.15.31:0/203352105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93faffd640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34f8f60 0x5589c38badf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fa7fc640 1 -- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34f8f60 msgr2=0x5589c38badf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34f8f60 0x5589c38badf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fa7fc640 1 -- 172.21.15.31:0/203352105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 msgr2=0x5589c38aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fa7fc640 1 -- 172.21.15.31:0/203352105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c37cea00 con 0x5589c34d8280 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34f8f60 0x5589c38badf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 0x5589c38af190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cdb00 tx=0x7f93e40c9ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33144482231ed85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93e27fc640 1 -- 172.21.15.31:0/203352105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4049070 con 0x5589c34d8280 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93e27fc640 1 -- 172.21.15.31:0/203352105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f93e403bbe0 con 0x5589c34d8280 2024-09-21T02:52:49.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f4004c50 con 0x5589c34d8280 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93e27fc640 1 -- 172.21.15.31:0/203352105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93e40cd680 con 0x5589c34d8280 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f94078bc740 10 client.15423 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589c38bd8a0 con 0x5589c34d8280 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.104+0000 7f93e27fc640 1 -- 172.21.15.31:0/203352105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f4004c50 con 0x5589c34d8280 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f93e27fc640 1 -- 172.21.15.31:0/203352105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f93e40d7100 con 0x5589c34d8280 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f94078bc740 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f94078bc740 10 client.15423 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f94078bc740 20 client.15423 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f94078bc740 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c34b78e0 0x5589c38c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f94078bc740 4 client.15423 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589c3818b60 con 0x5589c34b78e0 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.105+0000 7f93faffd640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c34b78e0 0x5589c38c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.106+0000 7f93e27fc640 10 client.15423 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:49.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.106+0000 7f93faffd640 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c34b78e0 0x5589c38c1360 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86f9480d93f01265 server_cookie=542ef4c45cd5704 in_seq=0 out_seq=0 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.106+0000 7f93e27fc640 1 -- 172.21.15.31:0/203352105 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589c3818b60 con 0x5589c34b78e0 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.106+0000 7f93e27fc640 10 client.15423 handle_command_reply: tid=0 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.106+0000 7f94078bc740 1 client.15423 shutdown 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 msgr2=0x5589c38af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 0x5589c38af190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cdb00 tx=0x7f93e40c9ad0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 shutdown_connections 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c34b78e0 0x5589c38c1360 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34f8f60 0x5589c38badf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34d8280 0x5589c38af190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 --2- 172.21.15.31:0/203352105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 >> 172.21.15.31:0/203352105 conn(0x5589c34ed350 msgr2=0x5589c38a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:49.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f93e27fc640 0 client.15423 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:49.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 shutdown_connections 2024-09-21T02:52:49.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 1 -- 172.21.15.31:0/203352105 wait complete. 2024-09-21T02:52:49.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.107+0000 7f94078bc740 20 client.15423 trim_cache size 0 max 16384 2024-09-21T02:52:49.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 Processor -- start 2024-09-21T02:52:49.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 -- start start 2024-09-21T02:52:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589c38bf6a0 con 0x5589c34f8f60 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5589c38bfd70 con 0x5589c34d00a0 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f94078bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c38c0440 con 0x5589c34d8280 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.125+0000 7f93fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38bf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45292/0 (socket says 172.21.15.31:45292) 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2232387894 learned_addr learned my addr 172.21.15.31:0/2232387894 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93faffd640 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2232387894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 msgr2=0x5589c38bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2232387894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 msgr2=0x5589c38ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2232387894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c389fcf0 con 0x5589c34f8f60 2024-09-21T02:52:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38bf130 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00a740 tx=0x7f93ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e1050e02a56f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93f0ff9640 1 -- 172.21.15.31:0/2232387894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec01e8f0 con 0x5589c34f8f60 2024-09-21T02:52:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93f0ff9640 1 -- 172.21.15.31:0/2232387894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec026400 con 0x5589c34f8f60 2024-09-21T02:52:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.126+0000 7f93f0ff9640 1 -- 172.21.15.31:0/2232387894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec010630 con 0x5589c34f8f60 2024-09-21T02:52:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 -- 172.21.15.31:0/2232387894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 msgr2=0x5589c38bf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38bf130 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00a740 tx=0x7f93ec012000 comp rx=0 tx=0).stop 2024-09-21T02:52:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 -- 172.21.15.31:0/2232387894 shutdown_connections 2024-09-21T02:52:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38bf130 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 --2- 172.21.15.31:0/2232387894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 -- 172.21.15.31:0/2232387894 >> 172.21.15.31:0/2232387894 conn(0x5589c34ed350 msgr2=0x5589c38a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 -- 172.21.15.31:0/2232387894 shutdown_connections 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.127+0000 7f94078bc740 1 -- 172.21.15.31:0/2232387894 wait complete. 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 -- start start 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 Processor -- start 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38c4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5589c38c11e0 con 0x5589c34f8f60 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5589c38c0b10 con 0x5589c34d00a0 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589c38c18b0 con 0x5589c34d8280 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f94078bc740 10 client.0 _ll_register_callbacks cb 0x7f93e4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:49.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45296/0 (socket says 172.21.15.31:45296) 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2907925791 learned_addr learned my addr 172.21.15.31:0/2907925791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93faffd640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fa7fc640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 msgr2=0x5589c38c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2907925791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 msgr2=0x5589c38c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.128+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93fb7fe640 1 -- 172.21.15.31:0/2907925791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c38c90c0 con 0x5589c34f8f60 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93fb7fe640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38c8b30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93ec026bf0 tx=0x7f93ec0267e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd826029fee92050 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93e27fc640 1 -- 172.21.15.31:0/2907925791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec036060 con 0x5589c34f8f60 2024-09-21T02:52:49.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93e27fc640 1 -- 172.21.15.31:0/2907925791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f93ec004c50 con 0x5589c34f8f60 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec026400 con 0x5589c34f8f60 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93e27fc640 1 -- 172.21.15.31:0/2907925791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93ec004e50 con 0x5589c34f8f60 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f94078bc740 10 client.15429 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589c38ca320 con 0x5589c34f8f60 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93e27fc640 1 -- 172.21.15.31:0/2907925791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93ec026400 con 0x5589c34f8f60 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.129+0000 7f93e27fc640 1 -- 172.21.15.31:0/2907925791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f93ec005370 con 0x5589c34f8f60 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f94078bc740 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f94078bc740 10 client.15429 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f94078bc740 20 client.15429 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f94078bc740 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c348cc90 0x5589c392fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f94078bc740 4 client.15429 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "status"}) -- 0x5589c3814610 con 0x5589c348cc90 2024-09-21T02:52:49.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f93faffd640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c348cc90 0x5589c392fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:49.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f93faffd640 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c348cc90 0x5589c392fa70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6412896be9af6a77 server_cookie=f22d7e1748b043d8 in_seq=0 out_seq=0 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.130+0000 7f93e27fc640 10 client.15429 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f93e27fc640 1 -- 172.21.15.31:0/2907925791 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5589c3814610 con 0x5589c348cc90 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f93e27fc640 10 client.15429 handle_command_reply: tid=0 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 client.15429 shutdown 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 msgr2=0x5589c38c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38c8b30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93ec026bf0 tx=0x7f93ec0267e0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 shutdown_connections 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5589c348cc90 0x5589c392fa70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589c34d8280 0x5589c38c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5589c34f8f60 0x5589c38c8b30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 --2- 172.21.15.31:0/2907925791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589c34d00a0 0x5589c38c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f93e27fc640 0 client.15429 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:52:49.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 >> 172.21.15.31:0/2907925791 conn(0x5589c34ed350 msgr2=0x5589c38a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 shutdown_connections 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 1 -- 172.21.15.31:0/2907925791 wait complete. 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.131+0000 7f94078bc740 20 client.15429 trim_cache size 0 max 16384 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24655, 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 109.729053544, 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 138.382241881 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:49.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93e80b6610 msgr2=0x7f93e80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93e80b6610 0x7f93e80b8b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00ae10 tx=0x7f93ec00a3c0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 msgr2=0x7f93fc19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc19ab00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f93e402a930 tx=0x7f93e4004270 comp rx=0 tx=0).stop 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 shutdown_connections 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93e80b6610 0x7f93e80b8b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93fc10de10 0x7f93fc19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93fc105300 0x7f93fc19ab00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 --2- 172.21.15.31:0/2353748779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93fc1049b0 0x7f93fc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 >> 172.21.15.31:0/2353748779 conn(0x7f93fc079730 msgr2=0x7f93fc1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 shutdown_connections 2024-09-21T02:52:49.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:49.132+0000 7f9401348640 1 -- 172.21.15.31:0/2353748779 wait complete. 2024-09-21T02:52:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:48 smithi165 ceph-mon[34818]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:48 smithi178 ceph-mon[34759]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:52:49.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T02:52:50.203 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 -- 172.21.15.31:0/1049683911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce64103b80 msgr2=0x7fce64103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/1049683911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce64103b80 0x7fce64103f90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fce54009900 tx=0x7fce5402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 -- 172.21.15.31:0/1049683911 shutdown_connections 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/1049683911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce641094c0 0x7fce641098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/1049683911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce64109e10 0x7fce64110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/1049683911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce64103b80 0x7fce64103f90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 -- 172.21.15.31:0/1049683911 >> 172.21.15.31:0/1049683911 conn(0x7fce640ff610 msgr2=0x7fce64101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 -- 172.21.15.31:0/1049683911 shutdown_connections 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.680+0000 7fce6a7e0640 1 -- 172.21.15.31:0/1049683911 wait complete. 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 Processor -- start 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 -- start start 2024-09-21T02:52:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce64103b80 0x7fce6419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 0x7fce6419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce64109e10 0x7fce6419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce64113ad0 con 0x7fce641094c0 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fce64113950 con 0x7fce64109e10 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce6a7e0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce64113c50 con 0x7fce64103b80 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 0x7fce6419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 0x7fce6419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45304/0 (socket says 172.21.15.31:45304) 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.681+0000 7fce637fe640 1 -- 172.21.15.31:0/3488729298 learned_addr learned my addr 172.21.15.31:0/3488729298 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce68d7d640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce64109e10 0x7fce6419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce63fff640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce64103b80 0x7fce6419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce637fe640 1 -- 172.21.15.31:0/3488729298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce64103b80 msgr2=0x7fce6419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce637fe640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce64103b80 0x7fce6419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce637fe640 1 -- 172.21.15.31:0/3488729298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce64109e10 msgr2=0x7fce6419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce637fe640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce64109e10 0x7fce6419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce637fe640 1 -- 172.21.15.31:0/3488729298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641a03d0 con 0x7fce641094c0 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce63fff640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce64103b80 0x7fce6419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce637fe640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 0x7fce6419aa00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fce5800d740 tx=0x7fce5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce617fa640 1 -- 172.21.15.31:0/3488729298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58017070 con 0x7fce641094c0 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce617fa640 1 -- 172.21.15.31:0/3488729298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce5800de50 con 0x7fce641094c0 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641a06c0 con 0x7fce641094c0 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce617fa640 1 -- 172.21.15.31:0/3488729298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5800f620 con 0x7fce641094c0 2024-09-21T02:52:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.682+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce641a6e20 con 0x7fce641094c0 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 Processor -- start 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 -- start start 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce617fa640 1 -- 172.21.15.31:0/3488729298 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fce5801a070 con 0x7fce641094c0 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a83ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a83f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a83f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c96a7b6c10 con 0x55c96a498f60 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c96a7a6040 con 0x55c96a4700a0 2024-09-21T02:52:50.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce70d54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c96a7a9e50 con 0x55c96a478280 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce68d7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a83ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce68d7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a83ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45310/0 (socket says 172.21.15.31:45310) 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.685+0000 7fce68d7d640 1 -- 172.21.15.31:0/3338442010 learned_addr learned my addr 172.21.15.31:0/3338442010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce63fff640 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a83f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce637fe640 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a83f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce68d7d640 1 -- 172.21.15.31:0/3338442010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 msgr2=0x55c96a83f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce68d7d640 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a83f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce68d7d640 1 -- 172.21.15.31:0/3338442010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 msgr2=0x55c96a83f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce68d7d640 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a83f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce68d7d640 1 -- 172.21.15.31:0/3338442010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96a504540 con 0x55c96a498f60 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce63fff640 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a83f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce617fa640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fce4c0b6560 0x7fce4c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce68d7d640 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a83ef00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fce5c010370 tx=0x7fce5c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04dbe543ee103f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce63fff640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fce4c0b6560 0x7fce4c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce617fa640 1 -- 172.21.15.31:0/3488729298 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fce580e6e90 con 0x7fce641094c0 2024-09-21T02:52:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce4a7fc640 1 -- 172.21.15.31:0/3338442010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c016070 con 0x55c96a498f60 2024-09-21T02:52:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce4a7fc640 1 -- 172.21.15.31:0/3338442010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce5c004d20 con 0x55c96a498f60 2024-09-21T02:52:50.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.686+0000 7fce4a7fc640 1 -- 172.21.15.31:0/3338442010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c005420 con 0x55c96a498f60 2024-09-21T02:52:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 -- 172.21.15.31:0/3338442010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 msgr2=0x55c96a83ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a83ef00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fce5c010370 tx=0x7fce5c010830 comp rx=0 tx=0).stop 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce63fff640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fce4c0b6560 0x7fce4c0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce54031440 tx=0x7fce54005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 -- 172.21.15.31:0/3338442010 shutdown_connections 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a83f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a83f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 --2- 172.21.15.31:0/3338442010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a83ef00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.687+0000 7fce70d54740 1 -- 172.21.15.31:0/3338442010 >> 172.21.15.31:0/3338442010 conn(0x55c96a48d350 msgr2=0x55c96a83cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 -- 172.21.15.31:0/3338442010 shutdown_connections 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 -- 172.21.15.31:0/3338442010 wait complete. 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 -- start start 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 Processor -- start 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a84a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a84ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a856a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c96a6075f0 con 0x55c96a498f60 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c96a741010 con 0x55c96a4700a0 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.688+0000 7fce70d54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c96a84d6e0 con 0x55c96a478280 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce70d54740 10 client.0 _ll_register_callbacks cb 0x55c96a7a0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce68d7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a856a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce68d7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a856a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45316/0 (socket says 172.21.15.31:45316) 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a84a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce68d7d640 1 -- 172.21.15.31:0/2756994907 learned_addr learned my addr 172.21.15.31:0/2756994907 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce637fe640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a84ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 -- 172.21.15.31:0/2756994907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 msgr2=0x55c96a84ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a84ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 -- 172.21.15.31:0/2756994907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 msgr2=0x55c96a856a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a856a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 -- 172.21.15.31:0/2756994907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96a76ea00 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce68d7d640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a856a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce63fff640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a84a7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce54009230 tx=0x7fce54009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ef059c50f9cfa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.689+0000 7fce437fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce437fe640 1 -- 172.21.15.31:0/2756994907 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5404b070 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce437fe640 1 -- 172.21.15.31:0/2756994907 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fce54005460 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c004d20 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce437fe640 1 -- 172.21.15.31:0/2756994907 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce54057050 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce70d54740 10 client.25109 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c96a8595b0 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.690+0000 7fce437fe640 1 -- 172.21.15.31:0/2756994907 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce5c004d20 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce437fe640 1 -- 172.21.15.31:0/2756994907 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fce54003610 con 0x55c96a4700a0 2024-09-21T02:52:50.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce70d54740 10 client.25109 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce70d54740 10 client.25109 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce70d54740 20 client.25109 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce70d54740 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a4578e0 0x55c96a85d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce70d54740 4 client.25109 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96a7b8b60 con 0x55c96a4578e0 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce637fe640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a4578e0 0x55c96a85d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce637fe640 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a4578e0 0x55c96a85d0a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90576c0675cb73 server_cookie=681876939bc17428 in_seq=0 out_seq=0 2024-09-21T02:52:50.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.691+0000 7fce437fe640 10 client.25109 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.692+0000 7fce437fe640 1 -- 172.21.15.31:0/2756994907 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c96a7b8b60 con 0x55c96a4578e0 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.692+0000 7fce437fe640 10 client.25109 handle_command_reply: tid=0 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.692+0000 7fce70d54740 1 client.25109 shutdown 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.692+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 msgr2=0x55c96a84a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.692+0000 7fce70d54740 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a84a7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce54009230 tx=0x7fce54009260 comp rx=0 tx=0).stop 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 shutdown_connections 2024-09-21T02:52:50.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a4578e0 0x55c96a85d0a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a84ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a498f60 0x55c96a856a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 --2- 172.21.15.31:0/2756994907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a84a7b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce437fe640 0 client.25109 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:50.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 >> 172.21.15.31:0/2756994907 conn(0x55c96a48d350 msgr2=0x55c96a8420e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:50.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 shutdown_connections 2024-09-21T02:52:50.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 1 -- 172.21.15.31:0/2756994907 wait complete. 2024-09-21T02:52:50.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.693+0000 7fce70d54740 20 client.25109 trim_cache size 0 max 16384 2024-09-21T02:52:50.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 Processor -- start 2024-09-21T02:52:50.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 -- start start 2024-09-21T02:52:50.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 0x55c96a856970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a856eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a498f60 0x55c96a865770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c96a85b3e0 con 0x55c96a4700a0 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c96a85bab0 con 0x55c96a498f60 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce70d54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c96a85c180 con 0x55c96a478280 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 0x55c96a856970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 0x55c96a856970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45332/0 (socket says 172.21.15.31:45332) 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.711+0000 7fce63fff640 1 -- 172.21.15.31:0/1513038408 learned_addr learned my addr 172.21.15.31:0/1513038408 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce68d7d640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a498f60 0x55c96a865770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce637fe640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a856eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce63fff640 1 -- 172.21.15.31:0/1513038408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 msgr2=0x55c96a856eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce63fff640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a856eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce63fff640 1 -- 172.21.15.31:0/1513038408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a498f60 msgr2=0x55c96a865770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce63fff640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a498f60 0x55c96a865770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce63fff640 1 -- 172.21.15.31:0/1513038408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96a84ba70 con 0x55c96a4700a0 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce637fe640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a856eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce68d7d640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a498f60 0x55c96a865770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce63fff640 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 0x55c96a856970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fce54003a60 tx=0x7fce54032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edc7038b92809f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce49ffb640 1 -- 172.21.15.31:0/1513038408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5404b070 con 0x55c96a4700a0 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce49ffb640 1 -- 172.21.15.31:0/1513038408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce54060070 con 0x55c96a4700a0 2024-09-21T02:52:50.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.712+0000 7fce49ffb640 1 -- 172.21.15.31:0/1513038408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54035050 con 0x55c96a4700a0 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 -- 172.21.15.31:0/1513038408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 msgr2=0x55c96a856970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 0x55c96a856970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fce54003a60 tx=0x7fce54032fa0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 -- 172.21.15.31:0/1513038408 shutdown_connections 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a478280 0x55c96a856eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a498f60 0x55c96a865770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 --2- 172.21.15.31:0/1513038408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a4700a0 0x55c96a856970 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 -- 172.21.15.31:0/1513038408 >> 172.21.15.31:0/1513038408 conn(0x55c96a48d350 msgr2=0x55c96a849030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 -- 172.21.15.31:0/1513038408 shutdown_connections 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.713+0000 7fce70d54740 1 -- 172.21.15.31:0/1513038408 wait complete. 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 -- start start 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 Processor -- start 2024-09-21T02:52:50.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a85e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 0x55c96a85ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a498f60 0x55c96a8c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c96a85cf20 con 0x55c96a478280 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c96a85c850 con 0x55c96a4700a0 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c96a85d5f0 con 0x55c96a498f60 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce70d54740 10 client.0 _ll_register_callbacks cb 0x7fce54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 0x55c96a85ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 0x55c96a85ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45344/0 (socket says 172.21.15.31:45344) 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce637fe640 1 -- 172.21.15.31:0/2179040538 learned_addr learned my addr 172.21.15.31:0/2179040538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.714+0000 7fce68d7d640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a498f60 0x55c96a8c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce63fff640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a85e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce637fe640 1 -- 172.21.15.31:0/2179040538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a498f60 msgr2=0x55c96a8c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce637fe640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a498f60 0x55c96a8c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce637fe640 1 -- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 msgr2=0x55c96a85e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce637fe640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a85e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce637fe640 1 -- 172.21.15.31:0/2179040538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96a8c5cc0 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce637fe640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 0x55c96a85ea60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fce580aceb0 tx=0x7fce58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e14c9dfd55475b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce437fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce437fe640 1 -- 172.21.15.31:0/2179040538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58021040 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce437fe640 1 -- 172.21.15.31:0/2179040538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fce580a9190 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce54060070 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce437fe640 1 -- 172.21.15.31:0/2179040538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce580aca30 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce70d54740 10 client.15447 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c96a8ccf50 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.715+0000 7fce437fe640 1 -- 172.21.15.31:0/2179040538 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce54060070 con 0x55c96a478280 2024-09-21T02:52:50.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce437fe640 1 -- 172.21.15.31:0/2179040538 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fce580b0070 con 0x55c96a478280 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce70d54740 10 client.15447 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce70d54740 10 client.15447 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce70d54740 20 client.15447 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce70d54740 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a42cc90 0x55c96a8d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce70d54740 4 client.15447 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "status"}) -- 0x55c96a7b4610 con 0x55c96a42cc90 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce63fff640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a42cc90 0x55c96a8d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce437fe640 10 client.15447 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.716+0000 7fce63fff640 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a42cc90 0x55c96a8d07a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e78c931dddee5eac server_cookie=404b059ff9918726 in_seq=0 out_seq=0 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce437fe640 1 -- 172.21.15.31:0/2179040538 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c96a7b4610 con 0x55c96a42cc90 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce437fe640 10 client.15447 handle_command_reply: tid=0 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 client.15447 shutdown 2024-09-21T02:52:50.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 msgr2=0x55c96a85ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 0x55c96a85ea60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fce580aceb0 tx=0x7fce58000bc0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 shutdown_connections 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c96a42cc90 0x55c96a8d07a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c96a498f60 0x55c96a8c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c96a478280 0x55c96a85ea60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce437fe640 0 client.15447 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 --2- 172.21.15.31:0/2179040538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c96a4700a0 0x55c96a85e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 >> 172.21.15.31:0/2179040538 conn(0x55c96a48d350 msgr2=0x55c96a84bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 shutdown_connections 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 1 -- 172.21.15.31:0/2179040538 wait complete. 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.717+0000 7fce70d54740 20 client.15447 trim_cache size 0 max 16384 2024-09-21T02:52:50.723 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24659, 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 112.32682981400001, 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 142.50717760200001 2024-09-21T02:52:50.724 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fce4c0b6560 msgr2=0x7fce4c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fce4c0b6560 0x7fce4c0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce54031440 tx=0x7fce54005a20 comp rx=0 tx=0).stop 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 msgr2=0x7fce6419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 0x7fce6419aa00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fce5800d740 tx=0x7fce5800dc00 comp rx=0 tx=0).stop 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 shutdown_connections 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fce4c0b6560 0x7fce4c0b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce64109e10 0x7fce6419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce641094c0 0x7fce6419aa00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 --2- 172.21.15.31:0/3488729298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce64103b80 0x7fce6419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:50.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 >> 172.21.15.31:0/3488729298 conn(0x7fce640ff610 msgr2=0x7fce64101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:50.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 shutdown_connections 2024-09-21T02:52:50.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:50.718+0000 7fce6a7e0640 1 -- 172.21.15.31:0/3488729298 wait complete. 2024-09-21T02:52:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:50 smithi031 ceph-mon[31209]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:50 smithi165 ceph-mon[34818]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:50 smithi178 ceph-mon[34759]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:51.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T02:52:51.823 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:52.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.299+0000 7fea2f67d640 1 -- 172.21.15.31:0/3473908935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 msgr2=0x7fea2810fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.299+0000 7fea2f67d640 1 --2- 172.21.15.31:0/3473908935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2810fcc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea1802a480 comp rx=0 tx=0).stop 2024-09-21T02:52:52.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 -- 172.21.15.31:0/3473908935 shutdown_connections 2024-09-21T02:52:52.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 --2- 172.21.15.31:0/3473908935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea28110200 0x7fea28112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 --2- 172.21.15.31:0/3473908935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2810fcc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 --2- 172.21.15.31:0/3473908935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea28102650 0x7fea28102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 -- 172.21.15.31:0/3473908935 >> 172.21.15.31:0/3473908935 conn(0x7fea28079760 msgr2=0x7fea28079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:52.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 -- 172.21.15.31:0/3473908935 shutdown_connections 2024-09-21T02:52:52.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 -- 172.21.15.31:0/3473908935 wait complete. 2024-09-21T02:52:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 Processor -- start 2024-09-21T02:52:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.300+0000 7fea2f67d640 1 -- start start 2024-09-21T02:52:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2f67d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea28102650 0x7fea2819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2f67d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2f67d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea28110200 0x7fea281a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2f67d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea28115ca0 con 0x7fea28102fa0 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2f67d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fea28115b20 con 0x7fea28102650 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2f67d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea28115e20 con 0x7fea28110200 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45368/0 (socket says 172.21.15.31:45368) 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 -- 172.21.15.31:0/1659575170 learned_addr learned my addr 172.21.15.31:0/1659575170 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2d419640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea28102650 0x7fea2819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea28110200 0x7fea281a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 -- 172.21.15.31:0/1659575170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea28110200 msgr2=0x7fea281a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea28110200 0x7fea281a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 -- 172.21.15.31:0/1659575170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea28102650 msgr2=0x7fea2819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea28102650 0x7fea2819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 -- 172.21.15.31:0/1659575170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea281a25c0 con 0x7fea28102fa0 2024-09-21T02:52:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.301+0000 7fea2cc18640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2819cbf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea18005940 tx=0x7fea180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.302+0000 7fea1e7fc640 1 -- 172.21.15.31:0/1659575170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1803f030 con 0x7fea28102fa0 2024-09-21T02:52:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.302+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea281a2850 con 0x7fea28102fa0 2024-09-21T02:52:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.302+0000 7fea1e7fc640 1 -- 172.21.15.31:0/1659575170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea18003cb0 con 0x7fea28102fa0 2024-09-21T02:52:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.302+0000 7fea1e7fc640 1 -- 172.21.15.31:0/1659575170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18035070 con 0x7fea28102fa0 2024-09-21T02:52:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.302+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281a9010 con 0x7fea28102fa0 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 Processor -- start 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 -- start start 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a1d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5601b9994c10 con 0x5601b9676f60 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601b9984040 con 0x5601b9656280 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea35bf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601b9987e50 con 0x5601b964e0a0 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea1e7fc640 1 -- 172.21.15.31:0/1659575170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fea18042030 con 0x7fea28102fa0 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea2dc1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea2dc1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45370/0 (socket says 172.21.15.31:45370) 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea2dc1a640 1 -- 172.21.15.31:0/3296291810 learned_addr learned my addr 172.21.15.31:0/3296291810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.305+0000 7fea2d419640 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2cc18640 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a1d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2dc1a640 1 -- 172.21.15.31:0/3296291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 msgr2=0x5601b9a1d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a1d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2dc1a640 1 -- 172.21.15.31:0/3296291810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 msgr2=0x5601b9a1d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a1d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2dc1a640 1 -- 172.21.15.31:0/3296291810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b96e2540 con 0x5601b9676f60 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1cf00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fea20010370 tx=0x7fea20010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e291fa1c327adb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea137fe640 1 -- 172.21.15.31:0/3296291810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20016070 con 0x5601b9676f60 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea137fe640 1 -- 172.21.15.31:0/3296291810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea20004d20 con 0x5601b9676f60 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea137fe640 1 -- 172.21.15.31:0/3296291810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20005350 con 0x5601b9676f60 2024-09-21T02:52:52.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea1e7fc640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fea140b6560 0x7fea140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.306+0000 7fea2d419640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fea140b6560 0x7fea140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea1e7fc640 1 -- 172.21.15.31:0/1659575170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fea18107cd0 con 0x7fea28102fa0 2024-09-21T02:52:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 -- 172.21.15.31:0/3296291810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 msgr2=0x5601b9a1cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea2d419640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fea140b6560 0x7fea140b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5601b9a1bb80 tx=0x7fea2400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1cf00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fea20010370 tx=0x7fea20010830 comp rx=0 tx=0).stop 2024-09-21T02:52:52.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 -- 172.21.15.31:0/3296291810 shutdown_connections 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a1d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a1d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 --2- 172.21.15.31:0/3296291810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1cf00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 -- 172.21.15.31:0/3296291810 >> 172.21.15.31:0/3296291810 conn(0x5601b966b350 msgr2=0x5601b9a1adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 -- 172.21.15.31:0/3296291810 shutdown_connections 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.307+0000 7fea35bf1740 1 -- 172.21.15.31:0/3296291810 wait complete. 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 -- start start 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 Processor -- start 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b964e0a0 0x5601b9a28a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9656280 0x5601b9a28f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a34a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5601b97e55f0 con 0x5601b9676f60 2024-09-21T02:52:52.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601b991f010 con 0x5601b964e0a0 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a2b6e0 con 0x5601b9656280 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea35bf1740 10 client.0 _ll_register_callbacks cb 0x5601b997e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea2dc1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a34a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea2dc1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a34a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45380/0 (socket says 172.21.15.31:45380) 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4031005662 learned_addr learned my addr 172.21.15.31:0/4031005662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.308+0000 7fea2d419640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b964e0a0 0x5601b9a28a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2cc18640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9656280 0x5601b9a28f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4031005662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9656280 msgr2=0x5601b9a28f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9656280 0x5601b9a28f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b964e0a0 msgr2=0x5601b9a28a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b964e0a0 0x5601b9a28a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4031005662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b994ca00 con 0x5601b9676f60 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a34a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fea20006010 tx=0x7fea20012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf173da4214bb63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea10ff9640 1 -- 172.21.15.31:0/4031005662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20029060 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea10ff9640 1 -- 172.21.15.31:0/4031005662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fea20012890 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea20004d20 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea10ff9640 1 -- 172.21.15.31:0/4031005662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea20016070 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea35bf1740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.309+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5601b9a37610 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea10ff9640 1 -- 172.21.15.31:0/4031005662 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea20004d20 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea10ff9640 1 -- 172.21.15.31:0/4031005662 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fea2000f040 con 0x5601b9676f60 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea35bf1740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea35bf1740 10 client.15459 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea35bf1740 20 client.15459 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b96358e0 0x5601b9a3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea35bf1740 4 client.15459 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:52:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b9996b60 con 0x5601b96358e0 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.310+0000 7fea2d419640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b96358e0 0x5601b9a3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.311+0000 7fea2d419640 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b96358e0 0x5601b9a3ac90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815f0f778a66ef4a server_cookie=227f5a2323f09dcd in_seq=0 out_seq=0 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.311+0000 7fea10ff9640 10 client.15459 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.311+0000 7fea10ff9640 1 -- 172.21.15.31:0/4031005662 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601b9996b60 con 0x5601b96358e0 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.311+0000 7fea10ff9640 10 client.15459 handle_command_reply: tid=0 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.311+0000 7fea35bf1740 1 client.15459 shutdown 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 msgr2=0x5601b9a34a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a34a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fea20006010 tx=0x7fea20012510 comp rx=0 tx=0).stop 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 shutdown_connections 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b96358e0 0x5601b9a3ac90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9656280 0x5601b9a28f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a34a00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4031005662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b964e0a0 0x5601b9a28a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 >> 172.21.15.31:0/4031005662 conn(0x5601b966b350 msgr2=0x5601b9a200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:52.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea10ff9640 0 client.15459 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 shutdown_connections 2024-09-21T02:52:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 1 -- 172.21.15.31:0/4031005662 wait complete. 2024-09-21T02:52:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.312+0000 7fea35bf1740 20 client.15459 trim_cache size 0 max 16384 2024-09-21T02:52:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 Processor -- start 2024-09-21T02:52:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 -- start start 2024-09-21T02:52:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a34eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a38fd0 con 0x5601b9676f60 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a396a0 con 0x5601b9656280 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.330+0000 7fea35bf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a39d70 con 0x5601b964e0a0 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2d419640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45388/0 (socket says 172.21.15.31:45388) 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2d419640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a34970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:43684/0 (socket says 172.21.15.31:43684) 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4210508126 learned_addr learned my addr 172.21.15.31:0/4210508126 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2cc18640 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a34eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4210508126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 msgr2=0x5601b9a34970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a34970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4210508126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 msgr2=0x5601b9a34eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a34eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 -- 172.21.15.31:0/4210508126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b9a1b6e0 con 0x5601b9676f60 2024-09-21T02:52:52.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2d419640 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a34970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:52.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1a600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fea20010870 tx=0x7fea20005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60c2cd83a7a35f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea12ffd640 1 -- 172.21.15.31:0/4210508126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20029040 con 0x5601b9676f60 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea12ffd640 1 -- 172.21.15.31:0/4210508126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea20004cc0 con 0x5601b9676f60 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.331+0000 7fea12ffd640 1 -- 172.21.15.31:0/4210508126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20012890 con 0x5601b9676f60 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 -- 172.21.15.31:0/4210508126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 msgr2=0x5601b9a1a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1a600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fea20010870 tx=0x7fea20005c50 comp rx=0 tx=0).stop 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 -- 172.21.15.31:0/4210508126 shutdown_connections 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b9676f60 0x5601b9a1a600 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a34eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 --2- 172.21.15.31:0/4210508126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b964e0a0 0x5601b9a34970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 -- 172.21.15.31:0/4210508126 >> 172.21.15.31:0/4210508126 conn(0x5601b966b350 msgr2=0x5601b9a289f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 -- 172.21.15.31:0/4210508126 shutdown_connections 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 -- 172.21.15.31:0/4210508126 wait complete. 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.332+0000 7fea35bf1740 1 -- start start 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 Processor -- start 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 0x5601b9a432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9676f60 0x5601b9a3c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a3ab10 con 0x5601b964e0a0 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a3a440 con 0x5601b9656280 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601b9a3b1e0 con 0x5601b9676f60 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea35bf1740 10 client.0 _ll_register_callbacks cb 0x7fea20009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea2d419640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 0x5601b9a432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea2d419640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 0x5601b9a432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45398/0 (socket says 172.21.15.31:45398) 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea2d419640 1 -- 172.21.15.31:0/179898351 learned_addr learned my addr 172.21.15.31:0/179898351 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea2cc18640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.333+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9676f60 0x5601b9a3c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2d419640 1 -- 172.21.15.31:0/179898351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9676f60 msgr2=0x5601b9a3c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2d419640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9676f60 0x5601b9a3c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2d419640 1 -- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 msgr2=0x5601b9a438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2d419640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2d419640 1 -- 172.21.15.31:0/179898351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b9a3d050 con 0x5601b964e0a0 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2dc1a640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9676f60 0x5601b9a3c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea2d419640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 0x5601b9a432d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fea24003bd0 tx=0x7fea240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178148607368a728 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea10ff9640 1 -- 172.21.15.31:0/179898351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24028610 con 0x5601b964e0a0 2024-09-21T02:52:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea10ff9640 1 -- 172.21.15.31:0/179898351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fea240287b0 con 0x5601b964e0a0 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea20004cc0 con 0x5601b964e0a0 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea10ff9640 1 -- 172.21.15.31:0/179898351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea240289b0 con 0x5601b964e0a0 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea35bf1740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5601b9a42aa0 con 0x5601b964e0a0 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.334+0000 7fea10ff9640 1 -- 172.21.15.31:0/179898351 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24028b50 con 0x5601b964e0a0 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea10ff9640 1 -- 172.21.15.31:0/179898351 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fea24035070 con 0x5601b964e0a0 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea35bf1740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea35bf1740 10 client.15465 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea35bf1740 20 client.15465 populate_metadata read hostname 'smithi031' 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea35bf1740 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b960ac90 0x5601b9aa9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea35bf1740 4 client.15465 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:52:52.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "status"}) -- 0x7fea20004cc0 con 0x5601b960ac90 2024-09-21T02:52:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.335+0000 7fea2cc18640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b960ac90 0x5601b9aa9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea10ff9640 10 client.15465 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea2cc18640 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b960ac90 0x5601b9aa9630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9090ea49f1c9f739 server_cookie=d8509cc0c787812a in_seq=0 out_seq=0 2024-09-21T02:52:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea10ff9640 1 -- 172.21.15.31:0/179898351 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7fea20004cc0 con 0x5601b960ac90 2024-09-21T02:52:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea10ff9640 10 client.15465 handle_command_reply: tid=0 2024-09-21T02:52:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 client.15465 shutdown 2024-09-21T02:52:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 msgr2=0x5601b9a432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 0x5601b9a432d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fea24003bd0 tx=0x7fea240046c0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 shutdown_connections 2024-09-21T02:52:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5601b960ac90 0x5601b9aa9630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5601b9676f60 0x5601b9a3c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601b9656280 0x5601b9a438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 --2- 172.21.15.31:0/179898351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5601b964e0a0 0x5601b9a432d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea10ff9640 0 client.15465 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 >> 172.21.15.31:0/179898351 conn(0x5601b966b350 msgr2=0x5601b9a1a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 shutdown_connections 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 1 -- 172.21.15.31:0/179898351 wait complete. 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.336+0000 7fea35bf1740 20 client.15465 trim_cache size 0 max 16384 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "fdd37b14-77c2-11ef-baf5-efdc52797490", 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-21T02:52:52.343 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24695, 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 120.98079944200001, 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 26, 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 78, 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 76, 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 132.246889482 2024-09-21T02:52:52.344 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fea140b6560 msgr2=0x7fea140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fea140b6560 0x7fea140b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5601b9a1bb80 tx=0x7fea2400b390 comp rx=0 tx=0).stop 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 msgr2=0x7fea2819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2819cbf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea18005940 tx=0x7fea180042f0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 shutdown_connections 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fea140b6560 0x7fea140b8a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea28110200 0x7fea281a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea28102fa0 0x7fea2819cbf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 --2- 172.21.15.31:0/1659575170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea28102650 0x7fea2819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 >> 172.21.15.31:0/1659575170 conn(0x7fea28079760 msgr2=0x7fea28108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:52.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 shutdown_connections 2024-09-21T02:52:52.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:52.337+0000 7fea2f67d640 1 -- 172.21.15.31:0/1659575170 wait complete. 2024-09-21T02:52:53.028 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:52 smithi031 ceph-mon[31209]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:53.029 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T02:52:53.054 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T02:52:53.054 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T02:52:53.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:53.054 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-21T02:52:53.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:53.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-21T02:52:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-09-21T02:52:53.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:52:53.146 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-21T02:52:53.146 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:52:53.146 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:52:53.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T02:52:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:52 smithi165 ceph-mon[34818]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:52 smithi178 ceph-mon[34759]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:52:53.446 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 -- 172.21.15.31:0/1075790562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3408103b60 msgr2=0x7f3408103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 --2- 172.21.15.31:0/1075790562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3408103b60 0x7f3408103f70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3404009900 tx=0x7f340402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 -- 172.21.15.31:0/1075790562 shutdown_connections 2024-09-21T02:52:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 --2- 172.21.15.31:0/1075790562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3408109df0 0x7f3408110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 --2- 172.21.15.31:0/1075790562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34081094a0 0x7f34081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 --2- 172.21.15.31:0/1075790562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3408103b60 0x7f3408103f70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 -- 172.21.15.31:0/1075790562 >> 172.21.15.31:0/1075790562 conn(0x7f340807b850 msgr2=0x7f340807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 -- 172.21.15.31:0/1075790562 shutdown_connections 2024-09-21T02:52:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 -- 172.21.15.31:0/1075790562 wait complete. 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.959+0000 7f3410029640 1 Processor -- start 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 -- start start 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3408103b60 0x7f3408075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 0x7f3408075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408109df0 0x7f3408076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3408113af0 con 0x7f34081094a0 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3408113970 con 0x7f3408109df0 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f3410029640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3408113c70 con 0x7f3408103b60 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 0x7f3408075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 0x7f3408075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45414/0 (socket says 172.21.15.31:45414) 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 -- 172.21.15.31:0/2475024811 learned_addr learned my addr 172.21.15.31:0/2475024811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:53.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340ddc5640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3408103b60 0x7f3408075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340e5c6640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408109df0 0x7f3408076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 -- 172.21.15.31:0/2475024811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3408103b60 msgr2=0x7f3408075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3408103b60 0x7f3408075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 -- 172.21.15.31:0/2475024811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408109df0 msgr2=0x7f3408076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408109df0 0x7f3408076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340d5c4640 1 -- 172.21.15.31:0/2475024811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34081a8710 con 0x7f34081094a0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.960+0000 7f340e5c6640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408109df0 0x7f3408076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.961+0000 7f340d5c4640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 0x7f3408075c50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006fd0 tx=0x7f33f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.961+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8011830 con 0x7f34081094a0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.961+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34081a8a00 con 0x7f34081094a0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.961+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f8016070 con 0x7f34081094a0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.961+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8010e00 con 0x7f34081094a0 2024-09-21T02:52:53.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.961+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34081a8fc0 con 0x7f34081094a0 2024-09-21T02:52:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.962+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3408103f70 con 0x7f34081094a0 2024-09-21T02:52:53.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.964+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f33f801b460 con 0x7f34081094a0 2024-09-21T02:52:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.965+0000 7f33feffd640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f33f40b6560 0x7f33f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:53.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.965+0000 7f340ddc5640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f33f40b6560 0x7f33f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:53.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.965+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f33f80e7150 con 0x7f34081094a0 2024-09-21T02:52:53.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.966+0000 7f340ddc5640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f33f40b6560 0x7f33f40b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3404000c00 tx=0x7f3404031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:53.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:53.969+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33f80ad880 con 0x7f34081094a0 2024-09-21T02:52:54.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.196+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3408076a10 con 0x7f34081094a0 2024-09-21T02:52:54.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.196+0000 7f33feffd640 1 -- 172.21.15.31:0/2475024811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f33f80b1c60 con 0x7f34081094a0 2024-09-21T02:52:54.198 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f33f40b6560 msgr2=0x7f33f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f33f40b6560 0x7f33f40b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3404000c00 tx=0x7f3404031400 comp rx=0 tx=0).stop 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 msgr2=0x7f3408075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 0x7f3408075c50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006fd0 tx=0x7f33f800b8e0 comp rx=0 tx=0).stop 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 shutdown_connections 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f33f40b6560 0x7f33f40b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3408109df0 0x7f3408076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:54.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34081094a0 0x7f3408075c50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:54.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 --2- 172.21.15.31:0/2475024811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3408103b60 0x7f3408075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:54.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.198+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 >> 172.21.15.31:0/2475024811 conn(0x7f340807b850 msgr2=0x7f3408107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:54.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.199+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 shutdown_connections 2024-09-21T02:52:54.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:54.199+0000 7f3410029640 1 -- 172.21.15.31:0/2475024811 wait complete. 2024-09-21T02:52:54.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:54 smithi031 ceph-mon[31209]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:54.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:54 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2475024811' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T02:52:55.035 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:52:55.036 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:52:55.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:55.036 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-09-21T02:52:55.055 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:52:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:52:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:52:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:52:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-09-21T02:52:55.055 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42143sec preferred_lft 42143sec 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-21T02:52:55.056 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:52:55.057 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T02:52:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-09-21T02:52:55.057 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-09-21T02:52:55.057 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:52:55.057 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:52:55.057 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T02:52:55.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:52:55.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-09-21T02:52:55.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-09-21T02:52:55.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:52:55.058 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T02:52:55.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:52:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:54 smithi165 ceph-mon[34818]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:54 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2475024811' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T02:52:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:54 smithi178 ceph-mon[34759]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:52:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:54 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2475024811' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T02:52:55.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:55.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:55.276 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:52:55.302 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-21T02:52:55.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:55.304 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-09-21T02:52:55.370 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-09-21T02:52:55.370 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:52:55.370 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T02:52:55.370 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T02:52:55.370 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:52:55.370 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:52:55.371 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T02:52:55.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T02:52:55.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T02:52:55.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T02:52:55.371 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T02:52:55.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:52:55.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:55.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:55.713 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-21T02:52:55.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:55.729 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-09-21T02:52:55.786 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:52:55.786 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T02:52:55.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:52:55.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T02:52:55.786 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T02:52:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:52:55.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:55.906 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T02:52:55.906 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T02:52:55.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:52:56.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:56.034 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:52:56.034 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T02:52:56.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-09-21T02:52:56.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:52:56.034 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T02:52:56.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:52:56.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:56.114 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:52:56.114 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:52:56.130 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:52:56.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:52:56.130 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:52:56.186 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T02:52:56.186 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T02:52:56.186 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/75fbed08-2606-481b-8769-d706cef27b6b 2024-09-21T02:52:56.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:56.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/75fbed08-2606-481b-8769-d706cef27b6b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-21T02:52:56.328 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-21T02:52:56.328 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-21T02:52:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-21T02:52:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-21T02:52:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.165:3300,172.21.15.178:3300" 2024-09-21T02:52:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@fdd37b14-77c2-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/75fbed08-2606-481b-8769-d706cef27b6b 2024-09-21T02:52:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.31:3300/172.21.15.165:3300/172.21.15.178:3300" will pass to kernel 2024-09-21T02:52:56.482 INFO:teuthology.orchestra.run.smithi031.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T02:52:56.482 INFO:teuthology.orchestra.run.smithi031.stdout: You just mounted a file system that supports labels which does not 2024-09-21T02:52:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T02:52:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T02:52:56.483 INFO:teuthology.orchestra.run.smithi031.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T02:52:56.484 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T02:52:56.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:52:56.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:52:56.512 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:52:56.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:52:56.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:56.610 INFO:teuthology.orchestra.run.smithi031.stdout:client15474 2024-09-21T02:52:56.611 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:52:56.611 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:52:56.628 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:52:56.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:52:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:56.727 INFO:teuthology.orchestra.run.smithi031.stdout:client15474 2024-09-21T02:52:56.727 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T02:52:56.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:52:56.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:56.786 INFO:teuthology.orchestra.run.smithi031.stdout:client15474 2024-09-21T02:52:56.786 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490.client15474/status') 2024-09-21T02:52:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:52:56.846 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.15474 (3)192.168.0.1:0/1984050645 2024-09-21T02:52:56.846 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-09-21T02:52:56.846 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-09-21T02:52:56.847 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-09-21T02:52:56.847 INFO:teuthology.orchestra.run.smithi031.stderr:70 bytes copied, 2.5333e-05 s, 2.8 MB/s 2024-09-21T02:52:56.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:52:56.853 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T02:52:56.863 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T02:52:56.863 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T02:52:56.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T02:52:56.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 -- 172.21.15.31:0/1427200979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 msgr2=0x7fb82c102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:56.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 --2- 172.21.15.31:0/1427200979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c102970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb828009900 tx=0x7fb82802a480 comp rx=0 tx=0).stop 2024-09-21T02:52:56.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 -- 172.21.15.31:0/1427200979 shutdown_connections 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 --2- 172.21.15.31:0/1427200979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c110110 0x7fb82c112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 --2- 172.21.15.31:0/1427200979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb82c102eb0 0x7fb82c10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 --2- 172.21.15.31:0/1427200979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c102970 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 -- 172.21.15.31:0/1427200979 >> 172.21.15.31:0/1427200979 conn(0x7fb82c0fe420 msgr2=0x7fb82c100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 -- 172.21.15.31:0/1427200979 shutdown_connections 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.976+0000 7fb833a15640 1 -- 172.21.15.31:0/1427200979 wait complete. 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 Processor -- start 2024-09-21T02:52:56.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 -- start start 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c102eb0 0x7fb82c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb82c110110 0x7fb82c1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c115c00 con 0x7fb82c102560 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c115a80 con 0x7fb82c110110 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb833a15640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c115d80 con 0x7fb82c102eb0 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.977+0000 7fb8317b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:56.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c19c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45430/0 (socket says 172.21.15.31:45430) 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 -- 172.21.15.31:0/762425459 learned_addr learned my addr 172.21.15.31:0/762425459 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb830fb0640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c102eb0 0x7fb82c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb831fb2640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb82c110110 0x7fb82c1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 -- 172.21.15.31:0/762425459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c102eb0 msgr2=0x7fb82c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c102eb0 0x7fb82c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 -- 172.21.15.31:0/762425459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb82c110110 msgr2=0x7fb82c1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb82c110110 0x7fb82c1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 -- 172.21.15.31:0/762425459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c1a2480 con 0x7fb82c102560 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb830fb0640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c102eb0 0x7fb82c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8317b1640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c19c5e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb828005940 tx=0x7fb8280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82803f030 con 0x7fb82c102560 2024-09-21T02:52:56.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb828003cb0 con 0x7fb82c102560 2024-09-21T02:52:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c1a2710 con 0x7fb82c102560 2024-09-21T02:52:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828035070 con 0x7fb82c102560 2024-09-21T02:52:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.978+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1a2c00 con 0x7fb82c102560 2024-09-21T02:52:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.979+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82c102970 con 0x7fb82c102560 2024-09-21T02:52:56.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.981+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fb828042030 con 0x7fb82c102560 2024-09-21T02:52:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.982+0000 7fb8227fc640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8180b6560 0x7fb8180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.982+0000 7fb830fb0640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8180b6560 0x7fb8180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.982+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb828108190 con 0x7fb82c102560 2024-09-21T02:52:56.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.982+0000 7fb830fb0640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8180b6560 0x7fb8180b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb82c1113f0 tx=0x7fb81c00a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:56.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:56.985+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8280ce8c0 con 0x7fb82c102560 2024-09-21T02:52:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:56 smithi031 ceph-mon[31209]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:56 smithi165 ceph-mon[34818]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:57.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.213+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb82c02e640 con 0x7fb82c102560 2024-09-21T02:52:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:56 smithi178 ceph-mon[34759]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:52:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.787+0000 7fb8227fc640 1 -- 172.21.15.31:0/762425459 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fb8280d2ca0 con 0x7fb82c102560 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8180b6560 msgr2=0x7fb8180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8180b6560 0x7fb8180b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb82c1113f0 tx=0x7fb81c00a5c0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 msgr2=0x7fb82c19c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c19c5e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb828005940 tx=0x7fb8280042f0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 shutdown_connections 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8180b6560 0x7fb8180b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb82c102eb0 0x7fb82c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb82c110110 0x7fb82c1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 --2- 172.21.15.31:0/762425459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb82c102560 0x7fb82c19c5e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.790+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 >> 172.21.15.31:0/762425459 conn(0x7fb82c0fe420 msgr2=0x7fb82c112240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.791+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 shutdown_connections 2024-09-21T02:52:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.791+0000 7fb833a15640 1 -- 172.21.15.31:0/762425459 wait complete. 2024-09-21T02:52:57.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T02:52:57.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 -- 172.21.15.31:0/293756481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 msgr2=0x7fa3e8104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/293756481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8104d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009900 tx=0x7fa3d802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 -- 172.21.15.31:0/293756481 shutdown_connections 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/293756481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e8105280 0x7fa3e810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/293756481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3e810dd90 0x7fa3e8110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/293756481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8104d40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 -- 172.21.15.31:0/293756481 >> 172.21.15.31:0/293756481 conn(0x7fa3e80fe3c0 msgr2=0x7fa3e8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 -- 172.21.15.31:0/293756481 shutdown_connections 2024-09-21T02:52:57.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.926+0000 7fa3ee02e640 1 -- 172.21.15.31:0/293756481 wait complete. 2024-09-21T02:52:57.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 Processor -- start 2024-09-21T02:52:57.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 -- start start 2024-09-21T02:52:57.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:57.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3e8105280 0x7fa3e8102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:57.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e810dd90 0x7fa3e81030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:57.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8113a10 con 0x7fa3e8104930 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8113890 con 0x7fa3e8105280 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3ee02e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8113b90 con 0x7fa3e810dd90 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45456/0 (socket says 172.21.15.31:45456) 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3e77fe640 1 -- 172.21.15.31:0/3559855634 learned_addr learned my addr 172.21.15.31:0/3559855634 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3e7fff640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e810dd90 0x7fa3e81030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.927+0000 7fa3deffd640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3e8105280 0x7fa3e8102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e77fe640 1 -- 172.21.15.31:0/3559855634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e810dd90 msgr2=0x7fa3e81030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e77fe640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e810dd90 0x7fa3e81030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e77fe640 1 -- 172.21.15.31:0/3559855634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3e8105280 msgr2=0x7fa3e8102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e77fe640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3e8105280 0x7fa3e8102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e77fe640 1 -- 172.21.15.31:0/3559855634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e81a8700 con 0x7fa3e8104930 2024-09-21T02:52:57.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e7fff640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e810dd90 0x7fa3e81030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e77fe640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8102660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005940 tx=0x7fa3d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d803f030 con 0x7fa3e8104930 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d8003cb0 con 0x7fa3e8104930 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81a8990 con 0x7fa3e8104930 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8035070 con 0x7fa3e8104930 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.928+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81a8e80 con 0x7fa3e8104930 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.929+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e8104d40 con 0x7fa3e8104930 2024-09-21T02:52:57.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.931+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fa3d8042030 con 0x7fa3e8104930 2024-09-21T02:52:57.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.932+0000 7fa3e57fa640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3d40b6590 0x7fa3d40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:57.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.932+0000 7fa3deffd640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3d40b6590 0x7fa3d40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:57.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.932+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa3d8107ce0 con 0x7fa3e8104930 2024-09-21T02:52:57.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.932+0000 7fa3deffd640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3d40b6590 0x7fa3d40b8aa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005fd0 tx=0x7fa3d0005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:57.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:57.935+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3d80ce410 con 0x7fa3e8104930 2024-09-21T02:52:58.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:57 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/762425459' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T02:52:58.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.149+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa3e8108030 con 0x7fa3d40b6590 2024-09-21T02:52:58.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.172+0000 7fa3e57fa640 1 -- 172.21.15.31:0/3559855634 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa3e8108030 con 0x7fa3d40b6590 2024-09-21T02:52:58.173 INFO:teuthology.orchestra.run.smithi031.stdout:{} 2024-09-21T02:52:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3d40b6590 msgr2=0x7fa3d40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3d40b6590 0x7fa3d40b8aa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005fd0 tx=0x7fa3d0005850 comp rx=0 tx=0).stop 2024-09-21T02:52:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 msgr2=0x7fa3e8102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8102660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005940 tx=0x7fa3d80042f0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 shutdown_connections 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa3d40b6590 0x7fa3d40b8aa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3e810dd90 0x7fa3e81030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3e8105280 0x7fa3e8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 --2- 172.21.15.31:0/3559855634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3e8104930 0x7fa3e8102660 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 >> 172.21.15.31:0/3559855634 conn(0x7fa3e80fe3c0 msgr2=0x7fa3e8107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 shutdown_connections 2024-09-21T02:52:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.174+0000 7fa3ee02e640 1 -- 172.21.15.31:0/3559855634 wait complete. 2024-09-21T02:52:58.187 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T02:52:58.198 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T02:52:58.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:52:58.210 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:57 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/762425459' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T02:52:58.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:57 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/762425459' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.321+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2229463377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1029b0 msgr2=0x7f6cbc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.321+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2229463377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1029b0 0x7f6cbc102dc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009900 tx=0x7f6cb802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2229463377 shutdown_connections 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2229463377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cbc108c30 0x7f6cbc10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2229463377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2229463377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1029b0 0x7f6cbc102dc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2229463377 >> 172.21.15.31:0/2229463377 conn(0x7f6cbc07a670 msgr2=0x7f6cbc07aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2229463377 shutdown_connections 2024-09-21T02:52:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2229463377 wait complete. 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 Processor -- start 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.322+0000 7f6cc45ad640 1 -- start start 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cbc1029b0 0x7f6cbc19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cbc108c30 0x7f6cbc19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc1128f0 con 0x7f6cbc1082e0 2024-09-21T02:52:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc112770 con 0x7f6cbc1029b0 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc112a70 con 0x7f6cbc108c30 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc19b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45496/0 (socket says 172.21.15.31:45496) 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 -- 172.21.15.31:0/2599230583 learned_addr learned my addr 172.21.15.31:0/2599230583 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc2349640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cbc1029b0 0x7f6cbc19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc2b4a640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cbc108c30 0x7f6cbc19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 -- 172.21.15.31:0/2599230583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cbc108c30 msgr2=0x7f6cbc19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cbc108c30 0x7f6cbc19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 -- 172.21.15.31:0/2599230583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cbc1029b0 msgr2=0x7f6cbc19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cbc1029b0 0x7f6cbc19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 -- 172.21.15.31:0/2599230583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc1a64d0 con 0x7f6cbc1082e0 2024-09-21T02:52:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc1b48640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc19b700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00d740 tx=0x7f6cac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac017070 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cac00de50 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc1a67c0 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00f620 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.323+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc1a6d10 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.324+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc102dc0 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.326+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f6cac01a070 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.327+0000 7f6cb37fe640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6ca80b6560 0x7f6ca80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.327+0000 7f6cc2349640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6ca80b6560 0x7f6ca80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.327+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6cac0e6e60 con 0x7f6cbc1082e0 2024-09-21T02:52:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.327+0000 7f6cc2349640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6ca80b6560 0x7f6ca80b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8000c00 tx=0x7f6cb8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:58.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.330+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6cac0ad5f0 con 0x7f6cbc1082e0 2024-09-21T02:52:58.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.525+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cbc02e640 con 0x7f6cbc1082e0 2024-09-21T02:52:58.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.527+0000 7f6cb37fe640 1 -- 172.21.15.31:0/2599230583 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+23145 (secure 0 0 0) 0x7f6cac0b19d0 con 0x7f6cbc1082e0 2024-09-21T02:52:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:52:58.529 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:51:16.567886+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6ca80b6560 msgr2=0x7f6ca80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6ca80b6560 0x7f6ca80b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8000c00 tx=0x7f6cb8031400 comp rx=0 tx=0).stop 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 msgr2=0x7f6cbc19b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc19b700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00d740 tx=0x7f6cac00dc00 comp rx=0 tx=0).stop 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 shutdown_connections 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6ca80b6560 0x7f6ca80b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cbc108c30 0x7f6cbc19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cbc1082e0 0x7f6cbc19b700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 --2- 172.21.15.31:0/2599230583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cbc1029b0 0x7f6cbc19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 >> 172.21.15.31:0/2599230583 conn(0x7f6cbc07a670 msgr2=0x7f6cbc106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 shutdown_connections 2024-09-21T02:52:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.529+0000 7f6cc45ad640 1 -- 172.21.15.31:0/2599230583 wait complete. 2024-09-21T02:52:58.541 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:52:58.154+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:52:58.541 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:52:58.154+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:52:58.541 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:52:58.154+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:52:58.541 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:52:58.154+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:52:58.541 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:52:58.154+0000 7f08e9179640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:52:58.542 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:46:13.535825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:50:47.175448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:50:48.994282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:52:58.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:52:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 -- 172.21.15.31:0/3367333228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 msgr2=0x7fe76810f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 --2- 172.21.15.31:0/3367333228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76810f270 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe760009ca0 tx=0x7fe76002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 -- 172.21.15.31:0/3367333228 shutdown_connections 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 --2- 172.21.15.31:0/3367333228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7681080f0 0x7fe768108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 --2- 172.21.15.31:0/3367333228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76810f270 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 --2- 172.21.15.31:0/3367333228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7681027c0 0x7fe768102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 -- 172.21.15.31:0/3367333228 >> 172.21.15.31:0/3367333228 conn(0x7fe7680fe1e0 msgr2=0x7fe768100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 -- 172.21.15.31:0/3367333228 shutdown_connections 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.663+0000 7fe76759e640 1 -- 172.21.15.31:0/3367333228 wait complete. 2024-09-21T02:52:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.664+0000 7fe76759e640 1 Processor -- start 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.664+0000 7fe76759e640 1 -- start start 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.664+0000 7fe76759e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7681027c0 0x7fe76819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76759e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7681080f0 0x7fe76819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76819f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681126f0 con 0x7fe768108a40 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76759e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe768112570 con 0x7fe7681080f0 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76759e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe768112870 con 0x7fe7681027c0 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76819f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76819f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45504/0 (socket says 172.21.15.31:45504) 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 -- 172.21.15.31:0/3036896893 learned_addr learned my addr 172.21.15.31:0/3036896893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76659c640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7681027c0 0x7fe76819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe765d9b640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7681080f0 0x7fe76819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 -- 172.21.15.31:0/3036896893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7681027c0 msgr2=0x7fe76819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7681027c0 0x7fe76819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 -- 172.21.15.31:0/3036896893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7681080f0 msgr2=0x7fe76819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7681080f0 0x7fe76819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 -- 172.21.15.31:0/3036896893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7681a00d0 con 0x7fe768108a40 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe766d9d640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76819f9d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7600098d0 tx=0x7fe7600317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760003b70 con 0x7fe768108a40 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.665+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7681a0360 con 0x7fe768108a40 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.666+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe760003d10 con 0x7fe768108a40 2024-09-21T02:52:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.666+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76002fa20 con 0x7fe768108a40 2024-09-21T02:52:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.666+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a6b50 con 0x7fe768108a40 2024-09-21T02:52:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.667+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe768102bd0 con 0x7fe768108a40 2024-09-21T02:52:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.668+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fe76002fc40 con 0x7fe768108a40 2024-09-21T02:52:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.669+0000 7fe7577fe640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe7500b6560 0x7fe7500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.669+0000 7fe76659c640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe7500b6560 0x7fe7500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.669+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe7601086a0 con 0x7fe768108a40 2024-09-21T02:52:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.669+0000 7fe76659c640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe7500b6560 0x7fe7500b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe758009bd0 tx=0x7fe758005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.673+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7600cee30 con 0x7fe768108a40 2024-09-21T02:52:58.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-mon[31209]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T02:52:58.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/762425459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T02:52:58.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-mon[31209]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:52:58.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2599230583' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.865+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe7681a6df0 con 0x7fe768108a40 2024-09-21T02:52:58.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.865+0000 7fe7577fe640 1 -- 172.21.15.31:0/3036896893 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fe7600d3210 con 0x7fe768108a40 2024-09-21T02:52:58.867 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-21T02:52:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe7500b6560 msgr2=0x7fe7500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe7500b6560 0x7fe7500b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe758009bd0 tx=0x7fe758005f90 comp rx=0 tx=0).stop 2024-09-21T02:52:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 msgr2=0x7fe76819f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76819f9d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7600098d0 tx=0x7fe7600317b0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 shutdown_connections 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe7500b6560 0x7fe7500b8a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe768108a40 0x7fe76819f9d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7681080f0 0x7fe76819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 --2- 172.21.15.31:0/3036896893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7681027c0 0x7fe76819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 >> 172.21.15.31:0/3036896893 conn(0x7fe7680fe1e0 msgr2=0x7fe76806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 shutdown_connections 2024-09-21T02:52:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.868+0000 7fe76759e640 1 -- 172.21.15.31:0/3036896893 wait complete. 2024-09-21T02:52:58.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:52:59.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 -- 172.21.15.31:0/3622520343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 msgr2=0x7f31081086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 --2- 172.21.15.31:0/3622520343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081086e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009900 tx=0x7f30fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 -- 172.21.15.31:0/3622520343 shutdown_connections 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 --2- 172.21.15.31:0/3622520343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3108108c20 0x7f310810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 --2- 172.21.15.31:0/3622520343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081086e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 --2- 172.21.15.31:0/3622520343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31081029a0 0x7f3108102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 -- 172.21.15.31:0/3622520343 >> 172.21.15.31:0/3622520343 conn(0x7f31080fe3c0 msgr2=0x7f3108100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 -- 172.21.15.31:0/3622520343 shutdown_connections 2024-09-21T02:52:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.998+0000 7f310eed5640 1 -- 172.21.15.31:0/3622520343 wait complete. 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 Processor -- start 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 -- start start 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31081029a0 0x7f3108199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3108108c20 0x7f310819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f31081128e0 con 0x7f31081082d0 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3108112760 con 0x7f3108108c20 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310eed5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3108112a60 con 0x7f31081029a0 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f3107fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f3107fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45526/0 (socket says 172.21.15.31:45526) 2024-09-21T02:52:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f3107fff640 1 -- 172.21.15.31:0/3524750867 learned_addr learned my addr 172.21.15.31:0/3524750867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:58.999+0000 7f310cc71640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31081029a0 0x7f3108199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f310d472640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3108108c20 0x7f310819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3107fff640 1 -- 172.21.15.31:0/3524750867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31081029a0 msgr2=0x7f3108199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3107fff640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31081029a0 0x7f3108199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3107fff640 1 -- 172.21.15.31:0/3524750867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3108108c20 msgr2=0x7f310819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3107fff640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3108108c20 0x7f310819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3107fff640 1 -- 172.21.15.31:0/3524750867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310819f250 con 0x7f31081082d0 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f310d472640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3108108c20 0x7f310819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3107fff640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081998c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f30fc02a930 tx=0x7f30fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc03f030 con 0x7f31081082d0 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc003aa0 con 0x7f31081082d0 2024-09-21T02:52:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc035070 con 0x7f31081082d0 2024-09-21T02:52:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310819f4e0 con 0x7f31081082d0 2024-09-21T02:52:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.000+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31081a5cd0 con 0x7f31081082d0 2024-09-21T02:52:59.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.001+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3108102db0 con 0x7f31081082d0 2024-09-21T02:52:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.003+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f30fc042030 con 0x7f31081082d0 2024-09-21T02:52:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.004+0000 7f3105ffb640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f00b65e0 0x7f30f00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.004+0000 7f310cc71640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f00b65e0 0x7f30f00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.004+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f30fc107f30 con 0x7f31081082d0 2024-09-21T02:52:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.005+0000 7f310cc71640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f00b65e0 0x7f30f00b8af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005d00 tx=0x7f30f800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.008+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30fc0ce6c0 con 0x7f31081082d0 2024-09-21T02:52:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:58 smithi165 ceph-mon[34818]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T02:52:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/762425459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T02:52:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:58 smithi165 ceph-mon[34818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:52:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2599230583' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.195+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f310810aa70 con 0x7f31081082d0 2024-09-21T02:52:59.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.195+0000 7f3105ffb640 1 -- 172.21.15.31:0/3524750867 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f30fc0d5e20 con 0x7f31081082d0 2024-09-21T02:52:59.196 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-21T02:52:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f00b65e0 msgr2=0x7f30f00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f00b65e0 0x7f30f00b8af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005d00 tx=0x7f30f800b410 comp rx=0 tx=0).stop 2024-09-21T02:52:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 msgr2=0x7f31081998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081998c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f30fc02a930 tx=0x7f30fc004270 comp rx=0 tx=0).stop 2024-09-21T02:52:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 shutdown_connections 2024-09-21T02:52:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f00b65e0 0x7f30f00b8af0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3108108c20 0x7f310819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31081082d0 0x7f31081998c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 --2- 172.21.15.31:0/3524750867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31081029a0 0x7f3108199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 >> 172.21.15.31:0/3524750867 conn(0x7f31080fe3c0 msgr2=0x7f31081007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 shutdown_connections 2024-09-21T02:52:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.198+0000 7f310eed5640 1 -- 172.21.15.31:0/3524750867 wait complete. 2024-09-21T02:52:59.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:52:59.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:58 smithi178 ceph-mon[34759]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T02:52:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/762425459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T02:52:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:58 smithi178 ceph-mon[34759]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T02:52:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2599230583' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:52:59.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 -- 172.21.15.31:0/333645418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb070108c40 msgr2=0x7fb07010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 --2- 172.21.15.31:0/333645418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb070108c40 0x7fb07010f470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb068009ca0 tx=0x7fb06802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 -- 172.21.15.31:0/333645418 shutdown_connections 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 --2- 172.21.15.31:0/333645418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb070108c40 0x7fb07010f470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 --2- 172.21.15.31:0/333645418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0701082f0 0x7fb070108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 --2- 172.21.15.31:0/333645418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0701029c0 0x7fb070102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 -- 172.21.15.31:0/333645418 >> 172.21.15.31:0/333645418 conn(0x7fb0700fe400 msgr2=0x7fb070100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 -- 172.21.15.31:0/333645418 shutdown_connections 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.326+0000 7fb07709b640 1 -- 172.21.15.31:0/333645418 wait complete. 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 Processor -- start 2024-09-21T02:52:59.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 -- start start 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0701029c0 0x7fb07019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 0x7fb07019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb070108c40 0x7fb07019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb070112900 con 0x7fb0701082f0 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb070112780 con 0x7fb0701029c0 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.327+0000 7fb07709b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb070112a80 con 0x7fb070108c40 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 0x7fb07019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 0x7fb07019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43238/0 (socket says 172.21.15.31:43238) 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 -- 172.21.15.31:0/1693849060 learned_addr learned my addr 172.21.15.31:0/1693849060 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb074e37640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0701029c0 0x7fb07019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb075638640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb070108c40 0x7fb07019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 -- 172.21.15.31:0/1693849060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb070108c40 msgr2=0x7fb07019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb070108c40 0x7fb07019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 -- 172.21.15.31:0/1693849060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0701029c0 msgr2=0x7fb07019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0701029c0 0x7fb07019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 -- 172.21.15.31:0/1693849060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0701a0300 con 0x7fb0701082f0 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb075638640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb070108c40 0x7fb07019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06ffff640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 0x7fb07019a970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d740 tx=0x7fb05800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058017070 con 0x7fb0701082f0 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05800de50 con 0x7fb0701082f0 2024-09-21T02:52:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0701a05f0 con 0x7fb0701082f0 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.328+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800f620 con 0x7fb0701082f0 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.329+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0701a6d80 con 0x7fb0701082f0 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.329+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb070103c30 con 0x7fb0701082f0 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.331+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fb05801a070 con 0x7fb0701082f0 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.332+0000 7fb06dffb640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb05c0b65e0 0x7fb05c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.332+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb0580e6e00 con 0x7fb0701082f0 2024-09-21T02:52:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.332+0000 7fb074e37640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb05c0b65e0 0x7fb05c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:59.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.333+0000 7fb074e37640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb05c0b65e0 0x7fb05c0b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0600058c0 tx=0x7fb06000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:59.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.335+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0580ad610 con 0x7fb0701082f0 2024-09-21T02:52:59.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.523+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb07019b4f0 con 0x7fb0701082f0 2024-09-21T02:52:59.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.524+0000 7fb06dffb640 1 -- 172.21.15.31:0/1693849060 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7fb07019b4f0 con 0x7fb0701082f0 2024-09-21T02:52:59.525 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-21T02:52:59.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb05c0b65e0 msgr2=0x7fb05c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb05c0b65e0 0x7fb05c0b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0600058c0 tx=0x7fb06000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 msgr2=0x7fb07019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 0x7fb07019a970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d740 tx=0x7fb05800dc00 comp rx=0 tx=0).stop 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 shutdown_connections 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb05c0b65e0 0x7fb05c0b8af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb070108c40 0x7fb07019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0701082f0 0x7fb07019a970 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 --2- 172.21.15.31:0/1693849060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0701029c0 0x7fb07019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 >> 172.21.15.31:0/1693849060 conn(0x7fb0700fe400 msgr2=0x7fb07006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 shutdown_connections 2024-09-21T02:52:59.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:52:59.526+0000 7fb07709b640 1 -- 172.21.15.31:0/1693849060 wait complete. 2024-09-21T02:52:59.539 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T02:52:59.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:52:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:00.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:59 smithi031 ceph-mon[31209]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:00.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:59 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3036896893' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:59 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3524750867' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:52:59 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1693849060' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:59 smithi165 ceph-mon[34818]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3036896893' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3524750867' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:52:59 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1693849060' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:59 smithi178 ceph-mon[34759]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:59 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3036896893' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:59 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3524750867' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:52:59 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1693849060' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T02:53:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 -- 172.21.15.31:0/1381663589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901048c0 msgr2=0x7f0e90104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/1381663589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901048c0 0x7f0e90104cd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009900 tx=0x7f0e8002a410 comp rx=0 tx=0).stop 2024-09-21T02:53:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 -- 172.21.15.31:0/1381663589 shutdown_connections 2024-09-21T02:53:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/1381663589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e9010de70 0x7f0e90110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/1381663589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e901052a0 0x7f0e9010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/1381663589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901048c0 0x7f0e90104cd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 -- 172.21.15.31:0/1381663589 >> 172.21.15.31:0/1381663589 conn(0x7f0e90079760 msgr2=0x7f0e90079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 -- 172.21.15.31:0/1381663589 shutdown_connections 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.290+0000 7f0e94ab2640 1 -- 172.21.15.31:0/1381663589 wait complete. 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 Processor -- start 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 -- start start 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e901048c0 0x7f0e901027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 0x7f0e90102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e9010de70 0x7f0e90103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113ab0 con 0x7f0e901052a0 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113930 con 0x7f0e9010de70 2024-09-21T02:53:00.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e94ab2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113c30 con 0x7f0e901048c0 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e8dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 0x7f0e90102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e8dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 0x7f0e90102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43248/0 (socket says 172.21.15.31:43248) 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.291+0000 7f0e8dd9b640 1 -- 172.21.15.31:0/2183648192 learned_addr learned my addr 172.21.15.31:0/2183648192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8e59c640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e901048c0 0x7f0e901027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8ed9d640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e9010de70 0x7f0e90103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8dd9b640 1 -- 172.21.15.31:0/2183648192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e901048c0 msgr2=0x7f0e901027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8dd9b640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e901048c0 0x7f0e901027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8dd9b640 1 -- 172.21.15.31:0/2183648192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e9010de70 msgr2=0x7f0e90103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8dd9b640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e9010de70 0x7f0e90103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8dd9b640 1 -- 172.21.15.31:0/2183648192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e901a86f0 con 0x7f0e901052a0 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8ed9d640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e9010de70 0x7f0e90103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e8dd9b640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 0x7f0e90102d10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400c8c0 tx=0x7f0e8400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84007bb0 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e84007d50 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e901a89e0 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e840187c0 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.292+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a8f20 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.293+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9006c190 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.295+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f0e8400fa20 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.295+0000 7f0e7f7fe640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e780b65f0 0x7f0e780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.296+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0e840e7700 con 0x7f0e901052a0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.296+0000 7f0e8e59c640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e780b65f0 0x7f0e780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.296+0000 7f0e8e59c640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e780b65f0 0x7f0e780b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009230 tx=0x7f0e80031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.300+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e84014030 con 0x7f0e901052a0 2024-09-21T02:53:00.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.526+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e90103ad0 con 0x7f0e901052a0 2024-09-21T02:53:00.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.526+0000 7f0e7f7fe640 1 -- 172.21.15.31:0/2183648192 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f0e90103ad0 con 0x7f0e901052a0 2024-09-21T02:53:00.528 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:53:00.528 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:53:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e780b65f0 msgr2=0x7f0e780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e780b65f0 0x7f0e780b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009230 tx=0x7f0e80031400 comp rx=0 tx=0).stop 2024-09-21T02:53:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 msgr2=0x7f0e90102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 0x7f0e90102d10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400c8c0 tx=0x7f0e8400cd80 comp rx=0 tx=0).stop 2024-09-21T02:53:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 shutdown_connections 2024-09-21T02:53:00.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0e780b65f0 0x7f0e780b8b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e9010de70 0x7f0e90103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901052a0 0x7f0e90102d10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 --2- 172.21.15.31:0/2183648192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e901048c0 0x7f0e901027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 >> 172.21.15.31:0/2183648192 conn(0x7f0e90079760 msgr2=0x7f0e900ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 shutdown_connections 2024-09-21T02:53:00.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:00.529+0000 7f0e94ab2640 1 -- 172.21.15.31:0/2183648192 wait complete. 2024-09-21T02:53:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:00 smithi031 ceph-mon[31209]: pgmap v328: 97 pgs: 97 active+clean; 648 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T02:53:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:00 smithi031 ceph-mon[31209]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T02:53:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:00 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2183648192' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:01.166 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T02:53:01.166 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T02:53:01.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:53:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:00 smithi165 ceph-mon[34818]: pgmap v328: 97 pgs: 97 active+clean; 648 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T02:53:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:00 smithi165 ceph-mon[34818]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T02:53:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:00 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2183648192' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:00 smithi178 ceph-mon[34759]: pgmap v328: 97 pgs: 97 active+clean; 648 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T02:53:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:00 smithi178 ceph-mon[34759]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T02:53:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:00 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2183648192' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 -- 172.21.15.31:0/3237620916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18104910 msgr2=0x7f1a18104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 --2- 172.21.15.31:0/3237620916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18104910 0x7f1a18104d20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009900 tx=0x7f1a1402a410 comp rx=0 tx=0).stop 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 -- 172.21.15.31:0/3237620916 shutdown_connections 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 --2- 172.21.15.31:0/3237620916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18105260 0x7f1a1810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 --2- 172.21.15.31:0/3237620916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 0x7f1a18110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 --2- 172.21.15.31:0/3237620916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18104910 0x7f1a18104d20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 -- 172.21.15.31:0/3237620916 >> 172.21.15.31:0/3237620916 conn(0x7f1a180fe3c0 msgr2=0x7f1a18100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 -- 172.21.15.31:0/3237620916 shutdown_connections 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 -- 172.21.15.31:0/3237620916 wait complete. 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.282+0000 7f1a1f407640 1 Processor -- start 2024-09-21T02:53:01.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 -- start start 2024-09-21T02:53:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18104910 0x7f1a1819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 0x7f1a1819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 0x7f1a1819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181139f0 con 0x7f1a18105260 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1a18113870 con 0x7f1a1810dd70 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1f407640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1a18113b70 con 0x7f1a18104910 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 0x7f1a1819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 0x7f1a1819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43272/0 (socket says 172.21.15.31:43272) 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 -- 172.21.15.31:0/740317449 learned_addr learned my addr 172.21.15.31:0/740317449 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1d1a3640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18104910 0x7f1a1819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1d9a4640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 0x7f1a1819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 -- 172.21.15.31:0/740317449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18104910 msgr2=0x7f1a1819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18104910 0x7f1a1819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 -- 172.21.15.31:0/740317449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 msgr2=0x7f1a1819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 0x7f1a1819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1c9a2640 1 -- 172.21.15.31:0/740317449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a181a0360 con 0x7f1a18105260 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.283+0000 7f1a1d9a4640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 0x7f1a1819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.284+0000 7f1a1c9a2640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 0x7f1a1819a9d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d750 tx=0x7f1a0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.284+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08017070 con 0x7f1a18105260 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.284+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a181a0650 con 0x7f1a18105260 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.284+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a080054d0 con 0x7f1a18105260 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.284+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08004c30 con 0x7f1a18105260 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.284+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181a6de0 con 0x7f1a18105260 2024-09-21T02:53:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.285+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a18104d20 con 0x7f1a18105260 2024-09-21T02:53:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.286+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f1a08004dd0 con 0x7f1a18105260 2024-09-21T02:53:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.287+0000 7f1a0e7fc640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a040b65f0 0x7f1a040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.288+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1a080e74d0 con 0x7f1a18105260 2024-09-21T02:53:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.288+0000 7f1a1d1a3640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a040b65f0 0x7f1a040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.288+0000 7f1a1d1a3640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a040b65f0 0x7f1a040b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a14031440 tx=0x7f1a14005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:01.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.290+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a080adb00 con 0x7f1a18105260 2024-09-21T02:53:01.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.477+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a18107f90 con 0x7f1a040b65f0 2024-09-21T02:53:01.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.482+0000 7f1a0e7fc640 1 -- 172.21.15.31:0/740317449 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259180 (secure 0 0 0) 0x7f1a18107f90 con 0x7f1a040b65f0 2024-09-21T02:53:01.486 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:01.487 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:53:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a040b65f0 msgr2=0x7f1a040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a040b65f0 0x7f1a040b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a14031440 tx=0x7f1a14005d50 comp rx=0 tx=0).stop 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 msgr2=0x7f1a1819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 0x7f1a1819a9d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d750 tx=0x7f1a0800dc10 comp rx=0 tx=0).stop 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 shutdown_connections 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1a040b65f0 0x7f1a040b8b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1810dd70 0x7f1a1819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a18105260 0x7f1a1819a9d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 --2- 172.21.15.31:0/740317449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18104910 0x7f1a1819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 >> 172.21.15.31:0/740317449 conn(0x7f1a180fe3c0 msgr2=0x7f1a18107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 shutdown_connections 2024-09-21T02:53:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.485+0000 7f1a1f407640 1 -- 172.21.15.31:0/740317449 wait complete. 2024-09-21T02:53:01.498 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-21T02:53:01.175607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663616,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1291,"num_read_kb":753,"num_write":384,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376452,"kb_used_data":10564,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565372,"statfs":{"total":1151940427776,"available":1151554940928,"internally_reserved":0,"allocated":10817536,"data_stored":7029248,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1398,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":58,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003453"},"pg_stats":[{"pgid":"3.3f","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567590+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:52:15.567590+0000","last_peered":"2024-09-21T02:52:15.567590+0000","last_clean":"2024-09-21T02:52:15.567590+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:52:15.567590+0000","last_undegraded":"2024-09-21T02:52:15.567590+0000","last_fullsized":"2024-09-21T02:52:15.567590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563476+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:52:15.563476+0000","last_peered":"2024-09-21T02:52:15.563476+0000","last_clean":"2024-09-21T02:52:15.563476+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:52:15.563476+0000","last_undegraded":"2024-09-21T02:52:15.563476+0000","last_fullsized":"2024-09-21T02:52:15.563476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569310+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:52:15.569310+0000","last_peered":"2024-09-21T02:52:15.569310+0000","last_clean":"2024-09-21T02:52:15.569310+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:52:15.569310+0000","last_undegraded":"2024-09-21T02:52:15.569310+0000","last_fullsized":"2024-09-21T02:52:15.569310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557143+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:52:15.557143+0000","last_peered":"2024-09-21T02:52:15.557143+0000","last_clean":"2024-09-21T02:52:15.557143+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:52:15.557143+0000","last_undegraded":"2024-09-21T02:52:15.557143+0000","last_fullsized":"2024-09-21T02:52:15.557143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.546827+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:52:15.546827+0000","last_peered":"2024-09-21T02:52:15.546827+0000","last_clean":"2024-09-21T02:52:15.546827+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:52:15.546827+0000","last_undegraded":"2024-09-21T02:52:15.546827+0000","last_fullsized":"2024-09-21T02:52:15.546827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.540853+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:52:15.540853+0000","last_peered":"2024-09-21T02:52:15.540853+0000","last_clean":"2024-09-21T02:52:15.540853+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:52:15.540853+0000","last_undegraded":"2024-09-21T02:52:15.540853+0000","last_fullsized":"2024-09-21T02:52:15.540853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569968+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:52:15.569968+0000","last_peered":"2024-09-21T02:52:15.569968+0000","last_clean":"2024-09-21T02:52:15.569968+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:52:15.569968+0000","last_undegraded":"2024-09-21T02:52:15.569968+0000","last_fullsized":"2024-09-21T02:52:15.569968+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558545+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:52:15.558545+0000","last_peered":"2024-09-21T02:52:15.558545+0000","last_clean":"2024-09-21T02:52:15.558545+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:52:15.558545+0000","last_undegraded":"2024-09-21T02:52:15.558545+0000","last_fullsized":"2024-09-21T02:52:15.558545+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567267+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:52:15.567267+0000","last_peered":"2024-09-21T02:52:15.567267+0000","last_clean":"2024-09-21T02:52:15.567267+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:52:15.567267+0000","last_undegraded":"2024-09-21T02:52:15.567267+0000","last_fullsized":"2024-09-21T02:52:15.567267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.561944+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:52:15.561944+0000","last_peered":"2024-09-21T02:52:15.561944+0000","last_clean":"2024-09-21T02:52:15.561944+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:52:15.561944+0000","last_undegraded":"2024-09-21T02:52:15.561944+0000","last_fullsized":"2024-09-21T02:52:15.561944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567208+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:52:15.567208+0000","last_peered":"2024-09-21T02:52:15.567208+0000","last_clean":"2024-09-21T02:52:15.567208+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:52:15.567208+0000","last_undegraded":"2024-09-21T02:52:15.567208+0000","last_fullsized":"2024-09-21T02:52:15.567208+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552898+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:52:15.552898+0000","last_peered":"2024-09-21T02:52:15.552898+0000","last_clean":"2024-09-21T02:52:15.552898+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:52:15.552898+0000","last_undegraded":"2024-09-21T02:52:15.552898+0000","last_fullsized":"2024-09-21T02:52:15.552898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.554532+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:52:15.554532+0000","last_peered":"2024-09-21T02:52:15.554532+0000","last_clean":"2024-09-21T02:52:15.554532+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:52:15.554532+0000","last_undegraded":"2024-09-21T02:52:15.554532+0000","last_fullsized":"2024-09-21T02:52:15.554532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563207+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:52:15.563207+0000","last_peered":"2024-09-21T02:52:15.563207+0000","last_clean":"2024-09-21T02:52:15.563207+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:52:15.563207+0000","last_undegraded":"2024-09-21T02:52:15.563207+0000","last_fullsized":"2024-09-21T02:52:15.563207+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563556+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:52:15.563556+0000","last_peered":"2024-09-21T02:52:15.563556+0000","last_clean":"2024-09-21T02:52:15.563556+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:52:15.563556+0000","last_undegraded":"2024-09-21T02:52:15.563556+0000","last_fullsized":"2024-09-21T02:52:15.563556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.568347+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:52:15.568347+0000","last_peered":"2024-09-21T02:52:15.568347+0000","last_clean":"2024-09-21T02:52:15.568347+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:52:15.568347+0000","last_undegraded":"2024-09-21T02:52:15.568347+0000","last_fullsized":"2024-09-21T02:52:15.568347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567564+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:52:15.567564+0000","last_peered":"2024-09-21T02:52:15.567564+0000","last_clean":"2024-09-21T02:52:15.567564+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:52:15.567564+0000","last_undegraded":"2024-09-21T02:52:15.567564+0000","last_fullsized":"2024-09-21T02:52:15.567564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557763+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:52:15.557763+0000","last_peered":"2024-09-21T02:52:15.557763+0000","last_clean":"2024-09-21T02:52:15.557763+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:52:15.557763+0000","last_undegraded":"2024-09-21T02:52:15.557763+0000","last_fullsized":"2024-09-21T02:52:15.557763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552086+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:52:15.552086+0000","last_peered":"2024-09-21T02:52:15.552086+0000","last_clean":"2024-09-21T02:52:15.552086+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:52:15.552086+0000","last_undegraded":"2024-09-21T02:52:15.552086+0000","last_fullsized":"2024-09-21T02:52:15.552086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558416+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:52:15.558416+0000","last_peered":"2024-09-21T02:52:15.558416+0000","last_clean":"2024-09-21T02:52:15.558416+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:52:15.558416+0000","last_undegraded":"2024-09-21T02:52:15.558416+0000","last_fullsized":"2024-09-21T02:52:15.558416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560852+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:52:15.560852+0000","last_peered":"2024-09-21T02:52:15.560852+0000","last_clean":"2024-09-21T02:52:15.560852+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:52:15.560852+0000","last_undegraded":"2024-09-21T02:52:15.560852+0000","last_fullsized":"2024-09-21T02:52:15.560852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.555555+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:52:15.555555+0000","last_peered":"2024-09-21T02:52:15.555555+0000","last_clean":"2024-09-21T02:52:15.555555+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:52:15.555555+0000","last_undegraded":"2024-09-21T02:52:15.555555+0000","last_fullsized":"2024-09-21T02:52:15.555555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557760+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:52:15.557760+0000","last_peered":"2024-09-21T02:52:15.557760+0000","last_clean":"2024-09-21T02:52:15.557760+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:52:15.557760+0000","last_undegraded":"2024-09-21T02:52:15.557760+0000","last_fullsized":"2024-09-21T02:52:15.557760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.548121+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:52:15.548121+0000","last_peered":"2024-09-21T02:52:15.548121+0000","last_clean":"2024-09-21T02:52:15.548121+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:52:15.548121+0000","last_undegraded":"2024-09-21T02:52:15.548121+0000","last_fullsized":"2024-09-21T02:52:15.548121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.549005+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:52:15.549005+0000","last_peered":"2024-09-21T02:52:15.549005+0000","last_clean":"2024-09-21T02:52:15.549005+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:52:15.549005+0000","last_undegraded":"2024-09-21T02:52:15.549005+0000","last_fullsized":"2024-09-21T02:52:15.549005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.565080+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:52:15.565080+0000","last_peered":"2024-09-21T02:52:15.565080+0000","last_clean":"2024-09-21T02:52:15.565080+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:52:15.565080+0000","last_undegraded":"2024-09-21T02:52:15.565080+0000","last_fullsized":"2024-09-21T02:52:15.565080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.554835+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:52:15.554835+0000","last_peered":"2024-09-21T02:52:15.554835+0000","last_clean":"2024-09-21T02:52:15.554835+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:52:15.554835+0000","last_undegraded":"2024-09-21T02:52:15.554835+0000","last_fullsized":"2024-09-21T02:52:15.554835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560546+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:52:15.560546+0000","last_peered":"2024-09-21T02:52:15.560546+0000","last_clean":"2024-09-21T02:52:15.560546+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:52:15.560546+0000","last_undegraded":"2024-09-21T02:52:15.560546+0000","last_fullsized":"2024-09-21T02:52:15.560546+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.571336+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:52:15.571336+0000","last_peered":"2024-09-21T02:52:15.571336+0000","last_clean":"2024-09-21T02:52:15.571336+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:52:15.571336+0000","last_undegraded":"2024-09-21T02:52:15.571336+0000","last_fullsized":"2024-09-21T02:52:15.571336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558911+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:52:15.558911+0000","last_peered":"2024-09-21T02:52:15.558911+0000","last_clean":"2024-09-21T02:52:15.558911+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:52:15.558911+0000","last_undegraded":"2024-09-21T02:52:15.558911+0000","last_fullsized":"2024-09-21T02:52:15.558911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.555496+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:52:15.555496+0000","last_peered":"2024-09-21T02:52:15.555496+0000","last_clean":"2024-09-21T02:52:15.555496+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:52:15.555496+0000","last_undegraded":"2024-09-21T02:52:15.555496+0000","last_fullsized":"2024-09-21T02:52:15.555496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557445+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:52:15.557445+0000","last_peered":"2024-09-21T02:52:15.557445+0000","last_clean":"2024-09-21T02:52:15.557445+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:52:15.557445+0000","last_undegraded":"2024-09-21T02:52:15.557445+0000","last_fullsized":"2024-09-21T02:52:15.557445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.544310+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:52:15.544310+0000","last_peered":"2024-09-21T02:52:15.544310+0000","last_clean":"2024-09-21T02:52:15.544310+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:52:15.544310+0000","last_undegraded":"2024-09-21T02:52:15.544310+0000","last_fullsized":"2024-09-21T02:52:15.544310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":165,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.733243+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:52:57.733243+0000","last_peered":"2024-09-21T02:52:57.733243+0000","last_clean":"2024-09-21T02:52:57.733243+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:52:57.733243+0000","last_undegraded":"2024-09-21T02:52:57.733243+0000","last_fullsized":"2024-09-21T02:52:57.733243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.551262+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:52:15.551262+0000","last_peered":"2024-09-21T02:52:15.551262+0000","last_clean":"2024-09-21T02:52:15.551262+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:52:15.551262+0000","last_undegraded":"2024-09-21T02:52:15.551262+0000","last_fullsized":"2024-09-21T02:52:15.551262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.556254+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:52:15.556254+0000","last_peered":"2024-09-21T02:52:15.556254+0000","last_clean":"2024-09-21T02:52:15.556254+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:52:15.556254+0000","last_undegraded":"2024-09-21T02:52:15.556254+0000","last_fullsized":"2024-09-21T02:52:15.556254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.551455+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:52:15.551455+0000","last_peered":"2024-09-21T02:52:15.551455+0000","last_clean":"2024-09-21T02:52:15.551455+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:52:15.551455+0000","last_undegraded":"2024-09-21T02:52:15.551455+0000","last_fullsized":"2024-09-21T02:52:15.551455+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'6","reported_seq":261,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:56.700068+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:52:56.700068+0000","last_peered":"2024-09-21T02:52:56.700068+0000","last_clean":"2024-09-21T02:52:56.700068+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:52:56.700068+0000","last_undegraded":"2024-09-21T02:52:56.700068+0000","last_fullsized":"2024-09-21T02:52:56.700068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563004+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:52:15.563004+0000","last_peered":"2024-09-21T02:52:15.563004+0000","last_clean":"2024-09-21T02:52:15.563004+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:52:15.563004+0000","last_undegraded":"2024-09-21T02:52:15.563004+0000","last_fullsized":"2024-09-21T02:52:15.563004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564496+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:52:15.564496+0000","last_peered":"2024-09-21T02:52:15.564496+0000","last_clean":"2024-09-21T02:52:15.564496+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:52:15.564496+0000","last_undegraded":"2024-09-21T02:52:15.564496+0000","last_fullsized":"2024-09-21T02:52:15.564496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.565141+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:52:15.565141+0000","last_peered":"2024-09-21T02:52:15.565141+0000","last_clean":"2024-09-21T02:52:15.565141+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:52:15.565141+0000","last_undegraded":"2024-09-21T02:52:15.565141+0000","last_fullsized":"2024-09-21T02:52:15.565141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'33","reported_seq":297,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:56.731591+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:52:56.731591+0000","last_peered":"2024-09-21T02:52:56.731591+0000","last_clean":"2024-09-21T02:52:56.731591+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:52:56.731591+0000","last_undegraded":"2024-09-21T02:52:56.731591+0000","last_fullsized":"2024-09-21T02:52:56.731591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":182,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.550696+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:52:15.550696+0000","last_peered":"2024-09-21T02:52:15.550696+0000","last_clean":"2024-09-21T02:52:15.550696+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:52:15.550696+0000","last_undegraded":"2024-09-21T02:52:15.550696+0000","last_fullsized":"2024-09-21T02:52:15.550696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552036+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:52:15.552036+0000","last_peered":"2024-09-21T02:52:15.552036+0000","last_clean":"2024-09-21T02:52:15.552036+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:52:15.552036+0000","last_undegraded":"2024-09-21T02:52:15.552036+0000","last_fullsized":"2024-09-21T02:52:15.552036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567369+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:52:15.567369+0000","last_peered":"2024-09-21T02:52:15.567369+0000","last_clean":"2024-09-21T02:52:15.567369+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:52:15.567369+0000","last_undegraded":"2024-09-21T02:52:15.567369+0000","last_fullsized":"2024-09-21T02:52:15.567369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.712619+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:52:57.712619+0000","last_peered":"2024-09-21T02:52:57.712619+0000","last_clean":"2024-09-21T02:52:57.712619+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:52:57.712619+0000","last_undegraded":"2024-09-21T02:52:57.712619+0000","last_fullsized":"2024-09-21T02:52:57.712619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569073+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:52:15.569073+0000","last_peered":"2024-09-21T02:52:15.569073+0000","last_clean":"2024-09-21T02:52:15.569073+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:52:15.569073+0000","last_undegraded":"2024-09-21T02:52:15.569073+0000","last_fullsized":"2024-09-21T02:52:15.569073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.566458+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:52:15.566458+0000","last_peered":"2024-09-21T02:52:15.566458+0000","last_clean":"2024-09-21T02:52:15.566458+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:52:15.566458+0000","last_undegraded":"2024-09-21T02:52:15.566458+0000","last_fullsized":"2024-09-21T02:52:15.566458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558056+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:52:15.558056+0000","last_peered":"2024-09-21T02:52:15.558056+0000","last_clean":"2024-09-21T02:52:15.558056+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:52:15.558056+0000","last_undegraded":"2024-09-21T02:52:15.558056+0000","last_fullsized":"2024-09-21T02:52:15.558056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.568405+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:52:15.568405+0000","last_peered":"2024-09-21T02:52:15.568405+0000","last_clean":"2024-09-21T02:52:15.568405+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:52:15.568405+0000","last_undegraded":"2024-09-21T02:52:15.568405+0000","last_fullsized":"2024-09-21T02:52:15.568405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570341+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:52:15.570341+0000","last_peered":"2024-09-21T02:52:15.570341+0000","last_clean":"2024-09-21T02:52:15.570341+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:52:15.570341+0000","last_undegraded":"2024-09-21T02:52:15.570341+0000","last_fullsized":"2024-09-21T02:52:15.570341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'8","reported_seq":152,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.702109+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:52:57.702109+0000","last_peered":"2024-09-21T02:52:57.702109+0000","last_clean":"2024-09-21T02:52:57.702109+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:52:57.702109+0000","last_undegraded":"2024-09-21T02:52:57.702109+0000","last_fullsized":"2024-09-21T02:52:57.702109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569660+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:52:15.569660+0000","last_peered":"2024-09-21T02:52:15.569660+0000","last_clean":"2024-09-21T02:52:15.569660+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:52:15.569660+0000","last_undegraded":"2024-09-21T02:52:15.569660+0000","last_fullsized":"2024-09-21T02:52:15.569660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576900+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576900+0000","last_peered":"2024-09-21T02:51:16.576900+0000","last_clean":"2024-09-21T02:51:16.576900+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576900+0000","last_undegraded":"2024-09-21T02:51:16.576900+0000","last_fullsized":"2024-09-21T02:51:16.576900+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573515+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573515+0000","last_peered":"2024-09-21T02:51:16.573515+0000","last_clean":"2024-09-21T02:51:16.573515+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573515+0000","last_undegraded":"2024-09-21T02:51:16.573515+0000","last_fullsized":"2024-09-21T02:51:16.573515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564758+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:52:15.564758+0000","last_peered":"2024-09-21T02:52:15.564758+0000","last_clean":"2024-09-21T02:52:15.564758+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:52:15.564758+0000","last_undegraded":"2024-09-21T02:52:15.564758+0000","last_fullsized":"2024-09-21T02:52:15.564758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576845+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576845+0000","last_peered":"2024-09-21T02:51:16.576845+0000","last_clean":"2024-09-21T02:51:16.576845+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576845+0000","last_undegraded":"2024-09-21T02:51:16.576845+0000","last_fullsized":"2024-09-21T02:51:16.576845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553787+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:52:15.553787+0000","last_peered":"2024-09-21T02:52:15.553787+0000","last_clean":"2024-09-21T02:52:15.553787+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:52:15.553787+0000","last_undegraded":"2024-09-21T02:52:15.553787+0000","last_fullsized":"2024-09-21T02:52:15.553787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'6","reported_seq":151,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:59.715214+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:52:59.715214+0000","last_peered":"2024-09-21T02:52:59.715214+0000","last_clean":"2024-09-21T02:52:59.715214+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:52:59.715214+0000","last_undegraded":"2024-09-21T02:52:59.715214+0000","last_fullsized":"2024-09-21T02:52:59.715214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.556144+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:52:15.556144+0000","last_peered":"2024-09-21T02:52:15.556144+0000","last_clean":"2024-09-21T02:52:15.556144+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:52:15.556144+0000","last_undegraded":"2024-09-21T02:52:15.556144+0000","last_fullsized":"2024-09-21T02:52:15.556144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'8","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:58.703005+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:52:58.703005+0000","last_peered":"2024-09-21T02:52:58.703005+0000","last_clean":"2024-09-21T02:52:58.703005+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:52:58.703005+0000","last_undegraded":"2024-09-21T02:52:58.703005+0000","last_fullsized":"2024-09-21T02:52:58.703005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.562844+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:52:15.562844+0000","last_peered":"2024-09-21T02:52:15.562844+0000","last_clean":"2024-09-21T02:52:15.562844+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:52:15.562844+0000","last_undegraded":"2024-09-21T02:52:15.562844+0000","last_fullsized":"2024-09-21T02:52:15.562844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571605+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571605+0000","last_peered":"2024-09-21T02:51:16.571605+0000","last_clean":"2024-09-21T02:51:16.571605+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571605+0000","last_undegraded":"2024-09-21T02:51:16.571605+0000","last_fullsized":"2024-09-21T02:51:16.571605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.549381+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:52:15.549381+0000","last_peered":"2024-09-21T02:52:15.549381+0000","last_clean":"2024-09-21T02:52:15.549381+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:52:15.549381+0000","last_undegraded":"2024-09-21T02:52:15.549381+0000","last_fullsized":"2024-09-21T02:52:15.549381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553921+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:52:15.553921+0000","last_peered":"2024-09-21T02:52:15.553921+0000","last_clean":"2024-09-21T02:52:15.553921+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:52:15.553921+0000","last_undegraded":"2024-09-21T02:52:15.553921+0000","last_fullsized":"2024-09-21T02:52:15.553921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:18.365997+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:18.365997+0000","last_peered":"2024-09-21T02:51:18.365997+0000","last_clean":"2024-09-21T02:51:18.365997+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:18.365997+0000","last_undegraded":"2024-09-21T02:51:18.365997+0000","last_fullsized":"2024-09-21T02:51:18.365997+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552768+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:52:15.552768+0000","last_peered":"2024-09-21T02:52:15.552768+0000","last_clean":"2024-09-21T02:52:15.552768+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:52:15.552768+0000","last_undegraded":"2024-09-21T02:52:15.552768+0000","last_fullsized":"2024-09-21T02:52:15.552768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574014+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.574014+0000","last_peered":"2024-09-21T02:51:16.574014+0000","last_clean":"2024-09-21T02:51:16.574014+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.574014+0000","last_undegraded":"2024-09-21T02:51:16.574014+0000","last_fullsized":"2024-09-21T02:51:16.574014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563682+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:52:15.563682+0000","last_peered":"2024-09-21T02:52:15.563682+0000","last_clean":"2024-09-21T02:52:15.563682+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:52:15.563682+0000","last_undegraded":"2024-09-21T02:52:15.563682+0000","last_fullsized":"2024-09-21T02:52:15.563682+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976982+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976982+0000","last_peered":"2024-09-21T02:51:16.976982+0000","last_clean":"2024-09-21T02:51:16.976982+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976982+0000","last_undegraded":"2024-09-21T02:51:16.976982+0000","last_fullsized":"2024-09-21T02:51:16.976982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.550450+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:52:15.550450+0000","last_peered":"2024-09-21T02:52:15.550450+0000","last_clean":"2024-09-21T02:52:15.550450+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:52:15.550450+0000","last_undegraded":"2024-09-21T02:52:15.550450+0000","last_fullsized":"2024-09-21T02:52:15.550450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976012+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.976012+0000","last_peered":"2024-09-21T02:51:16.976012+0000","last_clean":"2024-09-21T02:51:16.976012+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.976012+0000","last_undegraded":"2024-09-21T02:51:16.976012+0000","last_fullsized":"2024-09-21T02:51:16.976012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.559275+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:52:15.559275+0000","last_peered":"2024-09-21T02:52:15.559275+0000","last_clean":"2024-09-21T02:52:15.559275+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:52:15.559275+0000","last_undegraded":"2024-09-21T02:52:15.559275+0000","last_fullsized":"2024-09-21T02:52:15.559275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:58.432886+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:52:58.432886+0000","last_peered":"2024-09-21T02:52:58.432886+0000","last_clean":"2024-09-21T02:52:58.432886+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:52:58.432886+0000","last_undegraded":"2024-09-21T02:52:58.432886+0000","last_fullsized":"2024-09-21T02:52:58.432886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560328+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:52:15.560328+0000","last_peered":"2024-09-21T02:52:15.560328+0000","last_clean":"2024-09-21T02:52:15.560328+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:52:15.560328+0000","last_undegraded":"2024-09-21T02:52:15.560328+0000","last_fullsized":"2024-09-21T02:52:15.560328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570103+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:52:15.570103+0000","last_peered":"2024-09-21T02:52:15.570103+0000","last_clean":"2024-09-21T02:52:15.570103+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:52:15.570103+0000","last_undegraded":"2024-09-21T02:52:15.570103+0000","last_fullsized":"2024-09-21T02:52:15.570103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567443+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:52:15.567443+0000","last_peered":"2024-09-21T02:52:15.567443+0000","last_clean":"2024-09-21T02:52:15.567443+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:52:15.567443+0000","last_undegraded":"2024-09-21T02:52:15.567443+0000","last_fullsized":"2024-09-21T02:52:15.567443+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564038+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:52:15.564038+0000","last_peered":"2024-09-21T02:52:15.564038+0000","last_clean":"2024-09-21T02:52:15.564038+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:52:15.564038+0000","last_undegraded":"2024-09-21T02:52:15.564038+0000","last_fullsized":"2024-09-21T02:52:15.564038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570214+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:52:15.570214+0000","last_peered":"2024-09-21T02:52:15.570214+0000","last_clean":"2024-09-21T02:52:15.570214+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:52:15.570214+0000","last_undegraded":"2024-09-21T02:52:15.570214+0000","last_fullsized":"2024-09-21T02:52:15.570214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976625+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976625+0000","last_peered":"2024-09-21T02:51:16.976625+0000","last_clean":"2024-09-21T02:51:16.976625+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976625+0000","last_undegraded":"2024-09-21T02:51:16.976625+0000","last_fullsized":"2024-09-21T02:51:16.976625+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553717+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:52:15.553717+0000","last_peered":"2024-09-21T02:52:15.553717+0000","last_clean":"2024-09-21T02:52:15.553717+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:52:15.553717+0000","last_undegraded":"2024-09-21T02:52:15.553717+0000","last_fullsized":"2024-09-21T02:52:15.553717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'14","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:55.697859+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:52:55.697859+0000","last_peered":"2024-09-21T02:52:55.697859+0000","last_clean":"2024-09-21T02:52:55.697859+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:52:55.697859+0000","last_undegraded":"2024-09-21T02:52:55.697859+0000","last_fullsized":"2024-09-21T02:52:55.697859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73014,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1210,"num_read_kb":684,"num_write":259,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":230319,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":421333,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075316,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":1014887,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":421259,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013334,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":421659,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485285,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":491912,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":422598,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72415,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72013,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65950,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:53:01.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:53:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 -- 172.21.15.31:0/3644108299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641047e0 msgr2=0x7f8f64104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/3644108299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641047e0 0x7f8f64104bf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009900 tx=0x7f8f5802a410 comp rx=0 tx=0).stop 2024-09-21T02:53:01.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 -- 172.21.15.31:0/3644108299 shutdown_connections 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/3644108299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641051c0 0x7f8f6410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/3644108299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 0x7f8f64110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/3644108299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641047e0 0x7f8f64104bf0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 -- 172.21.15.31:0/3644108299 >> 172.21.15.31:0/3644108299 conn(0x7f8f640fe3b0 msgr2=0x7f8f64100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.650+0000 7f8f6b08b640 1 -- 172.21.15.31:0/3644108299 shutdown_connections 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.651+0000 7f8f6b08b640 1 -- 172.21.15.31:0/3644108299 wait complete. 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.651+0000 7f8f6b08b640 1 Processor -- start 2024-09-21T02:53:01.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.651+0000 7f8f6b08b640 1 -- start start 2024-09-21T02:53:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.651+0000 7f8f6b08b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641047e0 0x7f8f641026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.651+0000 7f8f6b08b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 0x7f8f64102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.651+0000 7f8f6b08b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 0x7f8f64103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f6b08b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641139f0 con 0x7f8f641051c0 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f6b08b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64113870 con 0x7f8f6410dd90 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f6b08b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64113b70 con 0x7f8f641047e0 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 0x7f8f64102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 0x7f8f64102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43286/0 (socket says 172.21.15.31:43286) 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 -- 172.21.15.31:0/1816465252 learned_addr learned my addr 172.21.15.31:0/1816465252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f69628640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 0x7f8f64103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f68e27640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641047e0 0x7f8f641026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 -- 172.21.15.31:0/1816465252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641047e0 msgr2=0x7f8f641026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641047e0 0x7f8f641026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 -- 172.21.15.31:0/1816465252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 msgr2=0x7f8f64103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 0x7f8f64103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 -- 172.21.15.31:0/1816465252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f641a8720 con 0x7f8f641051c0 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f69628640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 0x7f8f64103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f68e27640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641047e0 0x7f8f641026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f63fff640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 0x7f8f64102c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000d750 tx=0x7f8f5000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50017070 con 0x7f8f641051c0 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f500054d0 con 0x7f8f641051c0 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50004c30 con 0x7f8f641051c0 2024-09-21T02:53:01.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.652+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f641a8a10 con 0x7f8f641051c0 2024-09-21T02:53:01.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.653+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641a8f10 con 0x7f8f641051c0 2024-09-21T02:53:01.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.653+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f64104bf0 con 0x7f8f641051c0 2024-09-21T02:53:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.656+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f8f50004dd0 con 0x7f8f641051c0 2024-09-21T02:53:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.657+0000 7f8f61ffb640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8f540b6620 0x7f8f540b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.657+0000 7f8f68e27640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8f540b6620 0x7f8f540b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.657+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8f500e74d0 con 0x7f8f641051c0 2024-09-21T02:53:01.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.657+0000 7f8f68e27640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8f540b6620 0x7f8f540b8b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000c00 tx=0x7f8f58009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:01.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.660+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f500adaa0 con 0x7f8f641051c0 2024-09-21T02:53:01.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.846+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f64108070 con 0x7f8f540b6620 2024-09-21T02:53:01.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.851+0000 7f8f61ffb640 1 -- 172.21.15.31:0/1816465252 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259180 (secure 0 0 0) 0x7f8f64108070 con 0x7f8f540b6620 2024-09-21T02:53:01.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:01.856 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8f540b6620 msgr2=0x7f8f540b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8f540b6620 0x7f8f540b8b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000c00 tx=0x7f8f58009680 comp rx=0 tx=0).stop 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 msgr2=0x7f8f64102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 0x7f8f64102c30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000d750 tx=0x7f8f5000dc10 comp rx=0 tx=0).stop 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 shutdown_connections 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8f540b6620 0x7f8f540b8b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6410dd90 0x7f8f64103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f641051c0 0x7f8f64102c30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 --2- 172.21.15.31:0/1816465252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f641047e0 0x7f8f641026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 >> 172.21.15.31:0/1816465252 conn(0x7f8f640fe3b0 msgr2=0x7f8f64107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 shutdown_connections 2024-09-21T02:53:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:01.854+0000 7f8f6b08b640 1 -- 172.21.15.31:0/1816465252 wait complete. 2024-09-21T02:53:01.867 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-21T02:53:01.175607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663616,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1291,"num_read_kb":753,"num_write":384,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376452,"kb_used_data":10564,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565372,"statfs":{"total":1151940427776,"available":1151554940928,"internally_reserved":0,"allocated":10817536,"data_stored":7029248,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1398,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":58,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003453"},"pg_stats":[{"pgid":"3.3f","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567590+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:52:15.567590+0000","last_peered":"2024-09-21T02:52:15.567590+0000","last_clean":"2024-09-21T02:52:15.567590+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:52:15.567590+0000","last_undegraded":"2024-09-21T02:52:15.567590+0000","last_fullsized":"2024-09-21T02:52:15.567590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563476+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:52:15.563476+0000","last_peered":"2024-09-21T02:52:15.563476+0000","last_clean":"2024-09-21T02:52:15.563476+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:52:15.563476+0000","last_undegraded":"2024-09-21T02:52:15.563476+0000","last_fullsized":"2024-09-21T02:52:15.563476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569310+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:52:15.569310+0000","last_peered":"2024-09-21T02:52:15.569310+0000","last_clean":"2024-09-21T02:52:15.569310+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:52:15.569310+0000","last_undegraded":"2024-09-21T02:52:15.569310+0000","last_fullsized":"2024-09-21T02:52:15.569310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557143+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:52:15.557143+0000","last_peered":"2024-09-21T02:52:15.557143+0000","last_clean":"2024-09-21T02:52:15.557143+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:52:15.557143+0000","last_undegraded":"2024-09-21T02:52:15.557143+0000","last_fullsized":"2024-09-21T02:52:15.557143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.546827+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:52:15.546827+0000","last_peered":"2024-09-21T02:52:15.546827+0000","last_clean":"2024-09-21T02:52:15.546827+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:52:15.546827+0000","last_undegraded":"2024-09-21T02:52:15.546827+0000","last_fullsized":"2024-09-21T02:52:15.546827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.540853+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:52:15.540853+0000","last_peered":"2024-09-21T02:52:15.540853+0000","last_clean":"2024-09-21T02:52:15.540853+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:52:15.540853+0000","last_undegraded":"2024-09-21T02:52:15.540853+0000","last_fullsized":"2024-09-21T02:52:15.540853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569968+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:52:15.569968+0000","last_peered":"2024-09-21T02:52:15.569968+0000","last_clean":"2024-09-21T02:52:15.569968+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:52:15.569968+0000","last_undegraded":"2024-09-21T02:52:15.569968+0000","last_fullsized":"2024-09-21T02:52:15.569968+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558545+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:52:15.558545+0000","last_peered":"2024-09-21T02:52:15.558545+0000","last_clean":"2024-09-21T02:52:15.558545+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:52:15.558545+0000","last_undegraded":"2024-09-21T02:52:15.558545+0000","last_fullsized":"2024-09-21T02:52:15.558545+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567267+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:52:15.567267+0000","last_peered":"2024-09-21T02:52:15.567267+0000","last_clean":"2024-09-21T02:52:15.567267+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:52:15.567267+0000","last_undegraded":"2024-09-21T02:52:15.567267+0000","last_fullsized":"2024-09-21T02:52:15.567267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.561944+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:52:15.561944+0000","last_peered":"2024-09-21T02:52:15.561944+0000","last_clean":"2024-09-21T02:52:15.561944+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:52:15.561944+0000","last_undegraded":"2024-09-21T02:52:15.561944+0000","last_fullsized":"2024-09-21T02:52:15.561944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567208+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:52:15.567208+0000","last_peered":"2024-09-21T02:52:15.567208+0000","last_clean":"2024-09-21T02:52:15.567208+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:52:15.567208+0000","last_undegraded":"2024-09-21T02:52:15.567208+0000","last_fullsized":"2024-09-21T02:52:15.567208+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552898+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:52:15.552898+0000","last_peered":"2024-09-21T02:52:15.552898+0000","last_clean":"2024-09-21T02:52:15.552898+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:52:15.552898+0000","last_undegraded":"2024-09-21T02:52:15.552898+0000","last_fullsized":"2024-09-21T02:52:15.552898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.554532+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:52:15.554532+0000","last_peered":"2024-09-21T02:52:15.554532+0000","last_clean":"2024-09-21T02:52:15.554532+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:52:15.554532+0000","last_undegraded":"2024-09-21T02:52:15.554532+0000","last_fullsized":"2024-09-21T02:52:15.554532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563207+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:52:15.563207+0000","last_peered":"2024-09-21T02:52:15.563207+0000","last_clean":"2024-09-21T02:52:15.563207+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:52:15.563207+0000","last_undegraded":"2024-09-21T02:52:15.563207+0000","last_fullsized":"2024-09-21T02:52:15.563207+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563556+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:52:15.563556+0000","last_peered":"2024-09-21T02:52:15.563556+0000","last_clean":"2024-09-21T02:52:15.563556+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:52:15.563556+0000","last_undegraded":"2024-09-21T02:52:15.563556+0000","last_fullsized":"2024-09-21T02:52:15.563556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.568347+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:52:15.568347+0000","last_peered":"2024-09-21T02:52:15.568347+0000","last_clean":"2024-09-21T02:52:15.568347+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:52:15.568347+0000","last_undegraded":"2024-09-21T02:52:15.568347+0000","last_fullsized":"2024-09-21T02:52:15.568347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567564+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:52:15.567564+0000","last_peered":"2024-09-21T02:52:15.567564+0000","last_clean":"2024-09-21T02:52:15.567564+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:52:15.567564+0000","last_undegraded":"2024-09-21T02:52:15.567564+0000","last_fullsized":"2024-09-21T02:52:15.567564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557763+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:52:15.557763+0000","last_peered":"2024-09-21T02:52:15.557763+0000","last_clean":"2024-09-21T02:52:15.557763+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:52:15.557763+0000","last_undegraded":"2024-09-21T02:52:15.557763+0000","last_fullsized":"2024-09-21T02:52:15.557763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552086+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:52:15.552086+0000","last_peered":"2024-09-21T02:52:15.552086+0000","last_clean":"2024-09-21T02:52:15.552086+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:52:15.552086+0000","last_undegraded":"2024-09-21T02:52:15.552086+0000","last_fullsized":"2024-09-21T02:52:15.552086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558416+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:52:15.558416+0000","last_peered":"2024-09-21T02:52:15.558416+0000","last_clean":"2024-09-21T02:52:15.558416+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:52:15.558416+0000","last_undegraded":"2024-09-21T02:52:15.558416+0000","last_fullsized":"2024-09-21T02:52:15.558416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560852+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:52:15.560852+0000","last_peered":"2024-09-21T02:52:15.560852+0000","last_clean":"2024-09-21T02:52:15.560852+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:52:15.560852+0000","last_undegraded":"2024-09-21T02:52:15.560852+0000","last_fullsized":"2024-09-21T02:52:15.560852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.555555+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:52:15.555555+0000","last_peered":"2024-09-21T02:52:15.555555+0000","last_clean":"2024-09-21T02:52:15.555555+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:52:15.555555+0000","last_undegraded":"2024-09-21T02:52:15.555555+0000","last_fullsized":"2024-09-21T02:52:15.555555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557760+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:52:15.557760+0000","last_peered":"2024-09-21T02:52:15.557760+0000","last_clean":"2024-09-21T02:52:15.557760+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:52:15.557760+0000","last_undegraded":"2024-09-21T02:52:15.557760+0000","last_fullsized":"2024-09-21T02:52:15.557760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.548121+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:52:15.548121+0000","last_peered":"2024-09-21T02:52:15.548121+0000","last_clean":"2024-09-21T02:52:15.548121+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:52:15.548121+0000","last_undegraded":"2024-09-21T02:52:15.548121+0000","last_fullsized":"2024-09-21T02:52:15.548121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.549005+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:52:15.549005+0000","last_peered":"2024-09-21T02:52:15.549005+0000","last_clean":"2024-09-21T02:52:15.549005+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:52:15.549005+0000","last_undegraded":"2024-09-21T02:52:15.549005+0000","last_fullsized":"2024-09-21T02:52:15.549005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.565080+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:52:15.565080+0000","last_peered":"2024-09-21T02:52:15.565080+0000","last_clean":"2024-09-21T02:52:15.565080+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:52:15.565080+0000","last_undegraded":"2024-09-21T02:52:15.565080+0000","last_fullsized":"2024-09-21T02:52:15.565080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.554835+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:52:15.554835+0000","last_peered":"2024-09-21T02:52:15.554835+0000","last_clean":"2024-09-21T02:52:15.554835+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:52:15.554835+0000","last_undegraded":"2024-09-21T02:52:15.554835+0000","last_fullsized":"2024-09-21T02:52:15.554835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560546+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:52:15.560546+0000","last_peered":"2024-09-21T02:52:15.560546+0000","last_clean":"2024-09-21T02:52:15.560546+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:52:15.560546+0000","last_undegraded":"2024-09-21T02:52:15.560546+0000","last_fullsized":"2024-09-21T02:52:15.560546+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.571336+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:52:15.571336+0000","last_peered":"2024-09-21T02:52:15.571336+0000","last_clean":"2024-09-21T02:52:15.571336+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:52:15.571336+0000","last_undegraded":"2024-09-21T02:52:15.571336+0000","last_fullsized":"2024-09-21T02:52:15.571336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558911+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:52:15.558911+0000","last_peered":"2024-09-21T02:52:15.558911+0000","last_clean":"2024-09-21T02:52:15.558911+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:52:15.558911+0000","last_undegraded":"2024-09-21T02:52:15.558911+0000","last_fullsized":"2024-09-21T02:52:15.558911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.555496+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:52:15.555496+0000","last_peered":"2024-09-21T02:52:15.555496+0000","last_clean":"2024-09-21T02:52:15.555496+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:52:15.555496+0000","last_undegraded":"2024-09-21T02:52:15.555496+0000","last_fullsized":"2024-09-21T02:52:15.555496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557445+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:52:15.557445+0000","last_peered":"2024-09-21T02:52:15.557445+0000","last_clean":"2024-09-21T02:52:15.557445+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:52:15.557445+0000","last_undegraded":"2024-09-21T02:52:15.557445+0000","last_fullsized":"2024-09-21T02:52:15.557445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.544310+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:52:15.544310+0000","last_peered":"2024-09-21T02:52:15.544310+0000","last_clean":"2024-09-21T02:52:15.544310+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:52:15.544310+0000","last_undegraded":"2024-09-21T02:52:15.544310+0000","last_fullsized":"2024-09-21T02:52:15.544310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":165,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.733243+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:52:57.733243+0000","last_peered":"2024-09-21T02:52:57.733243+0000","last_clean":"2024-09-21T02:52:57.733243+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:52:57.733243+0000","last_undegraded":"2024-09-21T02:52:57.733243+0000","last_fullsized":"2024-09-21T02:52:57.733243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.551262+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:52:15.551262+0000","last_peered":"2024-09-21T02:52:15.551262+0000","last_clean":"2024-09-21T02:52:15.551262+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:52:15.551262+0000","last_undegraded":"2024-09-21T02:52:15.551262+0000","last_fullsized":"2024-09-21T02:52:15.551262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.556254+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:52:15.556254+0000","last_peered":"2024-09-21T02:52:15.556254+0000","last_clean":"2024-09-21T02:52:15.556254+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:52:15.556254+0000","last_undegraded":"2024-09-21T02:52:15.556254+0000","last_fullsized":"2024-09-21T02:52:15.556254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.551455+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:52:15.551455+0000","last_peered":"2024-09-21T02:52:15.551455+0000","last_clean":"2024-09-21T02:52:15.551455+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:52:15.551455+0000","last_undegraded":"2024-09-21T02:52:15.551455+0000","last_fullsized":"2024-09-21T02:52:15.551455+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'6","reported_seq":261,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:56.700068+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:52:56.700068+0000","last_peered":"2024-09-21T02:52:56.700068+0000","last_clean":"2024-09-21T02:52:56.700068+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:52:56.700068+0000","last_undegraded":"2024-09-21T02:52:56.700068+0000","last_fullsized":"2024-09-21T02:52:56.700068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563004+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:52:15.563004+0000","last_peered":"2024-09-21T02:52:15.563004+0000","last_clean":"2024-09-21T02:52:15.563004+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:52:15.563004+0000","last_undegraded":"2024-09-21T02:52:15.563004+0000","last_fullsized":"2024-09-21T02:52:15.563004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564496+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:52:15.564496+0000","last_peered":"2024-09-21T02:52:15.564496+0000","last_clean":"2024-09-21T02:52:15.564496+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:52:15.564496+0000","last_undegraded":"2024-09-21T02:52:15.564496+0000","last_fullsized":"2024-09-21T02:52:15.564496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.565141+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:52:15.565141+0000","last_peered":"2024-09-21T02:52:15.565141+0000","last_clean":"2024-09-21T02:52:15.565141+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:52:15.565141+0000","last_undegraded":"2024-09-21T02:52:15.565141+0000","last_fullsized":"2024-09-21T02:52:15.565141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'33","reported_seq":297,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:56.731591+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:52:56.731591+0000","last_peered":"2024-09-21T02:52:56.731591+0000","last_clean":"2024-09-21T02:52:56.731591+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:52:56.731591+0000","last_undegraded":"2024-09-21T02:52:56.731591+0000","last_fullsized":"2024-09-21T02:52:56.731591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":182,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.550696+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:52:15.550696+0000","last_peered":"2024-09-21T02:52:15.550696+0000","last_clean":"2024-09-21T02:52:15.550696+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:52:15.550696+0000","last_undegraded":"2024-09-21T02:52:15.550696+0000","last_fullsized":"2024-09-21T02:52:15.550696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552036+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:52:15.552036+0000","last_peered":"2024-09-21T02:52:15.552036+0000","last_clean":"2024-09-21T02:52:15.552036+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:52:15.552036+0000","last_undegraded":"2024-09-21T02:52:15.552036+0000","last_fullsized":"2024-09-21T02:52:15.552036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567369+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:52:15.567369+0000","last_peered":"2024-09-21T02:52:15.567369+0000","last_clean":"2024-09-21T02:52:15.567369+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:52:15.567369+0000","last_undegraded":"2024-09-21T02:52:15.567369+0000","last_fullsized":"2024-09-21T02:52:15.567369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.712619+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:52:57.712619+0000","last_peered":"2024-09-21T02:52:57.712619+0000","last_clean":"2024-09-21T02:52:57.712619+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:52:57.712619+0000","last_undegraded":"2024-09-21T02:52:57.712619+0000","last_fullsized":"2024-09-21T02:52:57.712619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569073+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:52:15.569073+0000","last_peered":"2024-09-21T02:52:15.569073+0000","last_clean":"2024-09-21T02:52:15.569073+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:52:15.569073+0000","last_undegraded":"2024-09-21T02:52:15.569073+0000","last_fullsized":"2024-09-21T02:52:15.569073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.566458+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:52:15.566458+0000","last_peered":"2024-09-21T02:52:15.566458+0000","last_clean":"2024-09-21T02:52:15.566458+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:52:15.566458+0000","last_undegraded":"2024-09-21T02:52:15.566458+0000","last_fullsized":"2024-09-21T02:52:15.566458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558056+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:52:15.558056+0000","last_peered":"2024-09-21T02:52:15.558056+0000","last_clean":"2024-09-21T02:52:15.558056+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:52:15.558056+0000","last_undegraded":"2024-09-21T02:52:15.558056+0000","last_fullsized":"2024-09-21T02:52:15.558056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.568405+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:52:15.568405+0000","last_peered":"2024-09-21T02:52:15.568405+0000","last_clean":"2024-09-21T02:52:15.568405+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:52:15.568405+0000","last_undegraded":"2024-09-21T02:52:15.568405+0000","last_fullsized":"2024-09-21T02:52:15.568405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570341+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:52:15.570341+0000","last_peered":"2024-09-21T02:52:15.570341+0000","last_clean":"2024-09-21T02:52:15.570341+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:52:15.570341+0000","last_undegraded":"2024-09-21T02:52:15.570341+0000","last_fullsized":"2024-09-21T02:52:15.570341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'8","reported_seq":152,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.702109+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:52:57.702109+0000","last_peered":"2024-09-21T02:52:57.702109+0000","last_clean":"2024-09-21T02:52:57.702109+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:52:57.702109+0000","last_undegraded":"2024-09-21T02:52:57.702109+0000","last_fullsized":"2024-09-21T02:52:57.702109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569660+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:52:15.569660+0000","last_peered":"2024-09-21T02:52:15.569660+0000","last_clean":"2024-09-21T02:52:15.569660+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:52:15.569660+0000","last_undegraded":"2024-09-21T02:52:15.569660+0000","last_fullsized":"2024-09-21T02:52:15.569660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576900+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576900+0000","last_peered":"2024-09-21T02:51:16.576900+0000","last_clean":"2024-09-21T02:51:16.576900+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576900+0000","last_undegraded":"2024-09-21T02:51:16.576900+0000","last_fullsized":"2024-09-21T02:51:16.576900+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573515+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573515+0000","last_peered":"2024-09-21T02:51:16.573515+0000","last_clean":"2024-09-21T02:51:16.573515+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573515+0000","last_undegraded":"2024-09-21T02:51:16.573515+0000","last_fullsized":"2024-09-21T02:51:16.573515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564758+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:52:15.564758+0000","last_peered":"2024-09-21T02:52:15.564758+0000","last_clean":"2024-09-21T02:52:15.564758+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:52:15.564758+0000","last_undegraded":"2024-09-21T02:52:15.564758+0000","last_fullsized":"2024-09-21T02:52:15.564758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576845+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576845+0000","last_peered":"2024-09-21T02:51:16.576845+0000","last_clean":"2024-09-21T02:51:16.576845+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576845+0000","last_undegraded":"2024-09-21T02:51:16.576845+0000","last_fullsized":"2024-09-21T02:51:16.576845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553787+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:52:15.553787+0000","last_peered":"2024-09-21T02:52:15.553787+0000","last_clean":"2024-09-21T02:52:15.553787+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:52:15.553787+0000","last_undegraded":"2024-09-21T02:52:15.553787+0000","last_fullsized":"2024-09-21T02:52:15.553787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'6","reported_seq":151,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:59.715214+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:52:59.715214+0000","last_peered":"2024-09-21T02:52:59.715214+0000","last_clean":"2024-09-21T02:52:59.715214+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:52:59.715214+0000","last_undegraded":"2024-09-21T02:52:59.715214+0000","last_fullsized":"2024-09-21T02:52:59.715214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.556144+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:52:15.556144+0000","last_peered":"2024-09-21T02:52:15.556144+0000","last_clean":"2024-09-21T02:52:15.556144+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:52:15.556144+0000","last_undegraded":"2024-09-21T02:52:15.556144+0000","last_fullsized":"2024-09-21T02:52:15.556144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'8","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:58.703005+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:52:58.703005+0000","last_peered":"2024-09-21T02:52:58.703005+0000","last_clean":"2024-09-21T02:52:58.703005+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:52:58.703005+0000","last_undegraded":"2024-09-21T02:52:58.703005+0000","last_fullsized":"2024-09-21T02:52:58.703005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.562844+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:52:15.562844+0000","last_peered":"2024-09-21T02:52:15.562844+0000","last_clean":"2024-09-21T02:52:15.562844+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:52:15.562844+0000","last_undegraded":"2024-09-21T02:52:15.562844+0000","last_fullsized":"2024-09-21T02:52:15.562844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571605+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571605+0000","last_peered":"2024-09-21T02:51:16.571605+0000","last_clean":"2024-09-21T02:51:16.571605+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571605+0000","last_undegraded":"2024-09-21T02:51:16.571605+0000","last_fullsized":"2024-09-21T02:51:16.571605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.549381+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:52:15.549381+0000","last_peered":"2024-09-21T02:52:15.549381+0000","last_clean":"2024-09-21T02:52:15.549381+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:52:15.549381+0000","last_undegraded":"2024-09-21T02:52:15.549381+0000","last_fullsized":"2024-09-21T02:52:15.549381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553921+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:52:15.553921+0000","last_peered":"2024-09-21T02:52:15.553921+0000","last_clean":"2024-09-21T02:52:15.553921+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:52:15.553921+0000","last_undegraded":"2024-09-21T02:52:15.553921+0000","last_fullsized":"2024-09-21T02:52:15.553921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:18.365997+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:18.365997+0000","last_peered":"2024-09-21T02:51:18.365997+0000","last_clean":"2024-09-21T02:51:18.365997+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:18.365997+0000","last_undegraded":"2024-09-21T02:51:18.365997+0000","last_fullsized":"2024-09-21T02:51:18.365997+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552768+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:52:15.552768+0000","last_peered":"2024-09-21T02:52:15.552768+0000","last_clean":"2024-09-21T02:52:15.552768+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:52:15.552768+0000","last_undegraded":"2024-09-21T02:52:15.552768+0000","last_fullsized":"2024-09-21T02:52:15.552768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574014+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.574014+0000","last_peered":"2024-09-21T02:51:16.574014+0000","last_clean":"2024-09-21T02:51:16.574014+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.574014+0000","last_undegraded":"2024-09-21T02:51:16.574014+0000","last_fullsized":"2024-09-21T02:51:16.574014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563682+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:52:15.563682+0000","last_peered":"2024-09-21T02:52:15.563682+0000","last_clean":"2024-09-21T02:52:15.563682+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:52:15.563682+0000","last_undegraded":"2024-09-21T02:52:15.563682+0000","last_fullsized":"2024-09-21T02:52:15.563682+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976982+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976982+0000","last_peered":"2024-09-21T02:51:16.976982+0000","last_clean":"2024-09-21T02:51:16.976982+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976982+0000","last_undegraded":"2024-09-21T02:51:16.976982+0000","last_fullsized":"2024-09-21T02:51:16.976982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.550450+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:52:15.550450+0000","last_peered":"2024-09-21T02:52:15.550450+0000","last_clean":"2024-09-21T02:52:15.550450+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:52:15.550450+0000","last_undegraded":"2024-09-21T02:52:15.550450+0000","last_fullsized":"2024-09-21T02:52:15.550450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976012+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.976012+0000","last_peered":"2024-09-21T02:51:16.976012+0000","last_clean":"2024-09-21T02:51:16.976012+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.976012+0000","last_undegraded":"2024-09-21T02:51:16.976012+0000","last_fullsized":"2024-09-21T02:51:16.976012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.559275+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:52:15.559275+0000","last_peered":"2024-09-21T02:52:15.559275+0000","last_clean":"2024-09-21T02:52:15.559275+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:52:15.559275+0000","last_undegraded":"2024-09-21T02:52:15.559275+0000","last_fullsized":"2024-09-21T02:52:15.559275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:58.432886+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:52:58.432886+0000","last_peered":"2024-09-21T02:52:58.432886+0000","last_clean":"2024-09-21T02:52:58.432886+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:52:58.432886+0000","last_undegraded":"2024-09-21T02:52:58.432886+0000","last_fullsized":"2024-09-21T02:52:58.432886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560328+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:52:15.560328+0000","last_peered":"2024-09-21T02:52:15.560328+0000","last_clean":"2024-09-21T02:52:15.560328+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:52:15.560328+0000","last_undegraded":"2024-09-21T02:52:15.560328+0000","last_fullsized":"2024-09-21T02:52:15.560328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570103+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:52:15.570103+0000","last_peered":"2024-09-21T02:52:15.570103+0000","last_clean":"2024-09-21T02:52:15.570103+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:52:15.570103+0000","last_undegraded":"2024-09-21T02:52:15.570103+0000","last_fullsized":"2024-09-21T02:52:15.570103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567443+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:52:15.567443+0000","last_peered":"2024-09-21T02:52:15.567443+0000","last_clean":"2024-09-21T02:52:15.567443+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:52:15.567443+0000","last_undegraded":"2024-09-21T02:52:15.567443+0000","last_fullsized":"2024-09-21T02:52:15.567443+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564038+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:52:15.564038+0000","last_peered":"2024-09-21T02:52:15.564038+0000","last_clean":"2024-09-21T02:52:15.564038+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:52:15.564038+0000","last_undegraded":"2024-09-21T02:52:15.564038+0000","last_fullsized":"2024-09-21T02:52:15.564038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570214+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:52:15.570214+0000","last_peered":"2024-09-21T02:52:15.570214+0000","last_clean":"2024-09-21T02:52:15.570214+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:52:15.570214+0000","last_undegraded":"2024-09-21T02:52:15.570214+0000","last_fullsized":"2024-09-21T02:52:15.570214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976625+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976625+0000","last_peered":"2024-09-21T02:51:16.976625+0000","last_clean":"2024-09-21T02:51:16.976625+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976625+0000","last_undegraded":"2024-09-21T02:51:16.976625+0000","last_fullsized":"2024-09-21T02:51:16.976625+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553717+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:52:15.553717+0000","last_peered":"2024-09-21T02:52:15.553717+0000","last_clean":"2024-09-21T02:52:15.553717+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:52:15.553717+0000","last_undegraded":"2024-09-21T02:52:15.553717+0000","last_fullsized":"2024-09-21T02:52:15.553717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'14","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:55.697859+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:52:55.697859+0000","last_peered":"2024-09-21T02:52:55.697859+0000","last_clean":"2024-09-21T02:52:55.697859+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:52:55.697859+0000","last_undegraded":"2024-09-21T02:52:55.697859+0000","last_fullsized":"2024-09-21T02:52:55.697859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73014,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1210,"num_read_kb":684,"num_write":259,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":230319,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":421333,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075316,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":1014887,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":421259,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013334,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":421659,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485285,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":491912,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":422598,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72415,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72013,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65950,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:53:01.871 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T02:53:01.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:53:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 -- 172.21.15.31:0/3352895572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 msgr2=0x7f7640107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 --2- 172.21.15.31:0/3352895572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f7640107ea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f762c009900 tx=0x7f762c02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 -- 172.21.15.31:0/3352895572 shutdown_connections 2024-09-21T02:53:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 --2- 172.21.15.31:0/3352895572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76401083e0 0x7f764010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 --2- 172.21.15.31:0/3352895572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f7640107ea0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 --2- 172.21.15.31:0/3352895572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76401021b0 0x7f76401025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 -- 172.21.15.31:0/3352895572 >> 172.21.15.31:0/3352895572 conn(0x7f7640079d00 msgr2=0x7f764007a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 -- 172.21.15.31:0/3352895572 shutdown_connections 2024-09-21T02:53:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.019+0000 7f7646a9b640 1 -- 172.21.15.31:0/3352895572 wait complete. 2024-09-21T02:53:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 Processor -- start 2024-09-21T02:53:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 -- start start 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76401021b0 0x7f764019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f764019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76401083e0 0x7f764019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112920 con 0x7f7640107a90 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76401127a0 con 0x7f76401083e0 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f7646a9b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7640112aa0 con 0x7f76401021b0 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.020+0000 7f763ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f764019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f764019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43302/0 (socket says 172.21.15.31:43302) 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 -- 172.21.15.31:0/1359959294 learned_addr learned my addr 172.21.15.31:0/1359959294 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f7644837640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76401021b0 0x7f764019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f7645038640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76401083e0 0x7f764019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 -- 172.21.15.31:0/1359959294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76401021b0 msgr2=0x7f764019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76401021b0 0x7f764019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 -- 172.21.15.31:0/1359959294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76401083e0 msgr2=0x7f764019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76401083e0 0x7f764019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 -- 172.21.15.31:0/1359959294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401a64a0 con 0x7f7640107a90 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763ffff640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f764019b750 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f762c02a930 tx=0x7f762c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c03f030 con 0x7f7640107a90 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c003aa0 con 0x7f7640107a90 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401a6730 con 0x7f7640107a90 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.021+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c035070 con 0x7f7640107a90 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.022+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a6c20 con 0x7f7640107a90 2024-09-21T02:53:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.023+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76401025c0 con 0x7f7640107a90 2024-09-21T02:53:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.024+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f762c042030 con 0x7f7640107a90 2024-09-21T02:53:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.025+0000 7f763dffb640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76300b65f0 0x7f76300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.025+0000 7f7644837640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76300b65f0 0x7f76300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.025+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f762c1080d0 con 0x7f7640107a90 2024-09-21T02:53:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.026+0000 7f7644837640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76300b65f0 0x7f76300b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7640110130 tx=0x7f7634009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.028+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f762c0d2a80 con 0x7f7640107a90 2024-09-21T02:53:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.216+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7640063800 con 0x7f76300b65f0 2024-09-21T02:53:02.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.221+0000 7f763dffb640 1 -- 172.21.15.31:0/1359959294 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259180 (secure 0 0 0) 0x7f7640063800 con 0x7f76300b65f0 2024-09-21T02:53:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T02:53:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.224+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76300b65f0 msgr2=0x7f76300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.224+0000 7f7646a9b640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76300b65f0 0x7f76300b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7640110130 tx=0x7f7634009760 comp rx=0 tx=0).stop 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 msgr2=0x7f764019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f764019b750 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f762c02a930 tx=0x7f762c004270 comp rx=0 tx=0).stop 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 shutdown_connections 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76300b65f0 0x7f76300b8b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76401083e0 0x7f764019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7640107a90 0x7f764019b750 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 --2- 172.21.15.31:0/1359959294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76401021b0 0x7f764019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 >> 172.21.15.31:0/1359959294 conn(0x7f7640079d00 msgr2=0x7f7640105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 shutdown_connections 2024-09-21T02:53:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:02.225+0000 7f7646a9b640 1 -- 172.21.15.31:0/1359959294 wait complete. 2024-09-21T02:53:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-21T02:53:01.175607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663616,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1291,"num_read_kb":753,"num_write":384,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952965,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":818,"ondisk_log_size":818,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376452,"kb_used_data":10564,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565372,"statfs":{"total":1151940427776,"available":1151554940928,"internally_reserved":0,"allocated":10817536,"data_stored":7029248,"data_compressed":183849,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1398,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":58,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003453"},"pg_stats":[{"pgid":"3.3f","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567590+0000","last_change":"2024-09-21T02:50:50.366392+0000","last_active":"2024-09-21T02:52:15.567590+0000","last_peered":"2024-09-21T02:52:15.567590+0000","last_clean":"2024-09-21T02:52:15.567590+0000","last_became_active":"2024-09-21T02:50:50.366034+0000","last_became_peered":"2024-09-21T02:50:50.366034+0000","last_unstale":"2024-09-21T02:52:15.567590+0000","last_undegraded":"2024-09-21T02:52:15.567590+0000","last_fullsized":"2024-09-21T02:52:15.567590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:30.236258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563476+0000","last_change":"2024-09-21T02:50:50.358275+0000","last_active":"2024-09-21T02:52:15.563476+0000","last_peered":"2024-09-21T02:52:15.563476+0000","last_clean":"2024-09-21T02:52:15.563476+0000","last_became_active":"2024-09-21T02:50:50.357893+0000","last_became_peered":"2024-09-21T02:50:50.357893+0000","last_unstale":"2024-09-21T02:52:15.563476+0000","last_undegraded":"2024-09-21T02:52:15.563476+0000","last_fullsized":"2024-09-21T02:52:15.563476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:27.006783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569310+0000","last_change":"2024-09-21T02:50:50.367476+0000","last_active":"2024-09-21T02:52:15.569310+0000","last_peered":"2024-09-21T02:52:15.569310+0000","last_clean":"2024-09-21T02:52:15.569310+0000","last_became_active":"2024-09-21T02:50:50.367074+0000","last_became_peered":"2024-09-21T02:50:50.367074+0000","last_unstale":"2024-09-21T02:52:15.569310+0000","last_undegraded":"2024-09-21T02:52:15.569310+0000","last_fullsized":"2024-09-21T02:52:15.569310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:24.585897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557143+0000","last_change":"2024-09-21T02:50:50.351890+0000","last_active":"2024-09-21T02:52:15.557143+0000","last_peered":"2024-09-21T02:52:15.557143+0000","last_clean":"2024-09-21T02:52:15.557143+0000","last_became_active":"2024-09-21T02:50:50.351421+0000","last_became_peered":"2024-09-21T02:50:50.351421+0000","last_unstale":"2024-09-21T02:52:15.557143+0000","last_undegraded":"2024-09-21T02:52:15.557143+0000","last_fullsized":"2024-09-21T02:52:15.557143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:37:51.021668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.546827+0000","last_change":"2024-09-21T02:50:50.356652+0000","last_active":"2024-09-21T02:52:15.546827+0000","last_peered":"2024-09-21T02:52:15.546827+0000","last_clean":"2024-09-21T02:52:15.546827+0000","last_became_active":"2024-09-21T02:50:50.356114+0000","last_became_peered":"2024-09-21T02:50:50.356114+0000","last_unstale":"2024-09-21T02:52:15.546827+0000","last_undegraded":"2024-09-21T02:52:15.546827+0000","last_fullsized":"2024-09-21T02:52:15.546827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:02.394590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.540853+0000","last_change":"2024-09-21T02:51:16.578212+0000","last_active":"2024-09-21T02:52:15.540853+0000","last_peered":"2024-09-21T02:52:15.540853+0000","last_clean":"2024-09-21T02:52:15.540853+0000","last_became_active":"2024-09-21T02:51:16.577741+0000","last_became_peered":"2024-09-21T02:51:16.577741+0000","last_unstale":"2024-09-21T02:52:15.540853+0000","last_undegraded":"2024-09-21T02:52:15.540853+0000","last_fullsized":"2024-09-21T02:52:15.540853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:41.397521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569968+0000","last_change":"2024-09-21T02:50:50.356310+0000","last_active":"2024-09-21T02:52:15.569968+0000","last_peered":"2024-09-21T02:52:15.569968+0000","last_clean":"2024-09-21T02:52:15.569968+0000","last_became_active":"2024-09-21T02:50:50.354912+0000","last_became_peered":"2024-09-21T02:50:50.354912+0000","last_unstale":"2024-09-21T02:52:15.569968+0000","last_undegraded":"2024-09-21T02:52:15.569968+0000","last_fullsized":"2024-09-21T02:52:15.569968+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:14.032668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558545+0000","last_change":"2024-09-21T02:50:50.355810+0000","last_active":"2024-09-21T02:52:15.558545+0000","last_peered":"2024-09-21T02:52:15.558545+0000","last_clean":"2024-09-21T02:52:15.558545+0000","last_became_active":"2024-09-21T02:50:50.355450+0000","last_became_peered":"2024-09-21T02:50:50.355450+0000","last_unstale":"2024-09-21T02:52:15.558545+0000","last_undegraded":"2024-09-21T02:52:15.558545+0000","last_fullsized":"2024-09-21T02:52:15.558545+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:13.308764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567267+0000","last_change":"2024-09-21T02:50:50.364707+0000","last_active":"2024-09-21T02:52:15.567267+0000","last_peered":"2024-09-21T02:52:15.567267+0000","last_clean":"2024-09-21T02:52:15.567267+0000","last_became_active":"2024-09-21T02:50:50.364341+0000","last_became_peered":"2024-09-21T02:50:50.364341+0000","last_unstale":"2024-09-21T02:52:15.567267+0000","last_undegraded":"2024-09-21T02:52:15.567267+0000","last_fullsized":"2024-09-21T02:52:15.567267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:36.518935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.561944+0000","last_change":"2024-09-21T02:50:50.363603+0000","last_active":"2024-09-21T02:52:15.561944+0000","last_peered":"2024-09-21T02:52:15.561944+0000","last_clean":"2024-09-21T02:52:15.561944+0000","last_became_active":"2024-09-21T02:50:50.363212+0000","last_became_peered":"2024-09-21T02:50:50.363212+0000","last_unstale":"2024-09-21T02:52:15.561944+0000","last_undegraded":"2024-09-21T02:52:15.561944+0000","last_fullsized":"2024-09-21T02:52:15.561944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:11.898720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567208+0000","last_change":"2024-09-21T02:50:50.368675+0000","last_active":"2024-09-21T02:52:15.567208+0000","last_peered":"2024-09-21T02:52:15.567208+0000","last_clean":"2024-09-21T02:52:15.567208+0000","last_became_active":"2024-09-21T02:50:50.368303+0000","last_became_peered":"2024-09-21T02:50:50.368303+0000","last_unstale":"2024-09-21T02:52:15.567208+0000","last_undegraded":"2024-09-21T02:52:15.567208+0000","last_fullsized":"2024-09-21T02:52:15.567208+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:22:11.862324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552898+0000","last_change":"2024-09-21T02:51:16.576316+0000","last_active":"2024-09-21T02:52:15.552898+0000","last_peered":"2024-09-21T02:52:15.552898+0000","last_clean":"2024-09-21T02:52:15.552898+0000","last_became_active":"2024-09-21T02:51:16.575827+0000","last_became_peered":"2024-09-21T02:51:16.575827+0000","last_unstale":"2024-09-21T02:52:15.552898+0000","last_undegraded":"2024-09-21T02:52:15.552898+0000","last_fullsized":"2024-09-21T02:52:15.552898+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:26.887428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.554532+0000","last_change":"2024-09-21T02:50:50.356073+0000","last_active":"2024-09-21T02:52:15.554532+0000","last_peered":"2024-09-21T02:52:15.554532+0000","last_clean":"2024-09-21T02:52:15.554532+0000","last_became_active":"2024-09-21T02:50:50.354365+0000","last_became_peered":"2024-09-21T02:50:50.354365+0000","last_unstale":"2024-09-21T02:52:15.554532+0000","last_undegraded":"2024-09-21T02:52:15.554532+0000","last_fullsized":"2024-09-21T02:52:15.554532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:51.100378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563207+0000","last_change":"2024-09-21T02:50:50.356934+0000","last_active":"2024-09-21T02:52:15.563207+0000","last_peered":"2024-09-21T02:52:15.563207+0000","last_clean":"2024-09-21T02:52:15.563207+0000","last_became_active":"2024-09-21T02:50:50.356564+0000","last_became_peered":"2024-09-21T02:50:50.356564+0000","last_unstale":"2024-09-21T02:52:15.563207+0000","last_undegraded":"2024-09-21T02:52:15.563207+0000","last_fullsized":"2024-09-21T02:52:15.563207+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:06.767991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563556+0000","last_change":"2024-09-21T02:50:50.365447+0000","last_active":"2024-09-21T02:52:15.563556+0000","last_peered":"2024-09-21T02:52:15.563556+0000","last_clean":"2024-09-21T02:52:15.563556+0000","last_became_active":"2024-09-21T02:50:50.365032+0000","last_became_peered":"2024-09-21T02:50:50.365032+0000","last_unstale":"2024-09-21T02:52:15.563556+0000","last_undegraded":"2024-09-21T02:52:15.563556+0000","last_fullsized":"2024-09-21T02:52:15.563556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:16.364089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.568347+0000","last_change":"2024-09-21T02:50:50.367512+0000","last_active":"2024-09-21T02:52:15.568347+0000","last_peered":"2024-09-21T02:52:15.568347+0000","last_clean":"2024-09-21T02:52:15.568347+0000","last_became_active":"2024-09-21T02:50:50.365318+0000","last_became_peered":"2024-09-21T02:50:50.365318+0000","last_unstale":"2024-09-21T02:52:15.568347+0000","last_undegraded":"2024-09-21T02:52:15.568347+0000","last_fullsized":"2024-09-21T02:52:15.568347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:32.333726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567564+0000","last_change":"2024-09-21T02:50:50.367950+0000","last_active":"2024-09-21T02:52:15.567564+0000","last_peered":"2024-09-21T02:52:15.567564+0000","last_clean":"2024-09-21T02:52:15.567564+0000","last_became_active":"2024-09-21T02:50:50.367601+0000","last_became_peered":"2024-09-21T02:50:50.367601+0000","last_unstale":"2024-09-21T02:52:15.567564+0000","last_undegraded":"2024-09-21T02:52:15.567564+0000","last_fullsized":"2024-09-21T02:52:15.567564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:37.078930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557763+0000","last_change":"2024-09-21T02:50:50.361156+0000","last_active":"2024-09-21T02:52:15.557763+0000","last_peered":"2024-09-21T02:52:15.557763+0000","last_clean":"2024-09-21T02:52:15.557763+0000","last_became_active":"2024-09-21T02:50:50.360628+0000","last_became_peered":"2024-09-21T02:50:50.360628+0000","last_unstale":"2024-09-21T02:52:15.557763+0000","last_undegraded":"2024-09-21T02:52:15.557763+0000","last_fullsized":"2024-09-21T02:52:15.557763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:22.743419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552086+0000","last_change":"2024-09-21T02:50:50.347309+0000","last_active":"2024-09-21T02:52:15.552086+0000","last_peered":"2024-09-21T02:52:15.552086+0000","last_clean":"2024-09-21T02:52:15.552086+0000","last_became_active":"2024-09-21T02:50:50.346870+0000","last_became_peered":"2024-09-21T02:50:50.346870+0000","last_unstale":"2024-09-21T02:52:15.552086+0000","last_undegraded":"2024-09-21T02:52:15.552086+0000","last_fullsized":"2024-09-21T02:52:15.552086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.684304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558416+0000","last_change":"2024-09-21T02:50:50.363570+0000","last_active":"2024-09-21T02:52:15.558416+0000","last_peered":"2024-09-21T02:52:15.558416+0000","last_clean":"2024-09-21T02:52:15.558416+0000","last_became_active":"2024-09-21T02:50:50.363181+0000","last_became_peered":"2024-09-21T02:50:50.363181+0000","last_unstale":"2024-09-21T02:52:15.558416+0000","last_undegraded":"2024-09-21T02:52:15.558416+0000","last_fullsized":"2024-09-21T02:52:15.558416+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:49.920490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560852+0000","last_change":"2024-09-21T02:50:50.355735+0000","last_active":"2024-09-21T02:52:15.560852+0000","last_peered":"2024-09-21T02:52:15.560852+0000","last_clean":"2024-09-21T02:52:15.560852+0000","last_became_active":"2024-09-21T02:50:50.355362+0000","last_became_peered":"2024-09-21T02:50:50.355362+0000","last_unstale":"2024-09-21T02:52:15.560852+0000","last_undegraded":"2024-09-21T02:52:15.560852+0000","last_fullsized":"2024-09-21T02:52:15.560852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:23.624912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.555555+0000","last_change":"2024-09-21T02:50:50.363662+0000","last_active":"2024-09-21T02:52:15.555555+0000","last_peered":"2024-09-21T02:52:15.555555+0000","last_clean":"2024-09-21T02:52:15.555555+0000","last_became_active":"2024-09-21T02:50:50.363315+0000","last_became_peered":"2024-09-21T02:50:50.363315+0000","last_unstale":"2024-09-21T02:52:15.555555+0000","last_undegraded":"2024-09-21T02:52:15.555555+0000","last_fullsized":"2024-09-21T02:52:15.555555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:10.585328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557760+0000","last_change":"2024-09-21T02:50:50.346239+0000","last_active":"2024-09-21T02:52:15.557760+0000","last_peered":"2024-09-21T02:52:15.557760+0000","last_clean":"2024-09-21T02:52:15.557760+0000","last_became_active":"2024-09-21T02:50:50.345867+0000","last_became_peered":"2024-09-21T02:50:50.345867+0000","last_unstale":"2024-09-21T02:52:15.557760+0000","last_undegraded":"2024-09-21T02:52:15.557760+0000","last_fullsized":"2024-09-21T02:52:15.557760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:35.840599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"78'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.548121+0000","last_change":"2024-09-21T02:51:16.977279+0000","last_active":"2024-09-21T02:52:15.548121+0000","last_peered":"2024-09-21T02:52:15.548121+0000","last_clean":"2024-09-21T02:52:15.548121+0000","last_became_active":"2024-09-21T02:51:16.976841+0000","last_became_peered":"2024-09-21T02:51:16.976841+0000","last_unstale":"2024-09-21T02:52:15.548121+0000","last_undegraded":"2024-09-21T02:52:15.548121+0000","last_fullsized":"2024-09-21T02:52:15.548121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:07.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.549005+0000","last_change":"2024-09-21T02:50:50.364091+0000","last_active":"2024-09-21T02:52:15.549005+0000","last_peered":"2024-09-21T02:52:15.549005+0000","last_clean":"2024-09-21T02:52:15.549005+0000","last_became_active":"2024-09-21T02:50:50.363726+0000","last_became_peered":"2024-09-21T02:50:50.363726+0000","last_unstale":"2024-09-21T02:52:15.549005+0000","last_undegraded":"2024-09-21T02:52:15.549005+0000","last_fullsized":"2024-09-21T02:52:15.549005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:39.210489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.565080+0000","last_change":"2024-09-21T02:50:50.360727+0000","last_active":"2024-09-21T02:52:15.565080+0000","last_peered":"2024-09-21T02:52:15.565080+0000","last_clean":"2024-09-21T02:52:15.565080+0000","last_became_active":"2024-09-21T02:50:50.360341+0000","last_became_peered":"2024-09-21T02:50:50.360341+0000","last_unstale":"2024-09-21T02:52:15.565080+0000","last_undegraded":"2024-09-21T02:52:15.565080+0000","last_fullsized":"2024-09-21T02:52:15.565080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:25.591949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.554835+0000","last_change":"2024-09-21T02:50:50.360624+0000","last_active":"2024-09-21T02:52:15.554835+0000","last_peered":"2024-09-21T02:52:15.554835+0000","last_clean":"2024-09-21T02:52:15.554835+0000","last_became_active":"2024-09-21T02:50:50.360203+0000","last_became_peered":"2024-09-21T02:50:50.360203+0000","last_unstale":"2024-09-21T02:52:15.554835+0000","last_undegraded":"2024-09-21T02:52:15.554835+0000","last_fullsized":"2024-09-21T02:52:15.554835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:52.223875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560546+0000","last_change":"2024-09-21T02:50:50.350800+0000","last_active":"2024-09-21T02:52:15.560546+0000","last_peered":"2024-09-21T02:52:15.560546+0000","last_clean":"2024-09-21T02:52:15.560546+0000","last_became_active":"2024-09-21T02:50:50.350419+0000","last_became_peered":"2024-09-21T02:50:50.350419+0000","last_unstale":"2024-09-21T02:52:15.560546+0000","last_undegraded":"2024-09-21T02:52:15.560546+0000","last_fullsized":"2024-09-21T02:52:15.560546+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:41.669499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.571336+0000","last_change":"2024-09-21T02:51:16.580720+0000","last_active":"2024-09-21T02:52:15.571336+0000","last_peered":"2024-09-21T02:52:15.571336+0000","last_clean":"2024-09-21T02:52:15.571336+0000","last_became_active":"2024-09-21T02:51:16.580267+0000","last_became_peered":"2024-09-21T02:51:16.580267+0000","last_unstale":"2024-09-21T02:52:15.571336+0000","last_undegraded":"2024-09-21T02:52:15.571336+0000","last_fullsized":"2024-09-21T02:52:15.571336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.184508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558911+0000","last_change":"2024-09-21T02:50:50.350789+0000","last_active":"2024-09-21T02:52:15.558911+0000","last_peered":"2024-09-21T02:52:15.558911+0000","last_clean":"2024-09-21T02:52:15.558911+0000","last_became_active":"2024-09-21T02:50:50.350478+0000","last_became_peered":"2024-09-21T02:50:50.350478+0000","last_unstale":"2024-09-21T02:52:15.558911+0000","last_undegraded":"2024-09-21T02:52:15.558911+0000","last_fullsized":"2024-09-21T02:52:15.558911+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:18.738903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"78'12","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.555496+0000","last_change":"2024-09-21T02:50:50.365231+0000","last_active":"2024-09-21T02:52:15.555496+0000","last_peered":"2024-09-21T02:52:15.555496+0000","last_clean":"2024-09-21T02:52:15.555496+0000","last_became_active":"2024-09-21T02:50:50.364849+0000","last_became_peered":"2024-09-21T02:50:50.364849+0000","last_unstale":"2024-09-21T02:52:15.555496+0000","last_undegraded":"2024-09-21T02:52:15.555496+0000","last_fullsized":"2024-09-21T02:52:15.555496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:51.721437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.557445+0000","last_change":"2024-09-21T02:50:50.358072+0000","last_active":"2024-09-21T02:52:15.557445+0000","last_peered":"2024-09-21T02:52:15.557445+0000","last_clean":"2024-09-21T02:52:15.557445+0000","last_became_active":"2024-09-21T02:50:50.357694+0000","last_became_peered":"2024-09-21T02:50:50.357694+0000","last_unstale":"2024-09-21T02:52:15.557445+0000","last_undegraded":"2024-09-21T02:52:15.557445+0000","last_fullsized":"2024-09-21T02:52:15.557445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:03.675140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.544310+0000","last_change":"2024-09-21T02:50:50.354098+0000","last_active":"2024-09-21T02:52:15.544310+0000","last_peered":"2024-09-21T02:52:15.544310+0000","last_clean":"2024-09-21T02:52:15.544310+0000","last_became_active":"2024-09-21T02:50:50.353484+0000","last_became_peered":"2024-09-21T02:50:50.353484+0000","last_unstale":"2024-09-21T02:52:15.544310+0000","last_undegraded":"2024-09-21T02:52:15.544310+0000","last_fullsized":"2024-09-21T02:52:15.544310+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:51.717565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'7","reported_seq":165,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.733243+0000","last_change":"2024-09-21T02:51:16.585939+0000","last_active":"2024-09-21T02:52:57.733243+0000","last_peered":"2024-09-21T02:52:57.733243+0000","last_clean":"2024-09-21T02:52:57.733243+0000","last_became_active":"2024-09-21T02:51:16.580922+0000","last_became_peered":"2024-09-21T02:51:16.580922+0000","last_unstale":"2024-09-21T02:52:57.733243+0000","last_undegraded":"2024-09-21T02:52:57.733243+0000","last_fullsized":"2024-09-21T02:52:57.733243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:40.110230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.551262+0000","last_change":"2024-09-21T02:50:50.363132+0000","last_active":"2024-09-21T02:52:15.551262+0000","last_peered":"2024-09-21T02:52:15.551262+0000","last_clean":"2024-09-21T02:52:15.551262+0000","last_became_active":"2024-09-21T02:50:50.362402+0000","last_became_peered":"2024-09-21T02:50:50.362402+0000","last_unstale":"2024-09-21T02:52:15.551262+0000","last_undegraded":"2024-09-21T02:52:15.551262+0000","last_fullsized":"2024-09-21T02:52:15.551262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:54.353153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"76'7","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574073+0000","last_change":"2024-09-21T02:50:48.334356+0000","last_active":"2024-09-21T02:51:16.574073+0000","last_peered":"2024-09-21T02:51:16.574073+0000","last_clean":"2024-09-21T02:51:16.574073+0000","last_became_active":"2024-09-21T02:50:48.333945+0000","last_became_peered":"2024-09-21T02:50:48.333945+0000","last_unstale":"2024-09-21T02:51:16.574073+0000","last_undegraded":"2024-09-21T02:51:16.574073+0000","last_fullsized":"2024-09-21T02:51:16.574073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:33.588053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"76'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573143+0000","last_change":"2024-09-21T02:50:48.330293+0000","last_active":"2024-09-21T02:51:16.573143+0000","last_peered":"2024-09-21T02:51:16.573143+0000","last_clean":"2024-09-21T02:51:16.573143+0000","last_became_active":"2024-09-21T02:50:48.329904+0000","last_became_peered":"2024-09-21T02:50:48.329904+0000","last_unstale":"2024-09-21T02:51:16.573143+0000","last_undegraded":"2024-09-21T02:51:16.573143+0000","last_fullsized":"2024-09-21T02:51:16.573143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:35.322161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.556254+0000","last_change":"2024-09-21T02:50:50.360049+0000","last_active":"2024-09-21T02:52:15.556254+0000","last_peered":"2024-09-21T02:52:15.556254+0000","last_clean":"2024-09-21T02:52:15.556254+0000","last_became_active":"2024-09-21T02:50:50.359695+0000","last_became_peered":"2024-09-21T02:50:50.359695+0000","last_unstale":"2024-09-21T02:52:15.556254+0000","last_undegraded":"2024-09-21T02:52:15.556254+0000","last_fullsized":"2024-09-21T02:52:15.556254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:38.737573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578822+0000","last_change":"2024-09-21T02:50:48.336668+0000","last_active":"2024-09-21T02:51:16.578822+0000","last_peered":"2024-09-21T02:51:16.578822+0000","last_clean":"2024-09-21T02:51:16.578822+0000","last_became_active":"2024-09-21T02:50:48.335803+0000","last_became_peered":"2024-09-21T02:50:48.335803+0000","last_unstale":"2024-09-21T02:51:16.578822+0000","last_undegraded":"2024-09-21T02:51:16.578822+0000","last_fullsized":"2024-09-21T02:51:16.578822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:55.890314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.551455+0000","last_change":"2024-09-21T02:50:50.352908+0000","last_active":"2024-09-21T02:52:15.551455+0000","last_peered":"2024-09-21T02:52:15.551455+0000","last_clean":"2024-09-21T02:52:15.551455+0000","last_became_active":"2024-09-21T02:50:50.352521+0000","last_became_peered":"2024-09-21T02:50:50.352521+0000","last_unstale":"2024-09-21T02:52:15.551455+0000","last_undegraded":"2024-09-21T02:52:15.551455+0000","last_fullsized":"2024-09-21T02:52:15.551455+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:17.118795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'6","reported_seq":261,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:56.700068+0000","last_change":"2024-09-21T02:50:48.335201+0000","last_active":"2024-09-21T02:52:56.700068+0000","last_peered":"2024-09-21T02:52:56.700068+0000","last_clean":"2024-09-21T02:52:56.700068+0000","last_became_active":"2024-09-21T02:50:48.334864+0000","last_became_peered":"2024-09-21T02:50:48.334864+0000","last_unstale":"2024-09-21T02:52:56.700068+0000","last_undegraded":"2024-09-21T02:52:56.700068+0000","last_fullsized":"2024-09-21T02:52:56.700068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:05.700121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563004+0000","last_change":"2024-09-21T02:50:50.358951+0000","last_active":"2024-09-21T02:52:15.563004+0000","last_peered":"2024-09-21T02:52:15.563004+0000","last_clean":"2024-09-21T02:52:15.563004+0000","last_became_active":"2024-09-21T02:50:50.358506+0000","last_became_peered":"2024-09-21T02:50:50.358506+0000","last_unstale":"2024-09-21T02:52:15.563004+0000","last_undegraded":"2024-09-21T02:52:15.563004+0000","last_fullsized":"2024-09-21T02:52:15.563004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.059251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"76'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576384+0000","last_change":"2024-09-21T02:50:48.335713+0000","last_active":"2024-09-21T02:51:16.576384+0000","last_peered":"2024-09-21T02:51:16.576384+0000","last_clean":"2024-09-21T02:51:16.576384+0000","last_became_active":"2024-09-21T02:50:48.335300+0000","last_became_peered":"2024-09-21T02:50:48.335300+0000","last_unstale":"2024-09-21T02:51:16.576384+0000","last_undegraded":"2024-09-21T02:51:16.576384+0000","last_fullsized":"2024-09-21T02:51:16.576384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:10.627528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564496+0000","last_change":"2024-09-21T02:50:50.364373+0000","last_active":"2024-09-21T02:52:15.564496+0000","last_peered":"2024-09-21T02:52:15.564496+0000","last_clean":"2024-09-21T02:52:15.564496+0000","last_became_active":"2024-09-21T02:50:50.363955+0000","last_became_peered":"2024-09-21T02:50:50.363955+0000","last_unstale":"2024-09-21T02:52:15.564496+0000","last_undegraded":"2024-09-21T02:52:15.564496+0000","last_fullsized":"2024-09-21T02:52:15.564496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:43.784871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578199+0000","last_change":"2024-09-21T02:50:48.629373+0000","last_active":"2024-09-21T02:51:16.578199+0000","last_peered":"2024-09-21T02:51:16.578199+0000","last_clean":"2024-09-21T02:51:16.578199+0000","last_became_active":"2024-09-21T02:50:48.628905+0000","last_became_peered":"2024-09-21T02:50:48.628905+0000","last_unstale":"2024-09-21T02:51:16.578199+0000","last_undegraded":"2024-09-21T02:51:16.578199+0000","last_fullsized":"2024-09-21T02:51:16.578199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.777520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.565141+0000","last_change":"2024-09-21T02:50:50.366580+0000","last_active":"2024-09-21T02:52:15.565141+0000","last_peered":"2024-09-21T02:52:15.565141+0000","last_clean":"2024-09-21T02:52:15.565141+0000","last_became_active":"2024-09-21T02:50:50.365930+0000","last_became_peered":"2024-09-21T02:50:50.365930+0000","last_unstale":"2024-09-21T02:52:15.565141+0000","last_undegraded":"2024-09-21T02:52:15.565141+0000","last_fullsized":"2024-09-21T02:52:15.565141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:27.291199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'33","reported_seq":297,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:56.731591+0000","last_change":"2024-09-21T02:50:48.336852+0000","last_active":"2024-09-21T02:52:56.731591+0000","last_peered":"2024-09-21T02:52:56.731591+0000","last_clean":"2024-09-21T02:52:56.731591+0000","last_became_active":"2024-09-21T02:50:48.336499+0000","last_became_peered":"2024-09-21T02:50:48.336499+0000","last_unstale":"2024-09-21T02:52:56.731591+0000","last_undegraded":"2024-09-21T02:52:56.731591+0000","last_fullsized":"2024-09-21T02:52:56.731591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:14.740089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":182,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.550696+0000","last_change":"2024-09-21T02:50:50.360263+0000","last_active":"2024-09-21T02:52:15.550696+0000","last_peered":"2024-09-21T02:52:15.550696+0000","last_clean":"2024-09-21T02:52:15.550696+0000","last_became_active":"2024-09-21T02:50:50.359873+0000","last_became_peered":"2024-09-21T02:50:50.359873+0000","last_unstale":"2024-09-21T02:52:15.550696+0000","last_undegraded":"2024-09-21T02:52:15.550696+0000","last_fullsized":"2024-09-21T02:52:15.550696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:07.812790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"76'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576487+0000","last_change":"2024-09-21T02:50:48.332908+0000","last_active":"2024-09-21T02:51:16.576487+0000","last_peered":"2024-09-21T02:51:16.576487+0000","last_clean":"2024-09-21T02:51:16.576487+0000","last_became_active":"2024-09-21T02:50:48.332014+0000","last_became_peered":"2024-09-21T02:50:48.332014+0000","last_unstale":"2024-09-21T02:51:16.576487+0000","last_undegraded":"2024-09-21T02:51:16.576487+0000","last_fullsized":"2024-09-21T02:51:16.576487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:03.530874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552036+0000","last_change":"2024-09-21T02:51:16.578471+0000","last_active":"2024-09-21T02:52:15.552036+0000","last_peered":"2024-09-21T02:52:15.552036+0000","last_clean":"2024-09-21T02:52:15.552036+0000","last_became_active":"2024-09-21T02:51:16.578146+0000","last_became_peered":"2024-09-21T02:51:16.578146+0000","last_unstale":"2024-09-21T02:52:15.552036+0000","last_undegraded":"2024-09-21T02:52:15.552036+0000","last_fullsized":"2024-09-21T02:52:15.552036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:56.400224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.577792+0000","last_change":"2024-09-21T02:50:48.337895+0000","last_active":"2024-09-21T02:51:16.577792+0000","last_peered":"2024-09-21T02:51:16.577792+0000","last_clean":"2024-09-21T02:51:16.577792+0000","last_became_active":"2024-09-21T02:50:48.337435+0000","last_became_peered":"2024-09-21T02:50:48.337435+0000","last_unstale":"2024-09-21T02:51:16.577792+0000","last_undegraded":"2024-09-21T02:51:16.577792+0000","last_fullsized":"2024-09-21T02:51:16.577792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:34.465731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567369+0000","last_change":"2024-09-21T02:50:50.363858+0000","last_active":"2024-09-21T02:52:15.567369+0000","last_peered":"2024-09-21T02:52:15.567369+0000","last_clean":"2024-09-21T02:52:15.567369+0000","last_became_active":"2024-09-21T02:50:50.363265+0000","last_became_peered":"2024-09-21T02:50:50.363265+0000","last_unstale":"2024-09-21T02:52:15.567369+0000","last_undegraded":"2024-09-21T02:52:15.567369+0000","last_fullsized":"2024-09-21T02:52:15.567369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:25.543352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.712619+0000","last_change":"2024-09-21T02:50:48.340219+0000","last_active":"2024-09-21T02:52:57.712619+0000","last_peered":"2024-09-21T02:52:57.712619+0000","last_clean":"2024-09-21T02:52:57.712619+0000","last_became_active":"2024-09-21T02:50:48.339710+0000","last_became_peered":"2024-09-21T02:50:48.339710+0000","last_unstale":"2024-09-21T02:52:57.712619+0000","last_undegraded":"2024-09-21T02:52:57.712619+0000","last_fullsized":"2024-09-21T02:52:57.712619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:24.881682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"78'6","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569073+0000","last_change":"2024-09-21T02:51:16.580709+0000","last_active":"2024-09-21T02:52:15.569073+0000","last_peered":"2024-09-21T02:52:15.569073+0000","last_clean":"2024-09-21T02:52:15.569073+0000","last_became_active":"2024-09-21T02:51:16.580164+0000","last_became_peered":"2024-09-21T02:51:16.580164+0000","last_unstale":"2024-09-21T02:52:15.569073+0000","last_undegraded":"2024-09-21T02:52:15.569073+0000","last_fullsized":"2024-09-21T02:52:15.569073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"76'17","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571921+0000","last_change":"2024-09-21T02:50:48.626918+0000","last_active":"2024-09-21T02:51:16.571921+0000","last_peered":"2024-09-21T02:51:16.571921+0000","last_clean":"2024-09-21T02:51:16.571921+0000","last_became_active":"2024-09-21T02:50:48.626417+0000","last_became_peered":"2024-09-21T02:50:48.626417+0000","last_unstale":"2024-09-21T02:51:16.571921+0000","last_undegraded":"2024-09-21T02:51:16.571921+0000","last_fullsized":"2024-09-21T02:51:16.571921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.229272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"78'6","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.566458+0000","last_change":"2024-09-21T02:50:50.366407+0000","last_active":"2024-09-21T02:52:15.566458+0000","last_peered":"2024-09-21T02:52:15.566458+0000","last_clean":"2024-09-21T02:52:15.566458+0000","last_became_active":"2024-09-21T02:50:50.366068+0000","last_became_peered":"2024-09-21T02:50:50.366068+0000","last_unstale":"2024-09-21T02:52:15.566458+0000","last_undegraded":"2024-09-21T02:52:15.566458+0000","last_fullsized":"2024-09-21T02:52:15.566458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:31.330753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572694+0000","last_change":"2024-09-21T02:50:48.332371+0000","last_active":"2024-09-21T02:51:16.572694+0000","last_peered":"2024-09-21T02:51:16.572694+0000","last_clean":"2024-09-21T02:51:16.572694+0000","last_became_active":"2024-09-21T02:50:48.331975+0000","last_became_peered":"2024-09-21T02:50:48.331975+0000","last_unstale":"2024-09-21T02:51:16.572694+0000","last_undegraded":"2024-09-21T02:51:16.572694+0000","last_fullsized":"2024-09-21T02:51:16.572694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:58.419780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.558056+0000","last_change":"2024-09-21T02:50:50.364292+0000","last_active":"2024-09-21T02:52:15.558056+0000","last_peered":"2024-09-21T02:52:15.558056+0000","last_clean":"2024-09-21T02:52:15.558056+0000","last_became_active":"2024-09-21T02:50:50.363899+0000","last_became_peered":"2024-09-21T02:50:50.363899+0000","last_unstale":"2024-09-21T02:52:15.558056+0000","last_undegraded":"2024-09-21T02:52:15.558056+0000","last_fullsized":"2024-09-21T02:52:15.558056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:17.426581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"76'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574109+0000","last_change":"2024-09-21T02:50:48.330673+0000","last_active":"2024-09-21T02:51:16.574109+0000","last_peered":"2024-09-21T02:51:16.574109+0000","last_clean":"2024-09-21T02:51:16.574109+0000","last_became_active":"2024-09-21T02:50:48.330314+0000","last_became_peered":"2024-09-21T02:50:48.330314+0000","last_unstale":"2024-09-21T02:51:16.574109+0000","last_undegraded":"2024-09-21T02:51:16.574109+0000","last_fullsized":"2024-09-21T02:51:16.574109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:03.558250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.568405+0000","last_change":"2024-09-21T02:50:50.359111+0000","last_active":"2024-09-21T02:52:15.568405+0000","last_peered":"2024-09-21T02:52:15.568405+0000","last_clean":"2024-09-21T02:52:15.568405+0000","last_became_active":"2024-09-21T02:50:50.358743+0000","last_became_peered":"2024-09-21T02:50:50.358743+0000","last_unstale":"2024-09-21T02:52:15.568405+0000","last_undegraded":"2024-09-21T02:52:15.568405+0000","last_fullsized":"2024-09-21T02:52:15.568405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:08:26.849606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'3","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.578249+0000","last_change":"2024-09-21T02:50:48.337671+0000","last_active":"2024-09-21T02:51:16.578249+0000","last_peered":"2024-09-21T02:51:16.578249+0000","last_clean":"2024-09-21T02:51:16.578249+0000","last_became_active":"2024-09-21T02:50:48.336655+0000","last_became_peered":"2024-09-21T02:50:48.336655+0000","last_unstale":"2024-09-21T02:51:16.578249+0000","last_undegraded":"2024-09-21T02:51:16.578249+0000","last_fullsized":"2024-09-21T02:51:16.578249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:36.263509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570341+0000","last_change":"2024-09-21T02:50:50.362045+0000","last_active":"2024-09-21T02:52:15.570341+0000","last_peered":"2024-09-21T02:52:15.570341+0000","last_clean":"2024-09-21T02:52:15.570341+0000","last_became_active":"2024-09-21T02:50:50.361672+0000","last_became_peered":"2024-09-21T02:50:50.361672+0000","last_unstale":"2024-09-21T02:52:15.570341+0000","last_undegraded":"2024-09-21T02:52:15.570341+0000","last_fullsized":"2024-09-21T02:52:15.570341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:22.843260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'8","reported_seq":152,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:57.702109+0000","last_change":"2024-09-21T02:50:48.629477+0000","last_active":"2024-09-21T02:52:57.702109+0000","last_peered":"2024-09-21T02:52:57.702109+0000","last_clean":"2024-09-21T02:52:57.702109+0000","last_became_active":"2024-09-21T02:50:48.629019+0000","last_became_peered":"2024-09-21T02:50:48.629019+0000","last_unstale":"2024-09-21T02:52:57.702109+0000","last_undegraded":"2024-09-21T02:52:57.702109+0000","last_fullsized":"2024-09-21T02:52:57.702109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"78'7","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.569660+0000","last_change":"2024-09-21T02:50:50.368843+0000","last_active":"2024-09-21T02:52:15.569660+0000","last_peered":"2024-09-21T02:52:15.569660+0000","last_clean":"2024-09-21T02:52:15.569660+0000","last_became_active":"2024-09-21T02:50:50.368497+0000","last_became_peered":"2024-09-21T02:50:50.368497+0000","last_unstale":"2024-09-21T02:52:15.569660+0000","last_undegraded":"2024-09-21T02:52:15.569660+0000","last_fullsized":"2024-09-21T02:52:15.569660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:26.625814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":98,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576900+0000","last_change":"2024-09-21T02:49:38.687440+0000","last_active":"2024-09-21T02:51:16.576900+0000","last_peered":"2024-09-21T02:51:16.576900+0000","last_clean":"2024-09-21T02:51:16.576900+0000","last_became_active":"2024-09-21T02:49:38.681746+0000","last_became_peered":"2024-09-21T02:49:38.681746+0000","last_unstale":"2024-09-21T02:51:16.576900+0000","last_undegraded":"2024-09-21T02:51:16.576900+0000","last_fullsized":"2024-09-21T02:51:16.576900+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:46:13.654366+0000","last_clean_scrub_stamp":"2024-09-21T02:46:13.654366+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.675757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'6","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.573515+0000","last_change":"2024-09-21T02:50:48.331714+0000","last_active":"2024-09-21T02:51:16.573515+0000","last_peered":"2024-09-21T02:51:16.573515+0000","last_clean":"2024-09-21T02:51:16.573515+0000","last_became_active":"2024-09-21T02:50:48.331240+0000","last_became_peered":"2024-09-21T02:50:48.331240+0000","last_unstale":"2024-09-21T02:51:16.573515+0000","last_undegraded":"2024-09-21T02:51:16.573515+0000","last_fullsized":"2024-09-21T02:51:16.573515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:15.157110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564758+0000","last_change":"2024-09-21T02:50:50.360748+0000","last_active":"2024-09-21T02:52:15.564758+0000","last_peered":"2024-09-21T02:52:15.564758+0000","last_clean":"2024-09-21T02:52:15.564758+0000","last_became_active":"2024-09-21T02:50:50.360394+0000","last_became_peered":"2024-09-21T02:50:50.360394+0000","last_unstale":"2024-09-21T02:52:15.564758+0000","last_undegraded":"2024-09-21T02:52:15.564758+0000","last_fullsized":"2024-09-21T02:52:15.564758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:22.697508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.576845+0000","last_change":"2024-09-21T02:50:48.336844+0000","last_active":"2024-09-21T02:51:16.576845+0000","last_peered":"2024-09-21T02:51:16.576845+0000","last_clean":"2024-09-21T02:51:16.576845+0000","last_became_active":"2024-09-21T02:50:48.336403+0000","last_became_peered":"2024-09-21T02:50:48.336403+0000","last_unstale":"2024-09-21T02:51:16.576845+0000","last_undegraded":"2024-09-21T02:51:16.576845+0000","last_fullsized":"2024-09-21T02:51:16.576845+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:07.531565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553787+0000","last_change":"2024-09-21T02:50:50.364109+0000","last_active":"2024-09-21T02:52:15.553787+0000","last_peered":"2024-09-21T02:52:15.553787+0000","last_clean":"2024-09-21T02:52:15.553787+0000","last_became_active":"2024-09-21T02:50:50.363754+0000","last_became_peered":"2024-09-21T02:50:50.363754+0000","last_unstale":"2024-09-21T02:52:15.553787+0000","last_undegraded":"2024-09-21T02:52:15.553787+0000","last_fullsized":"2024-09-21T02:52:15.553787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:37.217017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'6","reported_seq":151,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:59.715214+0000","last_change":"2024-09-21T02:50:48.627332+0000","last_active":"2024-09-21T02:52:59.715214+0000","last_peered":"2024-09-21T02:52:59.715214+0000","last_clean":"2024-09-21T02:52:59.715214+0000","last_became_active":"2024-09-21T02:50:48.626928+0000","last_became_peered":"2024-09-21T02:50:48.626928+0000","last_unstale":"2024-09-21T02:52:59.715214+0000","last_undegraded":"2024-09-21T02:52:59.715214+0000","last_fullsized":"2024-09-21T02:52:59.715214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:20.110642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"78'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.556144+0000","last_change":"2024-09-21T02:50:50.366917+0000","last_active":"2024-09-21T02:52:15.556144+0000","last_peered":"2024-09-21T02:52:15.556144+0000","last_clean":"2024-09-21T02:52:15.556144+0000","last_became_active":"2024-09-21T02:50:50.365432+0000","last_became_peered":"2024-09-21T02:50:50.365432+0000","last_unstale":"2024-09-21T02:52:15.556144+0000","last_undegraded":"2024-09-21T02:52:15.556144+0000","last_fullsized":"2024-09-21T02:52:15.556144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:09.490062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'8","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:58.703005+0000","last_change":"2024-09-21T02:50:48.335799+0000","last_active":"2024-09-21T02:52:58.703005+0000","last_peered":"2024-09-21T02:52:58.703005+0000","last_clean":"2024-09-21T02:52:58.703005+0000","last_became_active":"2024-09-21T02:50:48.334662+0000","last_became_peered":"2024-09-21T02:50:48.334662+0000","last_unstale":"2024-09-21T02:52:58.703005+0000","last_undegraded":"2024-09-21T02:52:58.703005+0000","last_fullsized":"2024-09-21T02:52:58.703005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:53.874330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.562844+0000","last_change":"2024-09-21T02:50:50.364047+0000","last_active":"2024-09-21T02:52:15.562844+0000","last_peered":"2024-09-21T02:52:15.562844+0000","last_clean":"2024-09-21T02:52:15.562844+0000","last_became_active":"2024-09-21T02:50:50.363665+0000","last_became_peered":"2024-09-21T02:50:50.363665+0000","last_unstale":"2024-09-21T02:52:15.562844+0000","last_undegraded":"2024-09-21T02:52:15.562844+0000","last_fullsized":"2024-09-21T02:52:15.562844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:17.572618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"76'1","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.571605+0000","last_change":"2024-09-21T02:50:48.628790+0000","last_active":"2024-09-21T02:51:16.571605+0000","last_peered":"2024-09-21T02:51:16.571605+0000","last_clean":"2024-09-21T02:51:16.571605+0000","last_became_active":"2024-09-21T02:50:48.628368+0000","last_became_peered":"2024-09-21T02:50:48.628368+0000","last_unstale":"2024-09-21T02:51:16.571605+0000","last_undegraded":"2024-09-21T02:51:16.571605+0000","last_fullsized":"2024-09-21T02:51:16.571605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:51.658138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.549381+0000","last_change":"2024-09-21T02:50:50.347696+0000","last_active":"2024-09-21T02:52:15.549381+0000","last_peered":"2024-09-21T02:52:15.549381+0000","last_clean":"2024-09-21T02:52:15.549381+0000","last_became_active":"2024-09-21T02:50:50.347184+0000","last_became_peered":"2024-09-21T02:50:50.347184+0000","last_unstale":"2024-09-21T02:52:15.549381+0000","last_undegraded":"2024-09-21T02:52:15.549381+0000","last_fullsized":"2024-09-21T02:52:15.549381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:03.131086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"76'4","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.587386+0000","last_change":"2024-09-21T02:51:16.586836+0000","last_active":"2024-09-21T02:51:16.587386+0000","last_peered":"2024-09-21T02:51:16.587386+0000","last_clean":"2024-09-21T02:51:16.587386+0000","last_became_active":"2024-09-21T02:51:16.578438+0000","last_became_peered":"2024-09-21T02:51:16.578438+0000","last_unstale":"2024-09-21T02:51:16.587386+0000","last_undegraded":"2024-09-21T02:51:16.587386+0000","last_fullsized":"2024-09-21T02:51:16.587386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:39.391978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553921+0000","last_change":"2024-09-21T02:50:50.346060+0000","last_active":"2024-09-21T02:52:15.553921+0000","last_peered":"2024-09-21T02:52:15.553921+0000","last_clean":"2024-09-21T02:52:15.553921+0000","last_became_active":"2024-09-21T02:50:50.345699+0000","last_became_peered":"2024-09-21T02:50:50.345699+0000","last_unstale":"2024-09-21T02:52:15.553921+0000","last_undegraded":"2024-09-21T02:52:15.553921+0000","last_fullsized":"2024-09-21T02:52:15.553921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:08.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:18.365997+0000","last_change":"2024-09-21T02:50:48.332618+0000","last_active":"2024-09-21T02:51:18.365997+0000","last_peered":"2024-09-21T02:51:18.365997+0000","last_clean":"2024-09-21T02:51:18.365997+0000","last_became_active":"2024-09-21T02:50:48.331864+0000","last_became_peered":"2024-09-21T02:50:48.331864+0000","last_unstale":"2024-09-21T02:51:18.365997+0000","last_undegraded":"2024-09-21T02:51:18.365997+0000","last_fullsized":"2024-09-21T02:51:18.365997+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:04.714690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"78'5","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.552768+0000","last_change":"2024-09-21T02:50:50.356293+0000","last_active":"2024-09-21T02:52:15.552768+0000","last_peered":"2024-09-21T02:52:15.552768+0000","last_clean":"2024-09-21T02:52:15.552768+0000","last_became_active":"2024-09-21T02:50:50.355953+0000","last_became_peered":"2024-09-21T02:50:50.355953+0000","last_unstale":"2024-09-21T02:52:15.552768+0000","last_undegraded":"2024-09-21T02:52:15.552768+0000","last_fullsized":"2024-09-21T02:52:15.552768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:53.498850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'7","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.574014+0000","last_change":"2024-09-21T02:50:48.339458+0000","last_active":"2024-09-21T02:51:16.574014+0000","last_peered":"2024-09-21T02:51:16.574014+0000","last_clean":"2024-09-21T02:51:16.574014+0000","last_became_active":"2024-09-21T02:50:48.339011+0000","last_became_peered":"2024-09-21T02:50:48.339011+0000","last_unstale":"2024-09-21T02:51:16.574014+0000","last_undegraded":"2024-09-21T02:51:16.574014+0000","last_fullsized":"2024-09-21T02:51:16.574014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:38.211221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.563682+0000","last_change":"2024-09-21T02:50:50.364634+0000","last_active":"2024-09-21T02:52:15.563682+0000","last_peered":"2024-09-21T02:52:15.563682+0000","last_clean":"2024-09-21T02:52:15.563682+0000","last_became_active":"2024-09-21T02:50:50.363877+0000","last_became_peered":"2024-09-21T02:50:50.363877+0000","last_unstale":"2024-09-21T02:52:15.563682+0000","last_undegraded":"2024-09-21T02:52:15.563682+0000","last_fullsized":"2024-09-21T02:52:15.563682+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:37.143463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976982+0000","last_change":"2024-09-21T02:50:48.627515+0000","last_active":"2024-09-21T02:51:16.976982+0000","last_peered":"2024-09-21T02:51:16.976982+0000","last_clean":"2024-09-21T02:51:16.976982+0000","last_became_active":"2024-09-21T02:50:48.627148+0000","last_became_peered":"2024-09-21T02:50:48.627148+0000","last_unstale":"2024-09-21T02:51:16.976982+0000","last_undegraded":"2024-09-21T02:51:16.976982+0000","last_fullsized":"2024-09-21T02:51:16.976982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:01.381314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.550450+0000","last_change":"2024-09-21T02:50:50.359661+0000","last_active":"2024-09-21T02:52:15.550450+0000","last_peered":"2024-09-21T02:52:15.550450+0000","last_clean":"2024-09-21T02:52:15.550450+0000","last_became_active":"2024-09-21T02:50:50.358048+0000","last_became_peered":"2024-09-21T02:50:50.358048+0000","last_unstale":"2024-09-21T02:52:15.550450+0000","last_undegraded":"2024-09-21T02:52:15.550450+0000","last_fullsized":"2024-09-21T02:52:15.550450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:10.883318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"76'7","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976012+0000","last_change":"2024-09-21T02:50:48.334148+0000","last_active":"2024-09-21T02:51:16.976012+0000","last_peered":"2024-09-21T02:51:16.976012+0000","last_clean":"2024-09-21T02:51:16.976012+0000","last_became_active":"2024-09-21T02:50:48.333719+0000","last_became_peered":"2024-09-21T02:50:48.333719+0000","last_unstale":"2024-09-21T02:51:16.976012+0000","last_undegraded":"2024-09-21T02:51:16.976012+0000","last_fullsized":"2024-09-21T02:51:16.976012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:58.187405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.559275+0000","last_change":"2024-09-21T02:50:50.360215+0000","last_active":"2024-09-21T02:52:15.559275+0000","last_peered":"2024-09-21T02:52:15.559275+0000","last_clean":"2024-09-21T02:52:15.559275+0000","last_became_active":"2024-09-21T02:50:50.359552+0000","last_became_peered":"2024-09-21T02:50:50.359552+0000","last_unstale":"2024-09-21T02:52:15.559275+0000","last_undegraded":"2024-09-21T02:52:15.559275+0000","last_fullsized":"2024-09-21T02:52:15.559275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:40.597517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'4","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:58.432886+0000","last_change":"2024-09-21T02:50:48.332031+0000","last_active":"2024-09-21T02:52:58.432886+0000","last_peered":"2024-09-21T02:52:58.432886+0000","last_clean":"2024-09-21T02:52:58.432886+0000","last_became_active":"2024-09-21T02:50:48.331394+0000","last_became_peered":"2024-09-21T02:50:48.331394+0000","last_unstale":"2024-09-21T02:52:58.432886+0000","last_undegraded":"2024-09-21T02:52:58.432886+0000","last_fullsized":"2024-09-21T02:52:58.432886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:37.439329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.560328+0000","last_change":"2024-09-21T02:50:50.355014+0000","last_active":"2024-09-21T02:52:15.560328+0000","last_peered":"2024-09-21T02:52:15.560328+0000","last_clean":"2024-09-21T02:52:15.560328+0000","last_became_active":"2024-09-21T02:50:50.354450+0000","last_became_peered":"2024-09-21T02:50:50.354450+0000","last_unstale":"2024-09-21T02:52:15.560328+0000","last_undegraded":"2024-09-21T02:52:15.560328+0000","last_fullsized":"2024-09-21T02:52:15.560328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:05.523327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'4","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.983185+0000","last_change":"2024-09-21T02:51:16.983075+0000","last_active":"2024-09-21T02:51:16.983185+0000","last_peered":"2024-09-21T02:51:16.983185+0000","last_clean":"2024-09-21T02:51:16.983185+0000","last_became_active":"2024-09-21T02:51:16.978178+0000","last_became_peered":"2024-09-21T02:51:16.978178+0000","last_unstale":"2024-09-21T02:51:16.983185+0000","last_undegraded":"2024-09-21T02:51:16.983185+0000","last_fullsized":"2024-09-21T02:51:16.983185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:15.615511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570103+0000","last_change":"2024-09-21T02:50:50.368874+0000","last_active":"2024-09-21T02:52:15.570103+0000","last_peered":"2024-09-21T02:52:15.570103+0000","last_clean":"2024-09-21T02:52:15.570103+0000","last_became_active":"2024-09-21T02:50:50.368444+0000","last_became_peered":"2024-09-21T02:50:50.368444+0000","last_unstale":"2024-09-21T02:52:15.570103+0000","last_undegraded":"2024-09-21T02:52:15.570103+0000","last_fullsized":"2024-09-21T02:52:15.570103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:04.532469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"78'15","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.567443+0000","last_change":"2024-09-21T02:50:50.351764+0000","last_active":"2024-09-21T02:52:15.567443+0000","last_peered":"2024-09-21T02:52:15.567443+0000","last_clean":"2024-09-21T02:52:15.567443+0000","last_became_active":"2024-09-21T02:50:50.351382+0000","last_became_peered":"2024-09-21T02:50:50.351382+0000","last_unstale":"2024-09-21T02:52:15.567443+0000","last_undegraded":"2024-09-21T02:52:15.567443+0000","last_fullsized":"2024-09-21T02:52:15.567443+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:07.305679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"76'5","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.572464+0000","last_change":"2024-09-21T02:50:48.333610+0000","last_active":"2024-09-21T02:51:16.572464+0000","last_peered":"2024-09-21T02:51:16.572464+0000","last_clean":"2024-09-21T02:51:16.572464+0000","last_became_active":"2024-09-21T02:50:48.333079+0000","last_became_peered":"2024-09-21T02:50:48.333079+0000","last_unstale":"2024-09-21T02:51:16.572464+0000","last_undegraded":"2024-09-21T02:51:16.572464+0000","last_fullsized":"2024-09-21T02:51:16.572464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:11.056831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.564038+0000","last_change":"2024-09-21T02:50:50.360185+0000","last_active":"2024-09-21T02:52:15.564038+0000","last_peered":"2024-09-21T02:52:15.564038+0000","last_clean":"2024-09-21T02:52:15.564038+0000","last_became_active":"2024-09-21T02:50:50.359476+0000","last_became_peered":"2024-09-21T02:50:50.359476+0000","last_unstale":"2024-09-21T02:52:15.564038+0000","last_undegraded":"2024-09-21T02:52:15.564038+0000","last_fullsized":"2024-09-21T02:52:15.564038+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:42.459687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"76'12","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.596831+0000","last_change":"2024-09-21T02:51:16.596375+0000","last_active":"2024-09-21T02:51:16.596831+0000","last_peered":"2024-09-21T02:51:16.596831+0000","last_clean":"2024-09-21T02:51:16.596831+0000","last_became_active":"2024-09-21T02:51:16.580403+0000","last_became_peered":"2024-09-21T02:51:16.580403+0000","last_unstale":"2024-09-21T02:51:16.596831+0000","last_undegraded":"2024-09-21T02:51:16.596831+0000","last_fullsized":"2024-09-21T02:51:16.596831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.190817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.570214+0000","last_change":"2024-09-21T02:50:50.366682+0000","last_active":"2024-09-21T02:52:15.570214+0000","last_peered":"2024-09-21T02:52:15.570214+0000","last_clean":"2024-09-21T02:52:15.570214+0000","last_became_active":"2024-09-21T02:50:50.366290+0000","last_became_peered":"2024-09-21T02:50:50.366290+0000","last_unstale":"2024-09-21T02:52:15.570214+0000","last_undegraded":"2024-09-21T02:52:15.570214+0000","last_fullsized":"2024-09-21T02:52:15.570214+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:07.212526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"76'2","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:51:16.976625+0000","last_change":"2024-09-21T02:50:48.627411+0000","last_active":"2024-09-21T02:51:16.976625+0000","last_peered":"2024-09-21T02:51:16.976625+0000","last_clean":"2024-09-21T02:51:16.976625+0000","last_became_active":"2024-09-21T02:50:48.626938+0000","last_became_peered":"2024-09-21T02:50:48.626938+0000","last_unstale":"2024-09-21T02:51:16.976625+0000","last_undegraded":"2024-09-21T02:51:16.976625+0000","last_fullsized":"2024-09-21T02:51:16.976625+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:05.531529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"78'5","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:15.553717+0000","last_change":"2024-09-21T02:50:50.354143+0000","last_active":"2024-09-21T02:52:15.553717+0000","last_peered":"2024-09-21T02:52:15.553717+0000","last_clean":"2024-09-21T02:52:15.553717+0000","last_became_active":"2024-09-21T02:50:50.353769+0000","last_became_peered":"2024-09-21T02:50:50.353769+0000","last_unstale":"2024-09-21T02:52:15.553717+0000","last_undegraded":"2024-09-21T02:52:15.553717+0000","last_fullsized":"2024-09-21T02:52:15.553717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:49.327862+0000","last_clean_scrub_stamp":"2024-09-21T02:50:49.327862+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:50.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'14","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T02:52:55.697859+0000","last_change":"2024-09-21T02:50:48.333896+0000","last_active":"2024-09-21T02:52:55.697859+0000","last_peered":"2024-09-21T02:52:55.697859+0000","last_clean":"2024-09-21T02:52:55.697859+0000","last_became_active":"2024-09-21T02:50:48.333487+0000","last_became_peered":"2024-09-21T02:50:48.333487+0000","last_unstale":"2024-09-21T02:52:55.697859+0000","last_undegraded":"2024-09-21T02:52:55.697859+0000","last_fullsized":"2024-09-21T02:52:55.697859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:50:47.311457+0000","last_clean_scrub_stamp":"2024-09-21T02:50:47.311457+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:25.974400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73014,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1210,"num_read_kb":684,"num_write":259,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1125,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":230319,"data_compressed":5724,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284255,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":421333,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1075316,"data_compressed":43856,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":420619,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":864256,"data_stored":420885,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":1014887,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":421259,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1013334,"data_compressed":41948,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":421659,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":876544,"data_stored":420161,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378709,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":485285,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":491912,"data_compressed":7717,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705759,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":422598,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36139,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72415,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72013,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65950,"data_compressed":1908,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:53:02.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:53:02.537 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:02.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:02 smithi031 ceph-mon[31209]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:02.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:02 smithi031 ceph-mon[31209]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:02.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:02 smithi031 ceph-mon[31209]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 -- 172.21.15.31:0/321658853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c10de10 msgr2=0x7fc17c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 --2- 172.21.15.31:0/321658853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c10de10 0x7fc17c110300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc174009ca0 tx=0x7fc17402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 -- 172.21.15.31:0/321658853 shutdown_connections 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 --2- 172.21.15.31:0/321658853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c10de10 0x7fc17c110300 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 --2- 172.21.15.31:0/321658853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc17c105300 0x7fc17c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 --2- 172.21.15.31:0/321658853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1049b0 0x7fc17c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 -- 172.21.15.31:0/321658853 >> 172.21.15.31:0/321658853 conn(0x7fc17c079730 msgr2=0x7fc17c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 -- 172.21.15.31:0/321658853 shutdown_connections 2024-09-21T02:53:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.049+0000 7fc18174d640 1 -- 172.21.15.31:0/321658853 wait complete. 2024-09-21T02:53:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.050+0000 7fc18174d640 1 Processor -- start 2024-09-21T02:53:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.050+0000 7fc18174d640 1 -- start start 2024-09-21T02:53:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc18174d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 0x7fc17c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc18174d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc17c105300 0x7fc17c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc18174d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c10de10 0x7fc17c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc18174d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c113a70 con 0x7fc17c1049b0 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc18174d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1138f0 con 0x7fc17c105300 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc18174d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c113bf0 con 0x7fc17c10de10 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 0x7fc17c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 0x7fc17c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43326/0 (socket says 172.21.15.31:43326) 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 -- 172.21.15.31:0/3606645787 learned_addr learned my addr 172.21.15.31:0/3606645787 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17a7fc640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc17c105300 0x7fc17c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17b7fe640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c10de10 0x7fc17c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 -- 172.21.15.31:0/3606645787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c10de10 msgr2=0x7fc17c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c10de10 0x7fc17c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 -- 172.21.15.31:0/3606645787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc17c105300 msgr2=0x7fc17c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc17c105300 0x7fc17c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 -- 172.21.15.31:0/3606645787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c1a03b0 con 0x7fc17c1049b0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.051+0000 7fc17affd640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 0x7fc17c19a440 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00d770 tx=0x7fc16c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.052+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c017070 con 0x7fc17c1049b0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.052+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c1a06a0 con 0x7fc17c1049b0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.052+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16c009c30 con 0x7fc17c1049b0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.052+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c003cc0 con 0x7fc17c1049b0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.052+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c1a6e00 con 0x7fc17c1049b0 2024-09-21T02:53:03.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.053+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17c104dc0 con 0x7fc17c1049b0 2024-09-21T02:53:03.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.054+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fc16c003e60 con 0x7fc17c1049b0 2024-09-21T02:53:03.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.055+0000 7fc163fff640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc1640b65f0 0x7fc1640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:03.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.055+0000 7fc17a7fc640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc1640b65f0 0x7fc1640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:03.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.055+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc16c0e73a0 con 0x7fc17c1049b0 2024-09-21T02:53:03.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.056+0000 7fc17a7fc640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc1640b65f0 0x7fc1640b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc17c10fa50 tx=0x7fc17000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.059+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc16c009de0 con 0x7fc17c1049b0 2024-09-21T02:53:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:02 smithi165 ceph-mon[34818]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:02 smithi165 ceph-mon[34818]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:02 smithi165 ceph-mon[34818]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:02 smithi178 ceph-mon[34759]: pgmap v329: 97 pgs: 97 active+clean; 648 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:02 smithi178 ceph-mon[34759]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:02 smithi178 ceph-mon[34759]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.282+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17c1a7130 con 0x7fc17c1049b0 2024-09-21T02:53:03.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.283+0000 7fc163fff640 1 -- 172.21.15.31:0/3606645787 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fc16c0ad9d0 con 0x7fc17c1049b0 2024-09-21T02:53:03.285 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:03.285 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:53:03.285 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:53:03.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc1640b65f0 msgr2=0x7fc1640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:03.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc1640b65f0 0x7fc1640b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc17c10fa50 tx=0x7fc17000b390 comp rx=0 tx=0).stop 2024-09-21T02:53:03.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 msgr2=0x7fc17c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 0x7fc17c19a440 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00d770 tx=0x7fc16c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 shutdown_connections 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc1640b65f0 0x7fc1640b8b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c10de10 0x7fc17c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc17c105300 0x7fc17c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 --2- 172.21.15.31:0/3606645787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc17c1049b0 0x7fc17c19a440 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 >> 172.21.15.31:0/3606645787 conn(0x7fc17c079730 msgr2=0x7fc17c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 shutdown_connections 2024-09-21T02:53:03.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:03.286+0000 7fc18174d640 1 -- 172.21.15.31:0/3606645787 wait complete. 2024-09-21T02:53:03.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:03 smithi031 ceph-mon[31209]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:03.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:03 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3606645787' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:04.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:53:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:03 smithi165 ceph-mon[34818]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:03 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3606645787' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:04.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:03 smithi178 ceph-mon[34759]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:53:04.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:03 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3606645787' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/524750448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a01083a0 msgr2=0x7fd8a01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/524750448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a01083a0 0x7fd8a01087b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd890009900 tx=0x7fd89002a410 comp rx=0 tx=0).stop 2024-09-21T02:53:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/524750448 shutdown_connections 2024-09-21T02:53:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/524750448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 0x7fd8a010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/524750448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a01083a0 0x7fd8a01087b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/524750448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a0102a60 0x7fd8a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/524750448 >> 172.21.15.31:0/524750448 conn(0x7fd8a0079730 msgr2=0x7fd8a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/524750448 shutdown_connections 2024-09-21T02:53:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/524750448 wait complete. 2024-09-21T02:53:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.826+0000 7fd8a7d0f640 1 Processor -- start 2024-09-21T02:53:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 -- start start 2024-09-21T02:53:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 0x7fd8a019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01083a0 0x7fd8a019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 0x7fd8a019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0112960 con 0x7fd8a0102a60 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a01127e0 con 0x7fd8a01083a0 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a7d0f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0112ae0 con 0x7fd8a0108cf0 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 0x7fd8a019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 0x7fd8a019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43346/0 (socket says 172.21.15.31:43346) 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 -- 172.21.15.31:0/1764839409 learned_addr learned my addr 172.21.15.31:0/1764839409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a52aa640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01083a0 0x7fd8a019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a62ac640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 0x7fd8a019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 -- 172.21.15.31:0/1764839409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 msgr2=0x7fd8a019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 0x7fd8a019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 -- 172.21.15.31:0/1764839409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01083a0 msgr2=0x7fd8a019b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01083a0 0x7fd8a019b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a5aab640 1 -- 172.21.15.31:0/1764839409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a01a6540 con 0x7fd8a0102a60 2024-09-21T02:53:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.827+0000 7fd8a62ac640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 0x7fd8a019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.828+0000 7fd8a5aab640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 0x7fd8a019b230 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00c890 tx=0x7fd89c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.828+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c004220 con 0x7fd8a0102a60 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.828+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd89c0043c0 con 0x7fd8a0102a60 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.828+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a01a6830 con 0x7fd8a0102a60 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.828+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c004a00 con 0x7fd8a0102a60 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.828+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01a6d90 con 0x7fd8a0102a60 2024-09-21T02:53:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.829+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a0103ce0 con 0x7fd8a0102a60 2024-09-21T02:53:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.831+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7fd89c0079b0 con 0x7fd8a0102a60 2024-09-21T02:53:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.831+0000 7fd896ffd640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd88c0b6670 0x7fd88c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.832+0000 7fd8a52aa640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd88c0b6670 0x7fd88c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.832+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd89c0e7730 con 0x7fd8a0102a60 2024-09-21T02:53:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.832+0000 7fd8a52aa640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd88c0b6670 0x7fd88c0b8b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd890000c00 tx=0x7fd890031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:04.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:04.835+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd89c014030 con 0x7fd8a0102a60 2024-09-21T02:53:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.027+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8a0102e70 con 0x7fd8a0102a60 2024-09-21T02:53:05.028 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:04 smithi031 ceph-mon[31209]: pgmap v330: 97 pgs: 97 active+clean; 656 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-21T02:53:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.028+0000 7fd896ffd640 1 -- 172.21.15.31:0/1764839409 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+23145 (secure 0 0 0) 0x7fd89c0adde0 con 0x7fd8a0102a60 2024-09-21T02:53:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:05.030 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":78,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T02:51:16.567886+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:53:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd88c0b6670 msgr2=0x7fd88c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd88c0b6670 0x7fd88c0b8b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd890000c00 tx=0x7fd890031400 comp rx=0 tx=0).stop 2024-09-21T02:53:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 msgr2=0x7fd8a019b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 0x7fd8a019b230 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00c890 tx=0x7fd89c00cd50 comp rx=0 tx=0).stop 2024-09-21T02:53:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 shutdown_connections 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd88c0b6670 0x7fd88c0b8b80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a0108cf0 0x7fd8a019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8a01083a0 0x7fd8a019b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 --2- 172.21.15.31:0/1764839409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8a0102a60 0x7fd8a019b230 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 >> 172.21.15.31:0/1764839409 conn(0x7fd8a0079730 msgr2=0x7fd8a0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 shutdown_connections 2024-09-21T02:53:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.031+0000 7fd8a7d0f640 1 -- 172.21.15.31:0/1764839409 wait complete. 2024-09-21T02:53:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:04 smithi165 ceph-mon[34818]: pgmap v330: 97 pgs: 97 active+clean; 656 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-21T02:53:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:04 smithi178 ceph-mon[34759]: pgmap v330: 97 pgs: 97 active+clean; 656 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-21T02:53:05.713 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T02:53:05.713 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T02:53:05.723 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T02:53:05.723 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T02:53:05.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T02:53:05.724 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T02:53:05.724 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:53:05.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 -- 172.21.15.31:0/3662669031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f581029b0 msgr2=0x7f9f58102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 --2- 172.21.15.31:0/3662669031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f581029b0 0x7f9f58102dc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f4802a410 comp rx=0 tx=0).stop 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 -- 172.21.15.31:0/3662669031 shutdown_connections 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 --2- 172.21.15.31:0/3662669031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f58108c30 0x7f9f5810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 --2- 172.21.15.31:0/3662669031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f581082e0 0x7f9f581086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 --2- 172.21.15.31:0/3662669031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f581029b0 0x7f9f58102dc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 -- 172.21.15.31:0/3662669031 >> 172.21.15.31:0/3662669031 conn(0x7f9f5807a740 msgr2=0x7f9f5807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 -- 172.21.15.31:0/3662669031 shutdown_connections 2024-09-21T02:53:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.844+0000 7f9f5e230640 1 -- 172.21.15.31:0/3662669031 wait complete. 2024-09-21T02:53:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 Processor -- start 2024-09-21T02:53:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 -- start start 2024-09-21T02:53:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f581029b0 0x7f9f580ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f581082e0 0x7f9f580ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 0x7f9f580fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58112940 con 0x7f9f58108c30 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581127c0 con 0x7f9f581029b0 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f5e230640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58112ac0 con 0x7f9f581082e0 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.845+0000 7f9f57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 0x7f9f580fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 0x7f9f580fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43362/0 (socket says 172.21.15.31:43362) 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 -- 172.21.15.31:0/371171061 learned_addr learned my addr 172.21.15.31:0/371171061 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f56ffd640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f581082e0 0x7f9f580ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f577fe640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f581029b0 0x7f9f580ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 -- 172.21.15.31:0/371171061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f581082e0 msgr2=0x7f9f580ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f581082e0 0x7f9f580ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 -- 172.21.15.31:0/371171061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f581029b0 msgr2=0x7f9f580ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f581029b0 0x7f9f580ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 -- 172.21.15.31:0/371171061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f581a87c0 con 0x7f9f58108c30 2024-09-21T02:53:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f57fff640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 0x7f9f580fffd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009fd0 tx=0x7f9f5000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50017070 con 0x7f9f58108c30 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f581a8ab0 con 0x7f9f58108c30 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.846+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a8fd0 con 0x7f9f58108c30 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.847+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f50003e20 con 0x7f9f58108c30 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.847+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50020060 con 0x7f9f58108c30 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.847+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f440052c0 con 0x7f9f58108c30 2024-09-21T02:53:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.849+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f9f50003970 con 0x7f9f58108c30 2024-09-21T02:53:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.850+0000 7f9f54ff9640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f400b66a0 0x7f9f400b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.850+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9f500e7420 con 0x7f9f58108c30 2024-09-21T02:53:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.850+0000 7f9f577fe640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f400b66a0 0x7f9f400b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.851+0000 7f9f577fe640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f400b66a0 0x7f9f400b8bb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f48005ab0 tx=0x7f9f48005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:05.858+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f500b1eb0 con 0x7f9f58108c30 2024-09-21T02:53:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:06.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.075+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9f44000bf0 con 0x7f9f400b66a0 2024-09-21T02:53:06.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:05 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1764839409' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:53:06.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:05 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1764839409' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:53:06.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:05 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1764839409' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:53:06.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.240+0000 7f9f54ff9640 1 -- 172.21.15.31:0/371171061 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9f44000bf0 con 0x7f9f400b66a0 2024-09-21T02:53:06.242 INFO:teuthology.orchestra.run.smithi031.stdout:Schedule set for path / 2024-09-21T02:53:06.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f400b66a0 msgr2=0x7f9f400b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f400b66a0 0x7f9f400b8bb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f48005ab0 tx=0x7f9f48005a20 comp rx=0 tx=0).stop 2024-09-21T02:53:06.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 msgr2=0x7f9f580fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 0x7f9f580fffd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009fd0 tx=0x7f9f5000b260 comp rx=0 tx=0).stop 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 shutdown_connections 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f400b66a0 0x7f9f400b8bb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f581082e0 0x7f9f580ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f58108c30 0x7f9f580fffd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 --2- 172.21.15.31:0/371171061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f581029b0 0x7f9f580ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 >> 172.21.15.31:0/371171061 conn(0x7f9f5807a740 msgr2=0x7f9f58101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 shutdown_connections 2024-09-21T02:53:06.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.243+0000 7f9f5e230640 1 -- 172.21.15.31:0/371171061 wait complete. 2024-09-21T02:53:06.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T02:53:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 -- 172.21.15.31:0/2720368110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c108c30 msgr2=0x7f310c10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 --2- 172.21.15.31:0/2720368110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c108c30 0x7f310c10f460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3104009860 tx=0x7f310402a340 comp rx=0 tx=0).stop 2024-09-21T02:53:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 -- 172.21.15.31:0/2720368110 shutdown_connections 2024-09-21T02:53:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 --2- 172.21.15.31:0/2720368110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c1082e0 0x7f310c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 --2- 172.21.15.31:0/2720368110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c108c30 0x7f310c10f460 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 --2- 172.21.15.31:0/2720368110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1029b0 0x7f310c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 -- 172.21.15.31:0/2720368110 >> 172.21.15.31:0/2720368110 conn(0x7f310c07a740 msgr2=0x7f310c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 -- 172.21.15.31:0/2720368110 shutdown_connections 2024-09-21T02:53:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.387+0000 7f3113007640 1 -- 172.21.15.31:0/2720368110 wait complete. 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 Processor -- start 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 -- start start 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 0x7f310c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1082e0 0x7f310c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c108c30 0x7f310c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1128f0 con 0x7f310c1029b0 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f310c112770 con 0x7f310c1082e0 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3113007640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f310c112a70 con 0x7f310c108c30 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3110da3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 0x7f310c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3110da3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 0x7f310c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43386/0 (socket says 172.21.15.31:43386) 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f3110da3640 1 -- 172.21.15.31:0/2626849895 learned_addr learned my addr 172.21.15.31:0/2626849895 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f31115a4640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c108c30 0x7f310c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.388+0000 7f310bfff640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1082e0 0x7f310c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3110da3640 1 -- 172.21.15.31:0/2626849895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c108c30 msgr2=0x7f310c19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3110da3640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c108c30 0x7f310c19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3110da3640 1 -- 172.21.15.31:0/2626849895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1082e0 msgr2=0x7f310c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3110da3640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1082e0 0x7f310c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3110da3640 1 -- 172.21.15.31:0/2626849895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c1a0330 con 0x7f310c1029b0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f310bfff640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1082e0 0x7f310c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3110da3640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 0x7f310c19a400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00ebc0 tx=0x7f30fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc010630 con 0x7f310c1029b0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc010c80 con 0x7f310c1029b0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c1a0620 con 0x7f310c1029b0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc018b60 con 0x7f310c1029b0 2024-09-21T02:53:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.389+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c1a6db0 con 0x7f310c1029b0 2024-09-21T02:53:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.390+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310c102dc0 con 0x7f310c1029b0 2024-09-21T02:53:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.392+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f30fc018d00 con 0x7f310c1029b0 2024-09-21T02:53:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.393+0000 7f3109ffb640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f80b65f0 0x7f30f80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.393+0000 7f310bfff640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f80b65f0 0x7f30f80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.393+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f30fc0e8110 con 0x7f310c1029b0 2024-09-21T02:53:06.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.394+0000 7f310bfff640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f80b65f0 0x7f30f80b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f310c07ac70 tx=0x7f30f4009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.397+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30fc0ae780 con 0x7f310c1029b0 2024-09-21T02:53:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 -- 172.21.15.31:0/1056979007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f041083b0 msgr2=0x7f7f041087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 --2- 172.21.15.31:0/1056979007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f041083b0 0x7f7f041087c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009ca0 tx=0x7f7ef402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 -- 172.21.15.31:0/1056979007 shutdown_connections 2024-09-21T02:53:06.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 --2- 172.21.15.31:0/1056979007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f041083b0 0x7f7f041087c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 --2- 172.21.15.31:0/1056979007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 0x7f7f0410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 --2- 172.21.15.31:0/1056979007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f04102a70 0x7f7f04102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 -- 172.21.15.31:0/1056979007 >> 172.21.15.31:0/1056979007 conn(0x7f7f04079730 msgr2=0x7f7f04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 -- 172.21.15.31:0/1056979007 shutdown_connections 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.539+0000 7f7f08af1640 1 -- 172.21.15.31:0/1056979007 wait complete. 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 Processor -- start 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 -- start start 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a70 0x7f7f0419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 0x7f7f0419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 0x7f7f0419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04112970 con 0x7f7f041083b0 2024-09-21T02:53:06.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041127f0 con 0x7f7f04108d00 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f08af1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04112af0 con 0x7f7f04102a70 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 0x7f7f0419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 0x7f7f0419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43404/0 (socket says 172.21.15.31:43404) 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f01d9b640 1 -- 172.21.15.31:0/3513441711 learned_addr learned my addr 172.21.15.31:0/3513441711 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f0259c640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a70 0x7f7f0419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.540+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 0x7f7f0419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f01d9b640 1 -- 172.21.15.31:0/3513441711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a70 msgr2=0x7f7f0419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a70 0x7f7f0419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f01d9b640 1 -- 172.21.15.31:0/3513441711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 msgr2=0x7f7f0419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 0x7f7f0419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f01d9b640 1 -- 172.21.15.31:0/3513441711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f041a6650 con 0x7f7f041083b0 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 0x7f7f0419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:06.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 0x7f7f0419b900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7ef402ade0 tx=0x7f7ef4031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7efb7fe640 1 -- 172.21.15.31:0/3513441711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003b70 con 0x7f7f041083b0 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f041a68e0 con 0x7f7f041083b0 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7efb7fe640 1 -- 172.21.15.31:0/3513441711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef4003d10 con 0x7f7f041083b0 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7efb7fe640 1 -- 172.21.15.31:0/3513441711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef402fa50 con 0x7f7f041083b0 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.541+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041a6e20 con 0x7f7f041083b0 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 Processor -- start 2024-09-21T02:53:06.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 -- start start 2024-09-21T02:53:06.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 0x557e036d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e03304490 0x557e036d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e0330c670 0x557e036d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557e0364ae60 con 0x557e0332d350 2024-09-21T02:53:06.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557e0363a290 con 0x557e03304490 2024-09-21T02:53:06.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7f0f065740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e0363e0a0 con 0x557e0330c670 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.544+0000 7f7efb7fe640 1 -- 172.21.15.31:0/3513441711 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f7ef4004580 con 0x7f7f041083b0 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 0x557e036d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 0x557e036d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43418/0 (socket says 172.21.15.31:43418) 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 -- 172.21.15.31:0/2412805685 learned_addr learned my addr 172.21.15.31:0/2412805685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e0330c670 0x557e036d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f0259c640 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e03304490 0x557e036d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 -- 172.21.15.31:0/2412805685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e0330c670 msgr2=0x557e036d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e0330c670 0x557e036d4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 -- 172.21.15.31:0/2412805685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e03304490 msgr2=0x557e036d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e03304490 0x557e036d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 -- 172.21.15.31:0/2412805685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e03398930 con 0x557e0332d350 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.545+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 0x557e036d4430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00dc70 tx=0x7f7efc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f830fb6bced4c1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.546+0000 7f7ef8ff9640 1 -- 172.21.15.31:0/2412805685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc015070 con 0x557e0332d350 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.546+0000 7f7ef8ff9640 1 -- 172.21.15.31:0/2412805685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7efc0035f0 con 0x557e0332d350 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.546+0000 7f7ef8ff9640 1 -- 172.21.15.31:0/2412805685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0116a0 con 0x557e0332d350 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.546+0000 7f7efb7fe640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ef00b6620 0x7f7ef00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.546+0000 7f7f0259c640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ef00b6620 0x7f7ef00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.546+0000 7f7efb7fe640 1 -- 172.21.15.31:0/3513441711 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7ef41088c0 con 0x7f7f041083b0 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 -- 172.21.15.31:0/2412805685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 msgr2=0x557e036d4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0259c640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ef00b6620 0x7f7ef00b8b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x557e036d30b0 tx=0x7f7eec005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 0x557e036d4430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00dc70 tx=0x7f7efc00f730 comp rx=0 tx=0).stop 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 -- 172.21.15.31:0/2412805685 shutdown_connections 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e0330c670 0x557e036d4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e03304490 0x557e036d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2412805685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0332d350 0x557e036d4430 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 -- 172.21.15.31:0/2412805685 >> 172.21.15.31:0/2412805685 conn(0x557e03321740 msgr2=0x557e036d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 -- 172.21.15.31:0/2412805685 shutdown_connections 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 -- 172.21.15.31:0/2412805685 wait complete. 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 -- start start 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.547+0000 7f7f0f065740 1 Processor -- start 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557e0349b890 con 0x557e0330c670 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557e035d4d40 con 0x557e0332d350 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e036e2c10 con 0x557e03304490 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0f065740 10 client.0 _ll_register_callbacks cb 0x557e03638c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036e0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43428/0 (socket says 172.21.15.31:43428) 2024-09-21T02:53:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2175880179 learned_addr learned my addr 172.21.15.31:0/2175880179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0259c640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2175880179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 msgr2=0x557e036e00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 msgr2=0x557e036ebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036ebf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2175880179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e036ec540 con 0x557e0330c670 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.548+0000 7f7f0259c640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036e0660 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40ce720 tx=0x7f7ef410c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c349f9d5ce3dd105 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/2175880179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40ca630 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/2175880179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7ef402fbf0 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7efc0035f0 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/2175880179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ef40cdf00 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7f0f065740 10 client.15582 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e036eee70 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/2175880179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7efc0035f0 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.549+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/2175880179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ef40cd9f0 con 0x557e0330c670 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0f065740 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0f065740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0f065740 20 client.15582 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032ebcd0 0x557e036f2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0f065740 4 client.15582 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e0364cdb0 con 0x557e032ebcd0 2024-09-21T02:53:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0259c640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032ebcd0 0x557e036f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7f0259c640 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032ebcd0 0x557e036f2390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c5679be516d92e server_cookie=287c4e7d501953f5 in_seq=0 out_seq=0 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.550+0000 7f7ee9ffb640 10 client.15582 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/2175880179 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e0364cdb0 con 0x557e032ebcd0 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7ee9ffb640 10 client.15582 handle_command_reply: tid=0 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 client.15582 shutdown 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 msgr2=0x557e036e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036e0660 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40ce720 tx=0x7f7ef410c040 comp rx=0 tx=0).stop 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 shutdown_connections 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032ebcd0 0x557e036f2390 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036e0660 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2175880179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 >> 172.21.15.31:0/2175880179 conn(0x557e03321740 msgr2=0x557e036d7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.551+0000 7f7ee9ffb640 0 client.15582 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.552+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 shutdown_connections 2024-09-21T02:53:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.552+0000 7f7f0f065740 1 -- 172.21.15.31:0/2175880179 wait complete. 2024-09-21T02:53:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.552+0000 7f7f0f065740 20 client.15582 trim_cache size 0 max 16384 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.569+0000 7f7f0f065740 1 Processor -- start 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.569+0000 7f7f0f065740 1 -- start start 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036d9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0f065740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ad620 con 0x557e0330c670 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0f065740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041af160 con 0x557e0332d350 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0f065740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041adcf0 con 0x557e03304490 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036d9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036d9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43442/0 (socket says 172.21.15.31:43442) 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2432255151 learned_addr learned my addr 172.21.15.31:0/2432255151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f0259c640 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2432255151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 msgr2=0x557e036ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2432255151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 msgr2=0x557e036d9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036d9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 -- 172.21.15.31:0/2432255151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e036d6950 con 0x557e0330c670 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.570+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036d9510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4004af0 tx=0x7f7ef4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49363abfe9be8d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.571+0000 7f7eebfff640 1 -- 172.21.15.31:0/2432255151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40ce0e0 con 0x557e0330c670 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.571+0000 7f7eebfff640 1 -- 172.21.15.31:0/2432255151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef4036070 con 0x557e0330c670 2024-09-21T02:53:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.571+0000 7f7eebfff640 1 -- 172.21.15.31:0/2432255151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40ce560 con 0x557e0330c670 2024-09-21T02:53:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 -- 172.21.15.31:0/2432255151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 msgr2=0x557e036d9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036d9510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4004af0 tx=0x7f7ef4000f40 comp rx=0 tx=0).stop 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 -- 172.21.15.31:0/2432255151 shutdown_connections 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036d9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036d9510 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 --2- 172.21.15.31:0/2432255151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 -- 172.21.15.31:0/2432255151 >> 172.21.15.31:0/2432255151 conn(0x557e03321740 msgr2=0x557e036de740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 -- 172.21.15.31:0/2432255151 shutdown_connections 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 -- 172.21.15.31:0/2432255151 wait complete. 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 -- start start 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 Processor -- start 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.572+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0f065740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0f065740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041aea90 con 0x557e0330c670 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0f065740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ae3c0 con 0x557e0332d350 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0f065740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ab600 con 0x557e03304490 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0f065740 10 client.0 _ll_register_callbacks cb 0x7f7ef4009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:49102/0 (socket says 172.21.15.31:49102) 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 -- 172.21.15.31:0/1021839232 learned_addr learned my addr 172.21.15.31:0/1021839232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f02d9d640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 -- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 msgr2=0x557e036f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 -- 172.21.15.31:0/1021839232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 msgr2=0x557e036f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 -- 172.21.15.31:0/1021839232 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e036e4220 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7f0259c640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e87d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f0406c640 tx=0x7f7eec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dce310020afd90f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.573+0000 7f7ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/1021839232 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec036060 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/1021839232 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7eec031440 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef4036070 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/1021839232 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7eec031640 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 10 client.25273 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e0375cf00 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/1021839232 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec031c90 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/1021839232 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7eec01f9e0 con 0x557e03304490 2024-09-21T02:53:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 10 client.25273 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 20 client.25273 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032c1080 0x557e0375f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 4 client.25273 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:53:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.574+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7ef4036070 con 0x557e032c1080 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.575+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032c1080 0x557e0375f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.575+0000 7f7f01d9b640 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032c1080 0x557e0375f720 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=791c8eaf1f46d1b5 server_cookie=10c4840fe862af7a in_seq=0 out_seq=0 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.575+0000 7f7ee9ffb640 10 client.25273 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7ee9ffb640 1 -- 172.21.15.31:0/1021839232 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7ef4036070 con 0x557e032c1080 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7ee9ffb640 10 client.25273 handle_command_reply: tid=0 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 client.25273 shutdown 2024-09-21T02:53:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 msgr2=0x557e036e87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e87d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f0406c640 tx=0x7f7eec0046c0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 shutdown_connections 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557e032c1080 0x557e0375f720 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7ee9ffb640 0 client.25273 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e0332d350 0x557e036f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557e0330c670 0x557e036f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 --2- 172.21.15.31:0/1021839232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e03304490 0x557e036e87d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 >> 172.21.15.31:0/1021839232 conn(0x557e03321740 msgr2=0x557e036d7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 shutdown_connections 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 1 -- 172.21.15.31:0/1021839232 wait complete. 2024-09-21T02:53:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.577+0000 7f7f0f065740 20 client.25273 trim_cache size 0 max 16384 2024-09-21T02:53:06.582 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:53:06.582 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:53:06.582 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "2733a6e0-8560-4410-bda8-b9913220c0b1", 2024-09-21T02:53:06.582 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:53:06.582 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:53:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ef00b6620 msgr2=0x7f7ef00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ef00b6620 0x7f7ef00b8b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x557e036d30b0 tx=0x7f7eec005f90 comp rx=0 tx=0).stop 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 msgr2=0x7f7f0419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 0x7f7f0419b900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7ef402ade0 tx=0x7f7ef4031770 comp rx=0 tx=0).stop 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 shutdown_connections 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ef00b6620 0x7f7ef00b8b30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04108d00 0x7f7f0419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f041083b0 0x7f7f0419b900 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 --2- 172.21.15.31:0/3513441711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a70 0x7f7f0419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 >> 172.21.15.31:0/3513441711 conn(0x7f7f04079730 msgr2=0x7f7f04104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 shutdown_connections 2024-09-21T02:53:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.578+0000 7f7f08af1640 1 -- 172.21.15.31:0/3513441711 wait complete. 2024-09-21T02:53:06.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.620+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f310c1016e0 con 0x7f30f80b65f0 2024-09-21T02:53:06.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.681+0000 7f3109ffb640 1 -- 172.21.15.31:0/2626849895 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f310c1016e0 con 0x7f30f80b65f0 2024-09-21T02:53:06.682 INFO:teuthology.orchestra.run.smithi031.stdout:Retention added to path / 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f80b65f0 msgr2=0x7f30f80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f80b65f0 0x7f30f80b8b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f310c07ac70 tx=0x7f30f4009260 comp rx=0 tx=0).stop 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 msgr2=0x7f310c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 0x7f310c19a400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00ebc0 tx=0x7f30fc00c6a0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 shutdown_connections 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f30f80b65f0 0x7f30f80b8b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c108c30 0x7f310c19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f310c1082e0 0x7f310c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 --2- 172.21.15.31:0/2626849895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c1029b0 0x7f310c19a400 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 >> 172.21.15.31:0/2626849895 conn(0x7f310c07a740 msgr2=0x7f310c06adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.683+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 shutdown_connections 2024-09-21T02:53:06.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.684+0000 7f3113007640 1 -- 172.21.15.31:0/2626849895 wait complete. 2024-09-21T02:53:06.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/1130415281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 msgr2=0x7f5c68108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/1130415281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c68108710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009900 tx=0x7f5c5c02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/1130415281 shutdown_connections 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/1130415281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c68108c50 0x7f5c6810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/1130415281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c68108710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/1130415281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 0x7f5c68102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/1130415281 >> 172.21.15.31:0/1130415281 conn(0x7f5c680784e0 msgr2=0x7f5c680788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:06.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/1130415281 shutdown_connections 2024-09-21T02:53:06.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.812+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/1130415281 wait complete. 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 Processor -- start 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 -- start start 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 0x7f5c6819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c6819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c68108c50 0x7f5c6819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68112910 con 0x7f5c68108300 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68112790 con 0x7f5c681029e0 2024-09-21T02:53:06.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6fb7b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68112a90 con 0x7f5c68108c50 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6d116640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c6819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.813+0000 7f5c6d116640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c6819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43466/0 (socket says 172.21.15.31:43466) 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 -- 172.21.15.31:0/2272340068 learned_addr learned my addr 172.21.15.31:0/2272340068 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6e118640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c68108c50 0x7f5c6819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d917640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 0x7f5c6819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 -- 172.21.15.31:0/2272340068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c68108c50 msgr2=0x7f5c6819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c68108c50 0x7f5c6819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 -- 172.21.15.31:0/2272340068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 msgr2=0x7f5c6819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 0x7f5c6819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 -- 172.21.15.31:0/2272340068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c681a03f0 con 0x7f5c68108300 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d917640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 0x7f5c6819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:06.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6d116640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c6819aa60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c006010 tx=0x7f5c5c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c03f030 con 0x7f5c68108300 2024-09-21T02:53:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c681a0680 con 0x7f5c68108300 2024-09-21T02:53:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c003aa0 con 0x7f5c68108300 2024-09-21T02:53:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c035070 con 0x7f5c68108300 2024-09-21T02:53:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.814+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c681a6e70 con 0x7f5c68108300 2024-09-21T02:53:06.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.815+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c68103c40 con 0x7f5c68108300 2024-09-21T02:53:06.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.817+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f5c5c042030 con 0x7f5c68108300 2024-09-21T02:53:06.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.818+0000 7f5c66ffd640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5c500b65f0 0x7f5c500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:06.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.818+0000 7f5c6d917640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5c500b65f0 0x7f5c500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:06.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.818+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5c5c107f40 con 0x7f5c68108300 2024-09-21T02:53:06.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.818+0000 7f5c6d917640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5c500b65f0 0x7f5c500b8b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c58005fd0 tx=0x7f5c58009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:06.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:06.821+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c5c0ce510 con 0x7f5c68108300 2024-09-21T02:53:07.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.035+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5c68063800 con 0x7f5c500b65f0 2024-09-21T02:53:07.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.042+0000 7f5c66ffd640 1 -- 172.21.15.31:0/2272340068 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f5c68063800 con 0x7f5c500b65f0 2024-09-21T02:53:07.044 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T02:53:06", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T02:53:07.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5c500b65f0 msgr2=0x7f5c500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5c500b65f0 0x7f5c500b8b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c58005fd0 tx=0x7f5c58009500 comp rx=0 tx=0).stop 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 msgr2=0x7f5c6819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c6819aa60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c006010 tx=0x7f5c5c004060 comp rx=0 tx=0).stop 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 shutdown_connections 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5c500b65f0 0x7f5c500b8b00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c68108c50 0x7f5c6819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c68108300 0x7f5c6819aa60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 --2- 172.21.15.31:0/2272340068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c681029e0 0x7f5c6819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 >> 172.21.15.31:0/2272340068 conn(0x7f5c680784e0 msgr2=0x7f5c6806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 shutdown_connections 2024-09-21T02:53:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.045+0000 7f5c6fb7b640 1 -- 172.21.15.31:0/2272340068 wait complete. 2024-09-21T02:53:07.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:06 smithi031 ceph-mon[31209]: pgmap v331: 97 pgs: 97 active+clean; 656 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-21T02:53:07.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T02:53:07.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 -- 172.21.15.31:0/453885766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8109420 msgr2=0x7f77e8109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 --2- 172.21.15.31:0/453885766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8109420 0x7f77e8109830 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009900 tx=0x7f77d802a410 comp rx=0 tx=0).stop 2024-09-21T02:53:07.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 -- 172.21.15.31:0/453885766 shutdown_connections 2024-09-21T02:53:07.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 --2- 172.21.15.31:0/453885766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e8109d70 0x7f77e81105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 --2- 172.21.15.31:0/453885766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8109420 0x7f77e8109830 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 --2- 172.21.15.31:0/453885766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8103af0 0x7f77e8103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 -- 172.21.15.31:0/453885766 >> 172.21.15.31:0/453885766 conn(0x7f77e80ff570 msgr2=0x7f77e81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:07.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 -- 172.21.15.31:0/453885766 shutdown_connections 2024-09-21T02:53:07.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 -- 172.21.15.31:0/453885766 wait complete. 2024-09-21T02:53:07.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.172+0000 7f77ef50b640 1 Processor -- start 2024-09-21T02:53:07.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 -- start start 2024-09-21T02:53:07.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 0x7f77e819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:07.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e8109420 0x7f77e819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:07.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8109d70 0x7f77e819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8113a30 con 0x7f77e8103af0 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81138b0 con 0x7f77e8109d70 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ef50b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8113bb0 con 0x7f77e8109420 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 0x7f77e819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 0x7f77e819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43478/0 (socket says 172.21.15.31:43478) 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 -- 172.21.15.31:0/4084763522 learned_addr learned my addr 172.21.15.31:0/4084763522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ecaa6640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e8109420 0x7f77e819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77edaa8640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8109d70 0x7f77e819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 -- 172.21.15.31:0/4084763522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e8109420 msgr2=0x7f77e819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e8109420 0x7f77e819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 -- 172.21.15.31:0/4084763522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8109d70 msgr2=0x7f77e819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8109d70 0x7f77e819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 -- 172.21.15.31:0/4084763522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81a03a0 con 0x7f77e8103af0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77edaa8640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8109d70 0x7f77e819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.173+0000 7f77ed2a7640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 0x7f77e819a470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f77e400d950 tx=0x7f77e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.174+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4017070 con 0x7f77e8103af0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.174+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e4009c30 con 0x7f77e8103af0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.174+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e81a0690 con 0x7f77e8103af0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.174+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e400f620 con 0x7f77e8103af0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.174+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81a6e20 con 0x7f77e8103af0 2024-09-21T02:53:07.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.175+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e8103f00 con 0x7f77e8103af0 2024-09-21T02:53:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.176+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154580+0+0 (secure 0 0 0) 0x7f77e4010460 con 0x7f77e8103af0 2024-09-21T02:53:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.177+0000 7f77de7fc640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77d40b6670 0x7f77d40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.177+0000 7f77ecaa6640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77d40b6670 0x7f77d40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.177+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f77e40e71e0 con 0x7f77e8103af0 2024-09-21T02:53:07.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.178+0000 7f77ecaa6640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77d40b6670 0x7f77d40b8b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009230 tx=0x7f77d8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:07.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.180+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77e40ad890 con 0x7f77e8103af0 2024-09-21T02:53:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:06 smithi165 ceph-mon[34818]: pgmap v331: 97 pgs: 97 active+clean; 656 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-21T02:53:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:06 smithi178 ceph-mon[34759]: pgmap v331: 97 pgs: 97 active+clean; 656 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-21T02:53:07.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:53:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.392+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f77e8063800 con 0x7f77d40b6670 2024-09-21T02:53:07.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.400+0000 7f77de7fc640 1 -- 172.21.15.31:0/4084763522 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f77e8063800 con 0x7f77d40b6670 2024-09-21T02:53:07.401 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-09-21T02:53:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77d40b6670 msgr2=0x7f77d40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77d40b6670 0x7f77d40b8b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009230 tx=0x7f77d8031400 comp rx=0 tx=0).stop 2024-09-21T02:53:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 msgr2=0x7f77e819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 0x7f77e819a470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f77e400d950 tx=0x7f77e400de10 comp rx=0 tx=0).stop 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 shutdown_connections 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77d40b6670 0x7f77d40b8b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77e8109420 0x7f77e819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77e8109d70 0x7f77e819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 --2- 172.21.15.31:0/4084763522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e8103af0 0x7f77e819a470 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 >> 172.21.15.31:0/4084763522 conn(0x7f77e80ff570 msgr2=0x7f77e8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 shutdown_connections 2024-09-21T02:53:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:07.403+0000 7f77ef50b640 1 -- 172.21.15.31:0/4084763522 wait complete. 2024-09-21T02:53:07.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T02:53:07.483 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T02:53:07.497 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T02:53:07.527 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:53:07.527 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T02:53:07.527 INFO:tasks.workunit:timeout=3h 2024-09-21T02:53:07.527 INFO:tasks.workunit:cleanup=True 2024-09-21T02:53:07.528 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:53:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T02:53:07.544 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 02:51:09.639858134 +0000 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 02:51:09.639858134 +0000 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 02:52:56.508673729 +0000 2024-09-21T02:53:07.545 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 02:51:09.639858134 +0000 2024-09-21T02:53:07.545 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T02:53:07.545 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T02:53:07.618 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:07.621 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:53:07.678 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T02:53:07.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:07 smithi031 ceph-mon[31209]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:07.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:07 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:53:07.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:07 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:07.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:07 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:53:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:07 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:07 smithi031 ceph-mon[31209]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 -- 172.21.15.31:0/2348227071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5078113eb0 msgr2=0x7f50781142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 --2- 172.21.15.31:0/2348227071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5078113eb0 0x7f50781142c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5060009900 tx=0x7f506002a410 comp rx=0 tx=0).stop 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 -- 172.21.15.31:0/2348227071 shutdown_connections 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 --2- 172.21.15.31:0/2348227071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078114800 0x7f507811b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 --2- 172.21.15.31:0/2348227071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5078113eb0 0x7f50781142c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 --2- 172.21.15.31:0/2348227071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f507810e5c0 0x7f507810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 -- 172.21.15.31:0/2348227071 >> 172.21.15.31:0/2348227071 conn(0x7f507810a3c0 msgr2=0x7f507810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 -- 172.21.15.31:0/2348227071 shutdown_connections 2024-09-21T02:53:08.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.118+0000 7f507ccb8640 1 -- 172.21.15.31:0/2348227071 wait complete. 2024-09-21T02:53:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 Processor -- start 2024-09-21T02:53:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 -- start start 2024-09-21T02:53:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 0x7f50781a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078113eb0 0x7f50781a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5078114800 0x7f50781aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f507811e490 con 0x7f507810e5c0 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f507811e310 con 0x7f5078113eb0 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.119+0000 7f507ccb8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f507811e610 con 0x7f5078114800 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 0x7f50781a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 0x7f50781a4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43500/0 (socket says 172.21.15.31:43500) 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 -- 172.21.15.31:0/30066803 learned_addr learned my addr 172.21.15.31:0/30066803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f506ffff640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078113eb0 0x7f50781a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f5076d9d640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5078114800 0x7f50781aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 -- 172.21.15.31:0/30066803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5078114800 msgr2=0x7f50781aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5078114800 0x7f50781aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 -- 172.21.15.31:0/30066803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078113eb0 msgr2=0x7f50781a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078113eb0 0x7f50781a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 -- 172.21.15.31:0/30066803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781aae10 con 0x7f507810e5c0 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f506ffff640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078113eb0 0x7f50781a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f5076d9d640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5078114800 0x7f50781aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507659c640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 0x7f50781a4ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f506800d770 tx=0x7f506800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f506f7fe640 1 -- 172.21.15.31:0/30066803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068017070 con 0x7f507810e5c0 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781ab100 con 0x7f507810e5c0 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f506f7fe640 1 -- 172.21.15.31:0/30066803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50680054d0 con 0x7f507810e5c0 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f506f7fe640 1 -- 172.21.15.31:0/30066803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068004c30 con 0x7f507810e5c0 2024-09-21T02:53:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.120+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781b1860 con 0x7f507810e5c0 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.123+0000 7f508322c740 1 Processor -- start 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.123+0000 7f508322c740 1 -- start start 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.123+0000 7f506f7fe640 1 -- 172.21.15.31:0/30066803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f5068010460 con 0x7f507810e5c0 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.123+0000 7f508322c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901431270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f508322c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x5599014317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f508322c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x559901431cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f508322c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5599013a7e50 con 0x5599010611b0 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f508322c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559901397280 con 0x55990108a070 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f508322c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55990139b090 con 0x559901069390 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x5599014317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x5599014317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43506/0 (socket says 172.21.15.31:43506) 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 -- 172.21.15.31:0/532500287 learned_addr learned my addr 172.21.15.31:0/532500287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f506ffff640 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x559901431cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f5076d9d640 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901431270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 -- 172.21.15.31:0/532500287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 msgr2=0x559901431cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x559901431cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 -- 172.21.15.31:0/532500287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 msgr2=0x559901431270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901431270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 -- 172.21.15.31:0/532500287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599010f5650 con 0x5599010611b0 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f507659c640 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x5599014317b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f50680e49e0 tx=0x7f50680ef9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9144f2f04674f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:08.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.124+0000 7f506f7fe640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50640b6680 0x7f50640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.125+0000 7f506cff9640 1 -- 172.21.15.31:0/532500287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680fb070 con 0x5599010611b0 2024-09-21T02:53:08.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.125+0000 7f5076d9d640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50640b6680 0x7f50640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.125+0000 7f506cff9640 1 -- 172.21.15.31:0/532500287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50680acc80 con 0x5599010611b0 2024-09-21T02:53:08.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.125+0000 7f506cff9640 1 -- 172.21.15.31:0/532500287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680a9010 con 0x5599010611b0 2024-09-21T02:53:08.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.125+0000 7f506f7fe640 1 -- 172.21.15.31:0/30066803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f50680e7300 con 0x7f507810e5c0 2024-09-21T02:53:08.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 -- 172.21.15.31:0/532500287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 msgr2=0x5599014317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f5076d9d640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50640b6680 0x7f50640b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5070005fd0 tx=0x7f5070009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x5599014317b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f50680e49e0 tx=0x7f50680ef9f0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 -- 172.21.15.31:0/532500287 shutdown_connections 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x559901431cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x5599014317b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 --2- 172.21.15.31:0/532500287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901431270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 -- 172.21.15.31:0/532500287 >> 172.21.15.31:0/532500287 conn(0x55990107e460 msgr2=0x55990142f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 -- 172.21.15.31:0/532500287 shutdown_connections 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 -- 172.21.15.31:0/532500287 wait complete. 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 -- start start 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.126+0000 7f508322c740 1 Processor -- start 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599010611b0 0x55990143c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990143ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901448d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5599011f8700 con 0x559901069390 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559901332110 con 0x55990108a070 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55990143fa50 con 0x5599010611b0 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f508322c740 10 client.0 _ll_register_callbacks cb 0x55990134b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990143ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990143ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43514/0 (socket says 172.21.15.31:43514) 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 -- 172.21.15.31:0/2945337050 learned_addr learned my addr 172.21.15.31:0/2945337050 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f507659c640 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599010611b0 0x55990143c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 -- 172.21.15.31:0/2945337050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599010611b0 msgr2=0x55990143c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599010611b0 0x55990143c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 -- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 msgr2=0x559901448d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901448d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.127+0000 7f506ffff640 1 -- 172.21.15.31:0/2945337050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990135fc40 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f506ffff640 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990143ce70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5060000c00 tx=0x7f50600317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d29dc3a19cc5a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f505dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f505dffb640 1 -- 172.21.15.31:0/2945337050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060035040 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f505dffb640 1 -- 172.21.15.31:0/2945337050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5060004240 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50680acc80 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f505dffb640 1 -- 172.21.15.31:0/2945337050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5060004440 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f508322c740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55990144b730 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f505dffb640 1 -- 172.21.15.31:0/2945337050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50680acc80 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f505dffb640 1 -- 172.21.15.31:0/2945337050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f506004c020 con 0x559901069390 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f508322c740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f508322c740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.128+0000 7f508322c740 20 client.15612 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.129+0000 7f508322c740 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5599010489f0 0x55990144ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.129+0000 7f508322c740 4 client.15612 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.129+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599013a9da0 con 0x5599010489f0 2024-09-21T02:53:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.129+0000 7f507659c640 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5599010489f0 0x55990144ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.129+0000 7f507659c640 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5599010489f0 0x55990144ec80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45034887a1119b0e server_cookie=b04a4548f6ef241f in_seq=0 out_seq=0 2024-09-21T02:53:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.129+0000 7f505dffb640 10 client.15612 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f505dffb640 1 -- 172.21.15.31:0/2945337050 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599013a9da0 con 0x5599010489f0 2024-09-21T02:53:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f505dffb640 10 client.15612 handle_command_reply: tid=0 2024-09-21T02:53:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 client.15612 shutdown 2024-09-21T02:53:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 msgr2=0x55990143ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990143ce70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5060000c00 tx=0x7f50600317a0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 shutdown_connections 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5599010489f0 0x55990144ec80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901448d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990143ce70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 --2- 172.21.15.31:0/2945337050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5599010611b0 0x55990143c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 >> 172.21.15.31:0/2945337050 conn(0x55990107e460 msgr2=0x559901434400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f505dffb640 0 client.15612 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.130+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 shutdown_connections 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.131+0000 7f508322c740 1 -- 172.21.15.31:0/2945337050 wait complete. 2024-09-21T02:53:08.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.131+0000 7f508322c740 20 client.15612 trim_cache size 0 max 16384 2024-09-21T02:53:08.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 Processor -- start 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 -- start start 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5599010611b0 0x55990143cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990142eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55990108a070 0x559901448c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50781ba050 con 0x559901069390 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50781bbb90 con 0x5599010611b0 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.148+0000 7f508322c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50781ba720 con 0x55990108a070 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990142eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990142eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43516/0 (socket says 172.21.15.31:43516) 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 -- 172.21.15.31:0/3818024953 learned_addr learned my addr 172.21.15.31:0/3818024953 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f507659c640 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5599010611b0 0x55990143cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f5076d9d640 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55990108a070 0x559901448c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 -- 172.21.15.31:0/3818024953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55990108a070 msgr2=0x559901448c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55990108a070 0x559901448c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 -- 172.21.15.31:0/3818024953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5599010611b0 msgr2=0x55990143cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5599010611b0 0x55990143cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 -- 172.21.15.31:0/3818024953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990143b890 con 0x559901069390 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f506ffff640 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990142eac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f506002a930 tx=0x7f506002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67082826f4e94234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f505ffff640 1 -- 172.21.15.31:0/3818024953 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060035040 con 0x559901069390 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f505ffff640 1 -- 172.21.15.31:0/3818024953 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5060053020 con 0x559901069390 2024-09-21T02:53:08.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.149+0000 7f505ffff640 1 -- 172.21.15.31:0/3818024953 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060003b00 con 0x559901069390 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 -- 172.21.15.31:0/3818024953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 msgr2=0x55990142eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990142eac0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f506002a930 tx=0x7f506002a410 comp rx=0 tx=0).stop 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 -- 172.21.15.31:0/3818024953 shutdown_connections 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55990108a070 0x559901448c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559901069390 0x55990142eac0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 --2- 172.21.15.31:0/3818024953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5599010611b0 0x55990143cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 -- 172.21.15.31:0/3818024953 >> 172.21.15.31:0/3818024953 conn(0x55990107e460 msgr2=0x55990143c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.150+0000 7f508322c740 1 -- 172.21.15.31:0/3818024953 shutdown_connections 2024-09-21T02:53:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 -- 172.21.15.31:0/3818024953 wait complete. 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 -- start start 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 Processor -- start 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x559901445520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x5599014409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901440f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50781bb4c0 con 0x5599010611b0 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50781badf0 con 0x55990108a070 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50781b8030 con 0x559901069390 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f508322c740 10 client.0 _ll_register_callbacks cb 0x7f50600314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.151+0000 7f507659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x559901445520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x559901445520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43522/0 (socket says 172.21.15.31:43522) 2024-09-21T02:53:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 -- 172.21.15.31:0/1009494669 learned_addr learned my addr 172.21.15.31:0/1009494669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f5076d9d640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901440f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f506ffff640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x5599014409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 -- 172.21.15.31:0/1009494669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 msgr2=0x5599014409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x5599014409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 -- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 msgr2=0x559901440f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901440f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 -- 172.21.15.31:0/1009494669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559901441700 con 0x5599010611b0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f507659c640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x559901445520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50680a9740 tx=0x7f5068009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b236734ce0c07f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f505dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f505dffb640 1 -- 172.21.15.31:0/1009494669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506805dbf0 con 0x5599010611b0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f505dffb640 1 -- 172.21.15.31:0/1009494669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f50680eac20 con 0x5599010611b0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.152+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5060053020 con 0x5599010611b0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f505dffb640 1 -- 172.21.15.31:0/1009494669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f506805e870 con 0x5599010611b0 2024-09-21T02:53:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 10 client.15618 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599014c1e10 con 0x5599010611b0 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f505dffb640 1 -- 172.21.15.31:0/1009494669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5060053020 con 0x5599010611b0 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f505dffb640 1 -- 172.21.15.31:0/1009494669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f50680e4cf0 con 0x5599010611b0 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 20 client.15618 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55990101dda0 0x5599014c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 4 client.15618 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599013a5850 con 0x55990101dda0 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.153+0000 7f506ffff640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55990101dda0 0x5599014c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f505dffb640 10 client.15618 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f506ffff640 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55990101dda0 0x5599014c4160 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646a6414f7f95c51 server_cookie=84c4267940655db4 in_seq=0 out_seq=0 2024-09-21T02:53:08.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f505dffb640 1 -- 172.21.15.31:0/1009494669 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x5599013a5850 con 0x55990101dda0 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f505dffb640 10 client.15618 handle_command_reply: tid=0 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 client.15618 shutdown 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 msgr2=0x559901445520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x559901445520 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50680a9740 tx=0x7f5068009f90 comp rx=0 tx=0).stop 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 shutdown_connections 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55990101dda0 0x5599014c4160 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559901069390 0x5599014409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f505dffb640 0 client.15618 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55990108a070 0x559901440f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 --2- 172.21.15.31:0/1009494669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5599010611b0 0x559901445520 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 >> 172.21.15.31:0/1009494669 conn(0x55990107e460 msgr2=0x55990142ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 shutdown_connections 2024-09-21T02:53:08.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.154+0000 7f508322c740 1 -- 172.21.15.31:0/1009494669 wait complete. 2024-09-21T02:53:08.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f508322c740 20 client.15618 trim_cache size 0 max 16384 2024-09-21T02:53:08.161 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:53:08.161 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:53:08.161 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:53:08.161 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "2733a6e0-8560-4410-bda8-b9913220c0b1", 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:53:08.162 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:53:08.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:53:08.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:53:08.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50640b6680 msgr2=0x7f50640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50640b6680 0x7f50640b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5070005fd0 tx=0x7f5070009800 comp rx=0 tx=0).stop 2024-09-21T02:53:08.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 msgr2=0x7f50781a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 0x7f50781a4ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f506800d770 tx=0x7f506800dc30 comp rx=0 tx=0).stop 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 shutdown_connections 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50640b6680 0x7f50640b8b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5078114800 0x7f50781aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5078113eb0 0x7f50781a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 --2- 172.21.15.31:0/30066803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f507810e5c0 0x7f50781a4ea0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 >> 172.21.15.31:0/30066803 conn(0x7f507810a3c0 msgr2=0x7f5078111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 shutdown_connections 2024-09-21T02:53:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:08.155+0000 7f507ccb8640 1 -- 172.21.15.31:0/30066803 wait complete. 2024-09-21T02:53:08.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:07 smithi165 ceph-mon[34818]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:08.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:07 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:53:08.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:07 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:08.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:07 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:53:08.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:07 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:08.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:07 smithi165 ceph-mon[34818]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:08.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:07 smithi178 ceph-mon[34759]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:08.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:07 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:53:08.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:07 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:08.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:07 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:53:08.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:07 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:08.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:07 smithi178 ceph-mon[34759]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:08.938 INFO:tasks.cephfs.filesystem:scrub status for tag:2733a6e0-8560-4410-bda8-b9913220c0b1 - {'path': '/', 'tag': '2733a6e0-8560-4410-bda8-b9913220c0b1', 'options': 'recursive,force'} 2024-09-21T02:53:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:08 smithi165 ceph-mon[34818]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:08 smithi165 ceph-mon[34818]: pgmap v332: 97 pgs: 97 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 92 KiB/s wr, 25 op/s 2024-09-21T02:53:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:08 smithi165 ceph-mon[34818]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:08 smithi165 ceph-mon[34818]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T02:53:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:08 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:08 smithi178 ceph-mon[34759]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:08 smithi178 ceph-mon[34759]: pgmap v332: 97 pgs: 97 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 92 KiB/s wr, 25 op/s 2024-09-21T02:53:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:08 smithi178 ceph-mon[34759]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:08 smithi178 ceph-mon[34759]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T02:53:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:08 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:08 smithi031 ceph-mon[31209]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:08 smithi031 ceph-mon[31209]: pgmap v332: 97 pgs: 97 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 92 KiB/s wr, 25 op/s 2024-09-21T02:53:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:08 smithi031 ceph-mon[31209]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:08 smithi031 ceph-mon[31209]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T02:53:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:08 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:10.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:10.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:09 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:10.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:09 smithi031 ceph-mon[31209]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 165 KiB/s wr, 32 op/s 2024-09-21T02:53:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:09 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:10.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:09 smithi165 ceph-mon[34818]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 165 KiB/s wr, 32 op/s 2024-09-21T02:53:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:09 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:10.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:09 smithi178 ceph-mon[34759]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 165 KiB/s wr, 32 op/s 2024-09-21T02:53:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:11.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:11.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:11.479 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:11.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:11.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:11.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:53:12.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:12 smithi165 ceph-mon[34818]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 165 KiB/s wr, 31 op/s 2024-09-21T02:53:12.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:12 smithi178 ceph-mon[34759]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 165 KiB/s wr, 31 op/s 2024-09-21T02:53:12.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:12 smithi031 ceph-mon[31209]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 165 KiB/s wr, 31 op/s 2024-09-21T02:53:14.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:14 smithi031 ceph-mon[31209]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 58 op/s 2024-09-21T02:53:14.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:14 smithi165 ceph-mon[34818]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 58 op/s 2024-09-21T02:53:14.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:14 smithi178 ceph-mon[34759]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 176 KiB/s wr, 58 op/s 2024-09-21T02:53:16.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:16 smithi031 ceph-mon[31209]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T02:53:16.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:16 smithi165 ceph-mon[34818]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T02:53:16.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:16 smithi178 ceph-mon[34759]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T02:53:18.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:18 smithi031 ceph-mon[31209]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T02:53:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:18 smithi165 ceph-mon[34818]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T02:53:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:18 smithi178 ceph-mon[34759]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T02:53:20.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:20 smithi031 ceph-mon[31209]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 44 op/s 2024-09-21T02:53:20.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:20 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:53:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:20 smithi165 ceph-mon[34818]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 44 op/s 2024-09-21T02:53:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:20 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:53:20.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:20 smithi178 ceph-mon[34759]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 44 op/s 2024-09-21T02:53:20.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:20 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:53:22.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:22 smithi031 ceph-mon[31209]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 35 op/s 2024-09-21T02:53:22.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:22 smithi165 ceph-mon[34818]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 35 op/s 2024-09-21T02:53:22.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:22 smithi178 ceph-mon[34759]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 KiB/s wr, 35 op/s 2024-09-21T02:53:24.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:24 smithi031 ceph-mon[31209]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 KiB/s wr, 36 op/s 2024-09-21T02:53:24.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:24 smithi165 ceph-mon[34818]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 KiB/s wr, 36 op/s 2024-09-21T02:53:24.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:24 smithi178 ceph-mon[34759]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 KiB/s wr, 36 op/s 2024-09-21T02:53:26.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:26 smithi031 ceph-mon[31209]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:53:26.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:26 smithi165 ceph-mon[34818]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:53:26.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:26 smithi178 ceph-mon[34759]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:53:28.529 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:28 smithi178 ceph-mon[34759]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:53:28.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:28 smithi031 ceph-mon[31209]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:53:28.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:28 smithi165 ceph-mon[34818]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:53:30.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:30 smithi031 ceph-mon[31209]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:53:30.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:30 smithi165 ceph-mon[34818]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:53:30.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:30 smithi178 ceph-mon[34759]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:53:32.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:32 smithi031 ceph-mon[31209]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T02:53:32.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:32 smithi165 ceph-mon[34818]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T02:53:32.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:32 smithi178 ceph-mon[34759]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T02:53:34.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:34 smithi031 ceph-mon[31209]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T02:53:34.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:34 smithi165 ceph-mon[34818]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T02:53:34.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:34 smithi178 ceph-mon[34759]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T02:53:36.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:36 smithi031 ceph-mon[31209]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:53:36.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:36 smithi165 ceph-mon[34818]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:53:36.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:36 smithi178 ceph-mon[34759]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:53:38.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:38 smithi031 ceph-mon[31209]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:53:38.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:38 smithi165 ceph-mon[34818]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:53:38.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:38 smithi178 ceph-mon[34759]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:53:38.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:53:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 -- 172.21.15.31:0/2402851664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48109e00 msgr2=0x7fca48110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/2402851664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48109e00 0x7fca48110630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ca0 tx=0x7fca4002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 -- 172.21.15.31:0/2402851664 shutdown_connections 2024-09-21T02:53:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/2402851664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca481094b0 0x7fca481098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/2402851664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48109e00 0x7fca48110630 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/2402851664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48103bc0 0x7fca48103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 -- 172.21.15.31:0/2402851664 >> 172.21.15.31:0/2402851664 conn(0x7fca4807b910 msgr2=0x7fca4807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 -- 172.21.15.31:0/2402851664 shutdown_connections 2024-09-21T02:53:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.799+0000 7fca4f1d9640 1 -- 172.21.15.31:0/2402851664 wait complete. 2024-09-21T02:53:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.800+0000 7fca4f1d9640 1 Processor -- start 2024-09-21T02:53:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.800+0000 7fca4f1d9640 1 -- start start 2024-09-21T02:53:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4f1d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 0x7fca4819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4f1d9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca481094b0 0x7fca4819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4f1d9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca48109e00 0x7fca4819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4f1d9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca48113ac0 con 0x7fca48103bc0 2024-09-21T02:53:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4f1d9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca48113940 con 0x7fca481094b0 2024-09-21T02:53:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4f1d9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca48113c40 con 0x7fca48109e00 2024-09-21T02:53:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4cf75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 0x7fca4819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4d776640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca48109e00 0x7fca4819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4cf75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 0x7fca4819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37580/0 (socket says 172.21.15.31:37580) 2024-09-21T02:53:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca4cf75640 1 -- 172.21.15.31:0/445685542 learned_addr learned my addr 172.21.15.31:0/445685542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.801+0000 7fca47fff640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca481094b0 0x7fca4819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4cf75640 1 -- 172.21.15.31:0/445685542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca48109e00 msgr2=0x7fca4819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4cf75640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca48109e00 0x7fca4819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4cf75640 1 -- 172.21.15.31:0/445685542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca481094b0 msgr2=0x7fca4819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4cf75640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca481094b0 0x7fca4819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4cf75640 1 -- 172.21.15.31:0/445685542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca481a0460 con 0x7fca48103bc0 2024-09-21T02:53:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4cf75640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 0x7fca4819a4f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00d950 tx=0x7fca3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca45ffb640 1 -- 172.21.15.31:0/445685542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c017070 con 0x7fca48103bc0 2024-09-21T02:53:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca481a0750 con 0x7fca48103bc0 2024-09-21T02:53:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.802+0000 7fca45ffb640 1 -- 172.21.15.31:0/445685542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca3c009c30 con 0x7fca48103bc0 2024-09-21T02:53:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.803+0000 7fca45ffb640 1 -- 172.21.15.31:0/445685542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00f620 con 0x7fca48103bc0 2024-09-21T02:53:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.804+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca481a6eb0 con 0x7fca48103bc0 2024-09-21T02:53:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.807+0000 7fca45ffb640 1 -- 172.21.15.31:0/445685542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fca3c00f7c0 con 0x7fca48103bc0 2024-09-21T02:53:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.807+0000 7fca5574d740 1 Processor -- start 2024-09-21T02:53:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.807+0000 7fca5574d740 1 -- start start 2024-09-21T02:53:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca5574d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb07d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca5574d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb07d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca5574d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb07dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca5574d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5582baff3e50 con 0x5582bacb5390 2024-09-21T02:53:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca5574d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5582bafe3280 con 0x5582bacad1b0 2024-09-21T02:53:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca5574d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5582bafe7090 con 0x5582bacd6070 2024-09-21T02:53:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb07dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb07dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37588/0 (socket says 172.21.15.31:37588) 2024-09-21T02:53:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca47fff640 1 -- 172.21.15.31:0/2613449360 learned_addr learned my addr 172.21.15.31:0/2613449360 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb07d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.808+0000 7fca4d776640 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb07d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca45ffb640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fca380b6680 0x7fca380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca45ffb640 1 -- 172.21.15.31:0/445685542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fca3c0e7290 con 0x7fca48103bc0 2024-09-21T02:53:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca4cf75640 1 -- 172.21.15.31:0/2613449360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 msgr2=0x5582bb07d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb07d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca4d776640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fca380b6680 0x7fca380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca4cf75640 1 -- 172.21.15.31:0/2613449360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 msgr2=0x5582bb07dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb07dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca4cf75640 1 -- 172.21.15.31:0/2613449360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bad41650 con 0x5582bacad1b0 2024-09-21T02:53:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.809+0000 7fca47fff640 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb07dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.810+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb07d7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0e4a00 tx=0x7fca3c010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3483f6b7b566325b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.810+0000 7fca4d776640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fca380b6680 0x7fca380b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca4002f500 tx=0x7fca4002a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.810+0000 7fca32ffd640 1 -- 172.21.15.31:0/2613449360 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c020030 con 0x5582bacad1b0 2024-09-21T02:53:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.810+0000 7fca32ffd640 1 -- 172.21.15.31:0/2613449360 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca3c0accc0 con 0x5582bacad1b0 2024-09-21T02:53:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.810+0000 7fca32ffd640 1 -- 172.21.15.31:0/2613449360 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0a8d60 con 0x5582bacad1b0 2024-09-21T02:53:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 -- 172.21.15.31:0/2613449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 msgr2=0x5582bb07d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb07d7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0e4a00 tx=0x7fca3c010cb0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 -- 172.21.15.31:0/2613449360 shutdown_connections 2024-09-21T02:53:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb07dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb07d7b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 --2- 172.21.15.31:0/2613449360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb07d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 -- 172.21.15.31:0/2613449360 >> 172.21.15.31:0/2613449360 conn(0x5582bacca460 msgr2=0x5582bb07b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 -- 172.21.15.31:0/2613449360 shutdown_connections 2024-09-21T02:53:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.811+0000 7fca5574d740 1 -- 172.21.15.31:0/2613449360 wait complete. 2024-09-21T02:53:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 -- start start 2024-09-21T02:53:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 Processor -- start 2024-09-21T02:53:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb088960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacb5390 0x5582bb088ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 0x5582bb094ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5582bae44700 con 0x5582bacd6070 2024-09-21T02:53:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5582baf7e110 con 0x5582bacb5390 2024-09-21T02:53:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5582bb08b9f0 con 0x5582bacad1b0 2024-09-21T02:53:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca5574d740 10 client.0 _ll_register_callbacks cb 0x5582baf97990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.812+0000 7fca4d776640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 0x5582bb094ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4cf75640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb088960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca47fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacb5390 0x5582bb088ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 0x5582bb094ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37596/0 (socket says 172.21.15.31:37596) 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4cf75640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb088960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:40646/0 (socket says 172.21.15.31:40646) 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 -- 172.21.15.31:0/2511635572 learned_addr learned my addr 172.21.15.31:0/2511635572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 -- 172.21.15.31:0/2511635572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 msgr2=0x5582bb088960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb088960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 -- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacb5390 msgr2=0x5582bb088ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacb5390 0x5582bb088ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 -- 172.21.15.31:0/2511635572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bafabc40 con 0x5582bacd6070 2024-09-21T02:53:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca4d776640 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 0x5582bb094ce0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca40035000 tx=0x7fca400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd994166dae1a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.813+0000 7fca27fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.814+0000 7fca27fff640 1 -- 172.21.15.31:0/2511635572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40004450 con 0x5582bacd6070 2024-09-21T02:53:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.814+0000 7fca27fff640 1 -- 172.21.15.31:0/2511635572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fca400045f0 con 0x5582bacd6070 2024-09-21T02:53:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.814+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca3c0accc0 con 0x5582bacd6070 2024-09-21T02:53:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.814+0000 7fca27fff640 1 -- 172.21.15.31:0/2511635572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca4005c050 con 0x5582bacd6070 2024-09-21T02:53:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.814+0000 7fca5574d740 10 client.15630 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.814+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582bb097730 con 0x5582bacd6070 2024-09-21T02:53:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca27fff640 1 -- 172.21.15.31:0/2511635572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca40036050 con 0x5582bacd6070 2024-09-21T02:53:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca27fff640 1 -- 172.21.15.31:0/2511635572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca4004a050 con 0x5582bacd6070 2024-09-21T02:53:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca5574d740 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca5574d740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca5574d740 20 client.15630 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca5574d740 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac949f0 0x5582bb09aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca5574d740 4 client.15630 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fca3c0accc0 con 0x5582bac949f0 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.815+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac949f0 0x5582bb09aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.816+0000 7fca27fff640 10 client.15630 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.816+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac949f0 0x5582bb09aba0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38dc36a31fd65c7 server_cookie=eb71661716090b4a in_seq=0 out_seq=0 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.816+0000 7fca27fff640 1 -- 172.21.15.31:0/2511635572 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fca3c0accc0 con 0x5582bac949f0 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.816+0000 7fca27fff640 10 client.15630 handle_command_reply: tid=0 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.816+0000 7fca5574d740 1 client.15630 shutdown 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 msgr2=0x5582bb094ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 0x5582bb094ce0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fca40035000 tx=0x7fca400095c0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 shutdown_connections 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac949f0 0x5582bb09aba0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacd6070 0x5582bb094ce0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacb5390 0x5582bb088ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 --2- 172.21.15.31:0/2511635572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb088960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 >> 172.21.15.31:0/2511635572 conn(0x5582bacca460 msgr2=0x5582bb080400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca27fff640 0 client.15630 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 shutdown_connections 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 1 -- 172.21.15.31:0/2511635572 wait complete. 2024-09-21T02:53:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.817+0000 7fca5574d740 20 client.15630 trim_cache size 0 max 16384 2024-09-21T02:53:39.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 Processor -- start 2024-09-21T02:53:39.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 -- start start 2024-09-21T02:53:39.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb080a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb088710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb088c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b1a90 con 0x5582bacb5390 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b35d0 con 0x5582bacad1b0 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca5574d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b2160 con 0x5582bacd6070 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.853+0000 7fca47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb088710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb088710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37604/0 (socket says 172.21.15.31:37604) 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 -- 172.21.15.31:0/2229129917 learned_addr learned my addr 172.21.15.31:0/2229129917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca4d776640 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb088c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca4cf75640 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb080a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 -- 172.21.15.31:0/2229129917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 msgr2=0x5582bb088c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb088c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 -- 172.21.15.31:0/2229129917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 msgr2=0x5582bb080a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb080a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 -- 172.21.15.31:0/2229129917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bb0800d0 con 0x5582bacb5390 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca4d776640 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb088c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.854+0000 7fca47fff640 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb088710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b9c0 tx=0x7fca3400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4593c546bd336e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.855+0000 7fca327fc640 1 -- 172.21.15.31:0/2229129917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34003c10 con 0x5582bacb5390 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.855+0000 7fca327fc640 1 -- 172.21.15.31:0/2229129917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca34003db0 con 0x5582bacb5390 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.855+0000 7fca327fc640 1 -- 172.21.15.31:0/2229129917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34004e30 con 0x5582bacb5390 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 -- 172.21.15.31:0/2229129917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 msgr2=0x5582bb088710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb088710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b9c0 tx=0x7fca3400be80 comp rx=0 tx=0).stop 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 -- 172.21.15.31:0/2229129917 shutdown_connections 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacd6070 0x5582bb088c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb088710 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 --2- 172.21.15.31:0/2229129917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacad1b0 0x5582bb080a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 -- 172.21.15.31:0/2229129917 >> 172.21.15.31:0/2229129917 conn(0x5582bacca460 msgr2=0x5582bb080370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.856+0000 7fca5574d740 1 -- 172.21.15.31:0/2229129917 shutdown_connections 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.857+0000 7fca5574d740 1 -- 172.21.15.31:0/2229129917 wait complete. 2024-09-21T02:53:39.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.857+0000 7fca5574d740 1 -- start start 2024-09-21T02:53:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.858+0000 7fca5574d740 1 Processor -- start 2024-09-21T02:53:39.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.858+0000 7fca5574d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb08cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca5574d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb08d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca5574d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacd6070 0x5582bb10cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca5574d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b2f00 con 0x5582bacb5390 2024-09-21T02:53:39.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca5574d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca481b2830 con 0x5582bacd6070 2024-09-21T02:53:39.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca4cf75640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb08cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca4cf75640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb08cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:40662/0 (socket says 172.21.15.31:40662) 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca4cf75640 1 -- 172.21.15.31:0/3373478122 learned_addr learned my addr 172.21.15.31:0/3373478122 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca47fff640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb08d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca4d776640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacd6070 0x5582bb10cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca481afa90 con 0x5582bacad1b0 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.859+0000 7fca5574d740 10 client.0 _ll_register_callbacks cb 0x7fca40009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca4d776640 1 -- 172.21.15.31:0/3373478122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 msgr2=0x5582bb08cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca4d776640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb08cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca4d776640 1 -- 172.21.15.31:0/3373478122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 msgr2=0x5582bb08d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca4d776640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb08d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca4d776640 1 -- 172.21.15.31:0/3373478122 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bb08dcd0 con 0x5582bacd6070 2024-09-21T02:53:39.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca47fff640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb08d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca4d776640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacd6070 0x5582bb10cd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca4002ade0 tx=0x7fca40045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e205457f0819cdb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca27fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.860+0000 7fca27fff640 1 -- 172.21.15.31:0/3373478122 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40065060 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.861+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca34003db0 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.861+0000 7fca27fff640 1 -- 172.21.15.31:0/3373478122 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fca40004250 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.861+0000 7fca27fff640 1 -- 172.21.15.31:0/3373478122 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca40031760 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.861+0000 7fca5574d740 10 client.25268 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.861+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582bb10e2a0 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.861+0000 7fca27fff640 1 -- 172.21.15.31:0/3373478122 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca34003db0 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca27fff640 1 -- 172.21.15.31:0/3373478122 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca4004a040 con 0x5582bacd6070 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca5574d740 10 client.25268 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca5574d740 10 client.25268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:39.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca5574d740 20 client.25268 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca5574d740 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac69da0 0x5582bb1109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca5574d740 4 client.25268 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582baff5da0 con 0x5582bac69da0 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.862+0000 7fca4cf75640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac69da0 0x5582bb1109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca27fff640 10 client.25268 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca4cf75640 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac69da0 0x5582bb1109d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6cdb90abd57962 server_cookie=a4011ac97909902a in_seq=0 out_seq=0 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca27fff640 1 -- 172.21.15.31:0/3373478122 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x5582baff5da0 con 0x5582bac69da0 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca27fff640 10 client.25268 handle_command_reply: tid=0 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca5574d740 1 client.25268 shutdown 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacd6070 msgr2=0x5582bb10cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca5574d740 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacd6070 0x5582bb10cd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca4002ade0 tx=0x7fca40045000 comp rx=0 tx=0).stop 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 shutdown_connections 2024-09-21T02:53:39.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca5574d740 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5582bac69da0 0x5582bb1109d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.863+0000 7fca5574d740 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582bacd6070 0x5582bb10cd30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca5574d740 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582bacb5390 0x5582bb08d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca5574d740 1 --2- 172.21.15.31:0/3373478122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582bacad1b0 0x5582bb08cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 >> 172.21.15.31:0/3373478122 conn(0x5582bacca460 msgr2=0x5582bb086de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca27fff640 0 client.25268 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 shutdown_connections 2024-09-21T02:53:39.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca5574d740 1 -- 172.21.15.31:0/3373478122 wait complete. 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.864+0000 7fca5574d740 20 client.25268 trim_cache size 0 max 16384 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:53:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:53:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:53:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:39.872 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fca380b6680 msgr2=0x7fca380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fca380b6680 0x7fca380b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca4002f500 tx=0x7fca4002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 msgr2=0x7fca4819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 0x7fca4819a4f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00d950 tx=0x7fca3c00de10 comp rx=0 tx=0).stop 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 shutdown_connections 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fca380b6680 0x7fca380b8b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca48109e00 0x7fca4819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca481094b0 0x7fca4819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 --2- 172.21.15.31:0/445685542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca48103bc0 0x7fca4819a4f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 >> 172.21.15.31:0/445685542 conn(0x7fca4807b910 msgr2=0x7fca48107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 shutdown_connections 2024-09-21T02:53:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:39.865+0000 7fca4f1d9640 1 -- 172.21.15.31:0/445685542 wait complete. 2024-09-21T02:53:40.544 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:40 smithi031 ceph-mon[31209]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:53:40.574 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:53:40.574 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:53:40.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:40 smithi165 ceph-mon[34818]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:53:40.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:40 smithi178 ceph-mon[34759]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:53:40.852 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:41.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.395+0000 7f6fcba38640 1 -- 172.21.15.31:0/795157519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4108cf0 msgr2=0x7f6fc410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.395+0000 7f6fcba38640 1 --2- 172.21.15.31:0/795157519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4108cf0 0x7f6fc410f520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009ca0 tx=0x7f6fbc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.395+0000 7f6fcba38640 1 -- 172.21.15.31:0/795157519 shutdown_connections 2024-09-21T02:53:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.395+0000 7f6fcba38640 1 --2- 172.21.15.31:0/795157519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 0x7f6fc41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 --2- 172.21.15.31:0/795157519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4108cf0 0x7f6fc410f520 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 --2- 172.21.15.31:0/795157519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc4102a60 0x7f6fc4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 -- 172.21.15.31:0/795157519 >> 172.21.15.31:0/795157519 conn(0x7f6fc4079730 msgr2=0x7f6fc4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 -- 172.21.15.31:0/795157519 shutdown_connections 2024-09-21T02:53:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 -- 172.21.15.31:0/795157519 wait complete. 2024-09-21T02:53:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 Processor -- start 2024-09-21T02:53:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.396+0000 7f6fcba38640 1 -- start start 2024-09-21T02:53:41.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fcba38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 0x7f6fc419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fcba38640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 0x7f6fc419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fcba38640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc4108cf0 0x7f6fc419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fcba38640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4112960 con 0x7f6fc4102a60 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fcba38640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc41127e0 con 0x7f6fc4108cf0 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fcba38640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4112ae0 con 0x7f6fc41083a0 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fc97d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 0x7f6fc419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:41.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fc97d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 0x7f6fc419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37616/0 (socket says 172.21.15.31:37616) 2024-09-21T02:53:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fc97d4640 1 -- 172.21.15.31:0/2304211925 learned_addr learned my addr 172.21.15.31:0/2304211925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fc8fd3640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 0x7f6fc419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.397+0000 7f6fc9fd5640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc4108cf0 0x7f6fc419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc8fd3640 1 -- 172.21.15.31:0/2304211925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc4108cf0 msgr2=0x7f6fc419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc8fd3640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc4108cf0 0x7f6fc419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc8fd3640 1 -- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 msgr2=0x7f6fc419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:41.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc8fd3640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 0x7f6fc419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc8fd3640 1 -- 172.21.15.31:0/2304211925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc41a0350 con 0x7f6fc41083a0 2024-09-21T02:53:41.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc97d4640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 0x7f6fc419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:41.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fc8fd3640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 0x7f6fc419aa40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e9c0 tx=0x7f6fb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400ccc0 con 0x7f6fc41083a0 2024-09-21T02:53:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc41a0640 con 0x7f6fc41083a0 2024-09-21T02:53:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb40108e0 con 0x7f6fc41083a0 2024-09-21T02:53:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40187c0 con 0x7f6fc41083a0 2024-09-21T02:53:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.398+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc40ff550 con 0x7f6fc41083a0 2024-09-21T02:53:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.399+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc4103ce0 con 0x7f6fc41083a0 2024-09-21T02:53:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.401+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f6fb4010430 con 0x7f6fc41083a0 2024-09-21T02:53:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.402+0000 7f6fba7fc640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6fb00b6730 0x7f6fb00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.402+0000 7f6fc97d4640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6fb00b6730 0x7f6fb00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.402+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6fb40e7d90 con 0x7f6fc41083a0 2024-09-21T02:53:41.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.403+0000 7f6fc97d4640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6fb00b6730 0x7f6fb00b8c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000f4d0 tx=0x7f6fc000a380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:41.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.405+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fb40b26e0 con 0x7f6fc41083a0 2024-09-21T02:53:41.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.704+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fc4102e70 con 0x7f6fc41083a0 2024-09-21T02:53:41.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.705+0000 7f6fba7fc640 1 -- 172.21.15.31:0/2304211925 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f6fb40b7620 con 0x7f6fc41083a0 2024-09-21T02:53:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:53:41.708 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:53:41.708 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6fb00b6730 msgr2=0x7f6fb00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6fb00b6730 0x7f6fb00b8c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000f4d0 tx=0x7f6fc000a380 comp rx=0 tx=0).stop 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 msgr2=0x7f6fc419aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 0x7f6fc419aa40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e9c0 tx=0x7f6fb400ee80 comp rx=0 tx=0).stop 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 shutdown_connections 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6fb00b6730 0x7f6fb00b8c40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fc41083a0 0x7f6fc419aa40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc4108cf0 0x7f6fc419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 --2- 172.21.15.31:0/2304211925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc4102a60 0x7f6fc419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.707+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 >> 172.21.15.31:0/2304211925 conn(0x7f6fc4079730 msgr2=0x7f6fc406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.708+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 shutdown_connections 2024-09-21T02:53:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:41.708+0000 7f6fcba38640 1 -- 172.21.15.31:0/2304211925 wait complete. 2024-09-21T02:53:42.496 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:42 smithi031 ceph-mon[31209]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:53:42.496 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:42 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2304211925' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:42.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:53:42.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:42 smithi165 ceph-mon[34818]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:53:42.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:42 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2304211925' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:42.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:42 smithi178 ceph-mon[34759]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T02:53:42.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:42 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2304211925' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:53:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 -- 172.21.15.31:0/1992098531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 msgr2=0x7f25bc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/1992098531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc102e70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009900 tx=0x7f25ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 -- 172.21.15.31:0/1992098531 shutdown_connections 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/1992098531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 0x7f25bc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/1992098531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25bc1083a0 0x7f25bc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/1992098531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc102e70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 -- 172.21.15.31:0/1992098531 >> 172.21.15.31:0/1992098531 conn(0x7f25bc079730 msgr2=0x7f25bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 -- 172.21.15.31:0/1992098531 shutdown_connections 2024-09-21T02:53:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.403+0000 7f25c1f3c640 1 -- 172.21.15.31:0/1992098531 wait complete. 2024-09-21T02:53:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.404+0000 7f25c1f3c640 1 Processor -- start 2024-09-21T02:53:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.404+0000 7f25c1f3c640 1 -- start start 2024-09-21T02:53:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25c1f3c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25c1f3c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25bc1083a0 0x7f25bc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25c1f3c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 0x7f25bc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25c1f3c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1129b0 con 0x7f25bc102a60 2024-09-21T02:53:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25c1f3c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc112830 con 0x7f25bc1083a0 2024-09-21T02:53:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25c1f3c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc112b30 con 0x7f25bc108cf0 2024-09-21T02:53:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37626/0 (socket says 172.21.15.31:37626) 2024-09-21T02:53:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25bb7fe640 1 -- 172.21.15.31:0/329146826 learned_addr learned my addr 172.21.15.31:0/329146826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25baffd640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25bc1083a0 0x7f25bc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.405+0000 7f25bbfff640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 0x7f25bc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bb7fe640 1 -- 172.21.15.31:0/329146826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 msgr2=0x7f25bc19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 0x7f25bc19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bb7fe640 1 -- 172.21.15.31:0/329146826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25bc1083a0 msgr2=0x7f25bc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25bc1083a0 0x7f25bc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bb7fe640 1 -- 172.21.15.31:0/329146826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc1a0390 con 0x7f25bc102a60 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bbfff640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 0x7f25bc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc19a420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005ab0 tx=0x7f25ac003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25b8ff9640 1 -- 172.21.15.31:0/329146826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac03f070 con 0x7f25bc102a60 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25b8ff9640 1 -- 172.21.15.31:0/329146826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25ac031d60 con 0x7f25bc102a60 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25b8ff9640 1 -- 172.21.15.31:0/329146826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac035030 con 0x7f25bc102a60 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1a0620 con 0x7f25bc102a60 2024-09-21T02:53:43.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.406+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc1a6de0 con 0x7f25bc102a60 2024-09-21T02:53:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.409+0000 7f25b8ff9640 1 -- 172.21.15.31:0/329146826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f25ac042070 con 0x7f25bc102a60 2024-09-21T02:53:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 Processor -- start 2024-09-21T02:53:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 -- start start 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873ef46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873ef4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873ef5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556873e6aea0 con 0x556873b4d070 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556873e5a2d0 con 0x556873b241b0 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.410+0000 7f25c84b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556873e5e0e0 con 0x556873b2c390 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873ef4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873ef4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:56602/0 (socket says 172.21.15.31:56602) 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25bb7fe640 1 -- 172.21.15.31:0/3918120935 learned_addr learned my addr 172.21.15.31:0/3918120935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25baffd640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873ef5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25bbfff640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873ef46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25baffd640 1 -- 172.21.15.31:0/3918120935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 msgr2=0x556873ef4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25baffd640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873ef4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25baffd640 1 -- 172.21.15.31:0/3918120935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 msgr2=0x556873ef46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25baffd640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873ef46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25baffd640 1 -- 172.21.15.31:0/3918120935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556873bb8650 con 0x556873b2c390 2024-09-21T02:53:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.411+0000 7f25bbfff640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873ef46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25b8ff9640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25a40b6700 0x7f25a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873ef4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25baffd640 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873ef5120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25bc06aaa0 tx=0x7f25b000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba0773e19676afb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25bbfff640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25a40b6700 0x7f25a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25b8ff9640 1 -- 172.21.15.31:0/329146826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25ac1080a0 con 0x7f25bc102a60 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25a1ffb640 1 -- 172.21.15.31:0/3918120935 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0015070 con 0x556873b2c390 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25a1ffb640 1 -- 172.21.15.31:0/3918120935 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b00035f0 con 0x556873b2c390 2024-09-21T02:53:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.412+0000 7f25a1ffb640 1 -- 172.21.15.31:0/3918120935 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0011640 con 0x556873b2c390 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.413+0000 7f25c84b0740 1 -- 172.21.15.31:0/3918120935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 msgr2=0x556873ef5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.413+0000 7f25bbfff640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25a40b6700 0x7f25a40b8c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25b40095f0 tx=0x7f25b4009580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.413+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873ef5120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25bc06aaa0 tx=0x7f25b000f730 comp rx=0 tx=0).stop 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.413+0000 7f25c84b0740 1 -- 172.21.15.31:0/3918120935 shutdown_connections 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.413+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873ef5120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.414+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873ef4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.414+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3918120935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873ef46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.414+0000 7f25c84b0740 1 -- 172.21.15.31:0/3918120935 >> 172.21.15.31:0/3918120935 conn(0x556873b41460 msgr2=0x556873ef2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.414+0000 7f25c84b0740 1 -- 172.21.15.31:0/3918120935 shutdown_connections 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.414+0000 7f25c84b0740 1 -- 172.21.15.31:0/3918120935 wait complete. 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.415+0000 7f25c84b0740 1 -- start start 2024-09-21T02:53:43.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.415+0000 7f25c84b0740 1 Processor -- start 2024-09-21T02:53:43.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.415+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873effdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.415+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873f002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873f0c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25c84b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556873cbb700 con 0x556873b4d070 2024-09-21T02:53:43.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25c84b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556873df5160 con 0x556873b241b0 2024-09-21T02:53:43.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25c84b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556873f02e80 con 0x556873b2c390 2024-09-21T02:53:43.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25c84b0740 10 client.0 _ll_register_callbacks cb 0x556873e0e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:43.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873effdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873effdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:56618/0 (socket says 172.21.15.31:56618) 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bb7fe640 1 -- 172.21.15.31:0/907346156 learned_addr learned my addr 172.21.15.31:0/907346156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bbfff640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873f0c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25baffd640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873f002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bbfff640 1 -- 172.21.15.31:0/907346156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 msgr2=0x556873f002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bbfff640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873f002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bbfff640 1 -- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 msgr2=0x556873effdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bbfff640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873effdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.416+0000 7f25bbfff640 1 -- 172.21.15.31:0/907346156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556873e22c90 con 0x556873b4d070 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f25bbfff640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873f0c1a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005fd0 tx=0x7f25b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260232b69c96b68a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f259affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f259affd640 1 -- 172.21.15.31:0/907346156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b402c060 con 0x556873b4d070 2024-09-21T02:53:43.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f259affd640 1 -- 172.21.15.31:0/907346156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25b40106f0 con 0x556873b4d070 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b00035f0 con 0x556873b4d070 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f259affd640 1 -- 172.21.15.31:0/907346156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25b40108f0 con 0x556873b4d070 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f25c84b0740 10 client.15654 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.417+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556873f13030 con 0x556873b4d070 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f259affd640 1 -- 172.21.15.31:0/907346156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b00035f0 con 0x556873b4d070 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f259affd640 1 -- 172.21.15.31:0/907346156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25b400f040 con 0x556873b4d070 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25c84b0740 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25c84b0740 10 client.15654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25c84b0740 20 client.15654 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25c84b0740 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873b0b9f0 0x556873f16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25c84b0740 4 client.15654 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:43.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556873e6cdf0 con 0x556873b0b9f0 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.418+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873b0b9f0 0x556873f16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.419+0000 7f259affd640 10 client.15654 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.419+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873b0b9f0 0x556873f16580 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=930bf924a1f6146d server_cookie=ad9daec7e0079d17 in_seq=0 out_seq=0 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f259affd640 1 -- 172.21.15.31:0/907346156 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556873e6cdf0 con 0x556873b0b9f0 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f259affd640 10 client.15654 handle_command_reply: tid=0 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 client.15654 shutdown 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 msgr2=0x556873f0c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873f0c1a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005fd0 tx=0x7f25b4004a40 comp rx=0 tx=0).stop 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 shutdown_connections 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873b0b9f0 0x556873f16580 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b2c390 0x556873f002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b4d070 0x556873f0c1a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 --2- 172.21.15.31:0/907346156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b241b0 0x556873effdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f259affd640 0 client.15654 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 >> 172.21.15.31:0/907346156 conn(0x556873b41460 msgr2=0x556873ef7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.420+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 shutdown_connections 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.421+0000 7f25c84b0740 1 -- 172.21.15.31:0/907346156 wait complete. 2024-09-21T02:53:43.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.421+0000 7f25c84b0740 20 client.15654 trim_cache size 0 max 16384 2024-09-21T02:53:43.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 Processor -- start 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 -- start start 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b241b0 0x556873f00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 0x556873ef77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b4d070 0x556873ef7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1ad440 con 0x556873b2c390 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1aef80 con 0x556873b4d070 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25c84b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1adb10 con 0x556873b241b0 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25baffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 0x556873ef77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25baffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 0x556873ef77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37652/0 (socket says 172.21.15.31:37652) 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25baffd640 1 -- 172.21.15.31:0/504972919 learned_addr learned my addr 172.21.15.31:0/504972919 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:43.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25bbfff640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b4d070 0x556873ef7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.447+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b241b0 0x556873f00270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bbfff640 1 -- 172.21.15.31:0/504972919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b241b0 msgr2=0x556873f00270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bbfff640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b241b0 0x556873f00270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bbfff640 1 -- 172.21.15.31:0/504972919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 msgr2=0x556873ef77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bbfff640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 0x556873ef77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bbfff640 1 -- 172.21.15.31:0/504972919 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556873efb640 con 0x556873b4d070 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b241b0 0x556873f00270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25baffd640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 0x556873ef77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:43.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25bbfff640 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b4d070 0x556873ef7ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25b4017040 tx=0x7f25b401d5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db87667ebe9a0f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25a17fa640 1 -- 172.21.15.31:0/504972919 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b40106f0 con 0x556873b4d070 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25a17fa640 1 -- 172.21.15.31:0/504972919 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b401de50 con 0x556873b4d070 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.448+0000 7f25a17fa640 1 -- 172.21.15.31:0/504972919 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b402fe40 con 0x556873b4d070 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 -- 172.21.15.31:0/504972919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b4d070 msgr2=0x556873ef7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b4d070 0x556873ef7ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25b4017040 tx=0x7f25b401d5a0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 -- 172.21.15.31:0/504972919 shutdown_connections 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b4d070 0x556873ef7ce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b2c390 0x556873ef77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 --2- 172.21.15.31:0/504972919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b241b0 0x556873f00270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 -- 172.21.15.31:0/504972919 >> 172.21.15.31:0/504972919 conn(0x556873b41460 msgr2=0x556873effb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 -- 172.21.15.31:0/504972919 shutdown_connections 2024-09-21T02:53:43.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 -- 172.21.15.31:0/504972919 wait complete. 2024-09-21T02:53:43.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.450+0000 7f25c84b0740 1 -- start start 2024-09-21T02:53:43.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 Processor -- start 2024-09-21T02:53:43.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 0x556873f04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b2c390 0x556873f04750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b4d070 0x556873f04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1ae8b0 con 0x556873b241b0 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1ae1e0 con 0x556873b2c390 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc1ab420 con 0x556873b4d070 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25c84b0740 10 client.0 _ll_register_callbacks cb 0x7f25b40098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 0x556873f04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 0x556873f04210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37654/0 (socket says 172.21.15.31:37654) 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.451+0000 7f25bb7fe640 1 -- 172.21.15.31:0/3923421859 learned_addr learned my addr 172.21.15.31:0/3923421859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25baffd640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b2c390 0x556873f04750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bbfff640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b4d070 0x556873f04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bb7fe640 1 -- 172.21.15.31:0/3923421859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b4d070 msgr2=0x556873f04c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b4d070 0x556873f04c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bb7fe640 1 -- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b2c390 msgr2=0x556873f04750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b2c390 0x556873f04750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bb7fe640 1 -- 172.21.15.31:0/3923421859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556873f80070 con 0x556873b241b0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bbfff640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b4d070 0x556873f04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25bb7fe640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 0x556873f04210 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f25ac105e10 tx=0x7f25ac10bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e03a7606bf156d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f259affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f259affd640 1 -- 172.21.15.31:0/3923421859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac049040 con 0x556873b241b0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f259affd640 1 -- 172.21.15.31:0/3923421859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25ac0ca180 con 0x556873b241b0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b401de50 con 0x556873b241b0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f259affd640 1 -- 172.21.15.31:0/3923421859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25ac0c9db0 con 0x556873b241b0 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.452+0000 7f25c84b0740 10 client.15660 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:43.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556873f81350 con 0x556873b241b0 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f259affd640 1 -- 172.21.15.31:0/3923421859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b401de50 con 0x556873b241b0 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f259affd640 1 -- 172.21.15.31:0/3923421859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25ac0b3ce0 con 0x556873b241b0 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 10 client.15660 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 20 client.15660 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873ae0da0 0x556873f83b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 4 client.15660 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:53:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.453+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556873e688a0 con 0x556873ae0da0 2024-09-21T02:53:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.454+0000 7f25baffd640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873ae0da0 0x556873f83b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.454+0000 7f259affd640 10 client.15660 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:53:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.454+0000 7f25baffd640 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873ae0da0 0x556873f83b40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8716840b923de1d server_cookie=931d9fe3af6404b5 in_seq=0 out_seq=0 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.454+0000 7f259affd640 1 -- 172.21.15.31:0/3923421859 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556873e688a0 con 0x556873ae0da0 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.454+0000 7f259affd640 10 client.15660 handle_command_reply: tid=0 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.454+0000 7f25c84b0740 1 client.15660 shutdown 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 msgr2=0x556873f04210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 0x556873f04210 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f25ac105e10 tx=0x7f25ac10bbf0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 shutdown_connections 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x556873ae0da0 0x556873f83b40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556873b4d070 0x556873f04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556873b2c390 0x556873f04750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 --2- 172.21.15.31:0/3923421859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556873b241b0 0x556873f04210 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 >> 172.21.15.31:0/3923421859 conn(0x556873b41460 msgr2=0x556873efe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f259affd640 0 client.15660 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:53:43.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 shutdown_connections 2024-09-21T02:53:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 1 -- 172.21.15.31:0/3923421859 wait complete. 2024-09-21T02:53:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.455+0000 7f25c84b0740 20 client.15660 trim_cache size 0 max 16384 2024-09-21T02:53:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:53:43.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25a40b6700 msgr2=0x7f25a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25a40b6700 0x7f25a40b8c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25b40095f0 tx=0x7f25b4009580 comp rx=0 tx=0).stop 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 msgr2=0x7f25bc19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc19a420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005ab0 tx=0x7f25ac003cd0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 shutdown_connections 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25a40b6700 0x7f25a40b8c10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc108cf0 0x7f25bc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25bc1083a0 0x7f25bc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 --2- 172.21.15.31:0/329146826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25bc102a60 0x7f25bc19a420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 >> 172.21.15.31:0/329146826 conn(0x7f25bc079730 msgr2=0x7f25bc100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 shutdown_connections 2024-09-21T02:53:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:43.456+0000 7f25c1f3c640 1 -- 172.21.15.31:0/329146826 wait complete. 2024-09-21T02:53:44.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:53:44.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:44 smithi031 ceph-mon[31209]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:44.609 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:44.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:44 smithi165 ceph-mon[34818]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:44.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:44 smithi178 ceph-mon[34759]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 -- 172.21.15.31:0/686028474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f901049b0 msgr2=0x7f4f90104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 --2- 172.21.15.31:0/686028474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f901049b0 0x7f4f90104dc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009900 tx=0x7f4f8c02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 -- 172.21.15.31:0/686028474 shutdown_connections 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 --2- 172.21.15.31:0/686028474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f90105300 0x7f4f9010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 --2- 172.21.15.31:0/686028474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f9010de10 0x7f4f90110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 --2- 172.21.15.31:0/686028474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f901049b0 0x7f4f90104dc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 -- 172.21.15.31:0/686028474 >> 172.21.15.31:0/686028474 conn(0x7f4f90079730 msgr2=0x7f4f90079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 -- 172.21.15.31:0/686028474 shutdown_connections 2024-09-21T02:53:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.162+0000 7f4f98775640 1 -- 172.21.15.31:0/686028474 wait complete. 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.163+0000 7f4f98775640 1 Processor -- start 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.163+0000 7f4f98775640 1 -- start start 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.163+0000 7f4f98775640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 0x7f4f9019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.163+0000 7f4f98775640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f90105300 0x7f4f9019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.163+0000 7f4f98775640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f9010de10 0x7f4f9019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f98775640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90113ac0 con 0x7f4f9010de10 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f98775640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90113940 con 0x7f4f90105300 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f98775640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90113c40 con 0x7f4f901049b0 2024-09-21T02:53:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 0x7f4f9019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 0x7f4f9019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:40756/0 (socket says 172.21.15.31:40756) 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 -- 172.21.15.31:0/1284420807 learned_addr learned my addr 172.21.15.31:0/1284420807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96d12640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f9010de10 0x7f4f9019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f95d10640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f90105300 0x7f4f9019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 -- 172.21.15.31:0/1284420807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f90105300 msgr2=0x7f4f9019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f90105300 0x7f4f9019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 -- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f9010de10 msgr2=0x7f4f9019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f9010de10 0x7f4f9019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96511640 1 -- 172.21.15.31:0/1284420807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f901a0440 con 0x7f4f901049b0 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.164+0000 7f4f96d12640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f9010de10 0x7f4f9019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:45.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.165+0000 7f4f96511640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 0x7f4f9019a4d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005ab0 tx=0x7f4f8c003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.165+0000 7f4f877fe640 1 -- 172.21.15.31:0/1284420807 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c03f070 con 0x7f4f901049b0 2024-09-21T02:53:45.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.165+0000 7f4f877fe640 1 -- 172.21.15.31:0/1284420807 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c031d60 con 0x7f4f901049b0 2024-09-21T02:53:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.165+0000 7f4f877fe640 1 -- 172.21.15.31:0/1284420807 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c035030 con 0x7f4f901049b0 2024-09-21T02:53:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.165+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f901a06d0 con 0x7f4f901049b0 2024-09-21T02:53:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.166+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f901025d0 con 0x7f4f901049b0 2024-09-21T02:53:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.168+0000 7f4f9ece9740 1 Processor -- start 2024-09-21T02:53:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.168+0000 7f4f9ece9740 1 -- start start 2024-09-21T02:53:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631498e070 0x556314d356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d35be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d36120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556314cabea0 con 0x55631496d390 2024-09-21T02:53:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556314c9b2d0 con 0x55631498e070 2024-09-21T02:53:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556314c9f0e0 con 0x5563149651b0 2024-09-21T02:53:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f95d10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d36120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f95d10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d36120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37688/0 (socket says 172.21.15.31:37688) 2024-09-21T02:53:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f95d10640 1 -- 172.21.15.31:0/3037846913 learned_addr learned my addr 172.21.15.31:0/3037846913 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f96d12640 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631498e070 0x556314d356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f877fe640 1 -- 172.21.15.31:0/1284420807 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f4f8c042070 con 0x7f4f901049b0 2024-09-21T02:53:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f95d10640 1 -- 172.21.15.31:0/3037846913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 msgr2=0x556314d35be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.169+0000 7f4f95d10640 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d35be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f95d10640 1 -- 172.21.15.31:0/3037846913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631498e070 msgr2=0x556314d356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f95d10640 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631498e070 0x556314d356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f95d10640 1 -- 172.21.15.31:0/3037846913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563149f9650 con 0x55631496d390 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f96d12640 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631498e070 0x556314d356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f95d10640 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d36120 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000bc80 tx=0x7f4f800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2566b8c5603ee19b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f84ff9640 1 -- 172.21.15.31:0/3037846913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004220 con 0x55631496d390 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f84ff9640 1 -- 172.21.15.31:0/3037846913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8000be30 con 0x55631496d390 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.170+0000 7f4f84ff9640 1 -- 172.21.15.31:0/3037846913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f800117b0 con 0x55631496d390 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.171+0000 7f4f877fe640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f7c0b6680 0x7f4f7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.171+0000 7f4f96d12640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f7c0b6680 0x7f4f7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.171+0000 7f4f877fe640 1 -- 172.21.15.31:0/1284420807 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4f8c107ed0 con 0x7f4f901049b0 2024-09-21T02:53:45.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.171+0000 7f4f9ece9740 1 -- 172.21.15.31:0/3037846913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 msgr2=0x556314d36120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.171+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d36120 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000bc80 tx=0x7f4f800107a0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.172+0000 7f4f9ece9740 1 -- 172.21.15.31:0/3037846913 shutdown_connections 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.172+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d35be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.172+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d36120 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.172+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/3037846913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631498e070 0x556314d356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.172+0000 7f4f9ece9740 1 -- 172.21.15.31:0/3037846913 >> 172.21.15.31:0/3037846913 conn(0x556314982460 msgr2=0x556314d33590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.172+0000 7f4f96d12640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f7c0b6680 0x7f4f7c0b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4f88005fd0 tx=0x7f4f88009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 -- 172.21.15.31:0/3037846913 shutdown_connections 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 -- 172.21.15.31:0/3037846913 wait complete. 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 -- start start 2024-09-21T02:53:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 Processor -- start 2024-09-21T02:53:45.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d49ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631496d390 0x556314d4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 0x556314d4f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556314afc700 con 0x55631498e070 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556314c36160 con 0x5563149651b0 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.173+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556314d43e80 con 0x55631496d390 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 0x556314d4f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f9ece9740 10 client.0 _ll_register_callbacks cb 0x556314c4f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 0x556314d4f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37692/0 (socket says 172.21.15.31:37692) 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 -- 172.21.15.31:0/4076090530 learned_addr learned my addr 172.21.15.31:0/4076090530 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96511640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d49ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f95d10640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631496d390 0x556314d4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 -- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631496d390 msgr2=0x556314d4a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631496d390 0x556314d4a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 -- 172.21.15.31:0/4076090530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 msgr2=0x556314d49ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d49ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 -- 172.21.15.31:0/4076090530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556314c63c90 con 0x55631498e070 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f96d12640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 0x556314d4f4f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4f88012cf0 tx=0x7f4f88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8605f0004789b504 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.174+0000 7f4f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f79ffb640 1 -- 172.21.15.31:0/4076090530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8801dd30 con 0x55631498e070 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f79ffb640 1 -- 172.21.15.31:0/4076090530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4f88029480 con 0x55631498e070 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8000be30 con 0x55631498e070 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f79ffb640 1 -- 172.21.15.31:0/4076090530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f88029680 con 0x55631498e070 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 10 client.15672 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556314d584a0 con 0x55631498e070 2024-09-21T02:53:45.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f79ffb640 1 -- 172.21.15.31:0/4076090530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f88029cd0 con 0x55631498e070 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f79ffb640 1 -- 172.21.15.31:0/4076090530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4f8800f040 con 0x55631498e070 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 10 client.15672 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 20 client.15672 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.175+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631494c9f0 0x556314d5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.176+0000 7f4f9ece9740 4 client.15672 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.176+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f8000be30 con 0x55631494c9f0 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.176+0000 7f4f96511640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631494c9f0 0x556314d5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.176+0000 7f4f79ffb640 10 client.15672 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:53:45.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.176+0000 7f4f96511640 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631494c9f0 0x556314d5b910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8ea12cfe349f5944 server_cookie=cec3bc7aee0ad122 in_seq=0 out_seq=0 2024-09-21T02:53:45.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f79ffb640 1 -- 172.21.15.31:0/4076090530 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4f8000be30 con 0x55631494c9f0 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f79ffb640 10 client.15672 handle_command_reply: tid=0 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 client.15672 shutdown 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 msgr2=0x556314d4f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 0x556314d4f4f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4f88012cf0 tx=0x7f4f88004a40 comp rx=0 tx=0).stop 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 shutdown_connections 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631494c9f0 0x556314d5b910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631496d390 0x556314d4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f79ffb640 0 client.15672 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631498e070 0x556314d4f4f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/4076090530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.177+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 >> 172.21.15.31:0/4076090530 conn(0x556314982460 msgr2=0x556314d38880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.178+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 shutdown_connections 2024-09-21T02:53:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.178+0000 7f4f9ece9740 1 -- 172.21.15.31:0/4076090530 wait complete. 2024-09-21T02:53:45.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.178+0000 7f4f9ece9740 20 client.15672 trim_cache size 0 max 16384 2024-09-21T02:53:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 Processor -- start 2024-09-21T02:53:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 -- start start 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631494c9f0 0x556314d4f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901ad480 con 0x55631496d390 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901aefc0 con 0x5563149651b0 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.204+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901adb50 con 0x55631494c9f0 2024-09-21T02:53:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96d12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96d12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d4b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37706/0 (socket says 172.21.15.31:37706) 2024-09-21T02:53:45.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96d12640 1 -- 172.21.15.31:0/2564852395 learned_addr learned my addr 172.21.15.31:0/2564852395 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96511640 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631494c9f0 0x556314d4f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f95d10640 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96511640 1 -- 172.21.15.31:0/2564852395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 msgr2=0x556314d4b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96511640 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d4b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96511640 1 -- 172.21.15.31:0/2564852395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 msgr2=0x556314d4b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96511640 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d4b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.205+0000 7f4f96511640 1 -- 172.21.15.31:0/2564852395 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556314d397b0 con 0x55631494c9f0 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.206+0000 7f4f96d12640 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.206+0000 7f4f96511640 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631494c9f0 0x556314d4f360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c105640 tx=0x7f4f8c03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2aab6b0c0d8cabf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.206+0000 7f4f7bfff640 1 -- 172.21.15.31:0/2564852395 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c049040 con 0x55631494c9f0 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.206+0000 7f4f7bfff640 1 -- 172.21.15.31:0/2564852395 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0cd900 con 0x55631494c9f0 2024-09-21T02:53:45.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.206+0000 7f4f7bfff640 1 -- 172.21.15.31:0/2564852395 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c03b4f0 con 0x55631494c9f0 2024-09-21T02:53:45.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2564852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631494c9f0 msgr2=0x556314d4f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631494c9f0 0x556314d4f360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c105640 tx=0x7f4f8c03b460 comp rx=0 tx=0).stop 2024-09-21T02:53:45.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2564852395 shutdown_connections 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631496d390 0x556314d4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5563149651b0 0x556314d4b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2564852395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55631494c9f0 0x556314d4f360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2564852395 >> 172.21.15.31:0/2564852395 conn(0x556314982460 msgr2=0x556314d48570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.207+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2564852395 shutdown_connections 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.208+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2564852395 wait complete. 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.208+0000 7f4f9ece9740 1 -- start start 2024-09-21T02:53:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.208+0000 7f4f9ece9740 1 Processor -- start 2024-09-21T02:53:45.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 0x556314d4d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d4c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631496d390 0x556314d4c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901ae8f0 con 0x55631494c9f0 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901ae220 con 0x55631496d390 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901ab460 con 0x5563149651b0 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f9ece9740 10 client.0 _ll_register_callbacks cb 0x7f4f88005c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f96511640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 0x556314d4d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f96511640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 0x556314d4d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37720/0 (socket says 172.21.15.31:37720) 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f96511640 1 -- 172.21.15.31:0/2088637878 learned_addr learned my addr 172.21.15.31:0/2088637878 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f96d12640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631496d390 0x556314d4c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.209+0000 7f4f95d10640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d4c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f96511640 1 -- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 msgr2=0x556314d4c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f96511640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d4c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f96511640 1 -- 172.21.15.31:0/2088637878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631496d390 msgr2=0x556314d4c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f96511640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631496d390 0x556314d4c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f96511640 1 -- 172.21.15.31:0/2088637878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556314d4dfd0 con 0x55631494c9f0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f96511640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 0x556314d4d960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c031440 tx=0x7f4f8c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699f573111aefbc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f79ffb640 1 -- 172.21.15.31:0/2088637878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c049040 con 0x55631494c9f0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c0cd900 con 0x55631494c9f0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f79ffb640 1 -- 172.21.15.31:0/2088637878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4f8c0f5240 con 0x55631494c9f0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f79ffb640 1 -- 172.21.15.31:0/2088637878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f8c03b4f0 con 0x55631494c9f0 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f9ece9740 10 client.15678 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.210+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556314dc6d00 con 0x55631494c9f0 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f79ffb640 1 -- 172.21.15.31:0/2088637878 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c0cd900 con 0x55631494c9f0 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f79ffb640 1 -- 172.21.15.31:0/2088637878 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4f8c0b3bc0 con 0x55631494c9f0 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f9ece9740 10 client.15678 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f9ece9740 10 client.15678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f9ece9740 20 client.15678 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631498e070 0x556314dc9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f9ece9740 4 client.15678 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:53:45.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556314caddf0 con 0x55631498e070 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.211+0000 7f4f95d10640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631498e070 0x556314dc9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f95d10640 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631498e070 0x556314dc9280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1eec3b4a52a0a468 server_cookie=e24c48e7735f0208 in_seq=0 out_seq=0 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f79ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f79ffb640 1 -- 172.21.15.31:0/2088637878 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556314caddf0 con 0x55631498e070 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f79ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 client.15678 shutdown 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 msgr2=0x556314d4d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 0x556314d4d960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c031440 tx=0x7f4f8c02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 shutdown_connections 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55631498e070 0x556314dc9280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563149651b0 0x556314d4c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55631496d390 0x556314d4c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 --2- 172.21.15.31:0/2088637878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55631494c9f0 0x556314d4d960 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 >> 172.21.15.31:0/2088637878 conn(0x556314982460 msgr2=0x556314d387a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:45.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f79ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:53:45.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 shutdown_connections 2024-09-21T02:53:45.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.212+0000 7f4f9ece9740 1 -- 172.21.15.31:0/2088637878 wait complete. 2024-09-21T02:53:45.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f9ece9740 20 client.15678 trim_cache size 0 max 16384 2024-09-21T02:53:45.217 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f7c0b6680 msgr2=0x7f4f7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f7c0b6680 0x7f4f7c0b8b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4f88005fd0 tx=0x7f4f88009600 comp rx=0 tx=0).stop 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 msgr2=0x7f4f9019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 0x7f4f9019a4d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005ab0 tx=0x7f4f8c003cd0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 shutdown_connections 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4f7c0b6680 0x7f4f7c0b8b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f9010de10 0x7f4f9019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f90105300 0x7f4f9019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 --2- 172.21.15.31:0/1284420807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f901049b0 0x7f4f9019a4d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:45.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.213+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 >> 172.21.15.31:0/1284420807 conn(0x7f4f90079730 msgr2=0x7f4f90100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.214+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 shutdown_connections 2024-09-21T02:53:45.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:45.214+0000 7f4f98775640 1 -- 172.21.15.31:0/1284420807 wait complete. 2024-09-21T02:53:46.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:53:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:46.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:46 smithi031 ceph-mon[31209]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:46.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:46 smithi165 ceph-mon[34818]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:46.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:46 smithi178 ceph-mon[34759]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T02:53:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.035+0000 7f66a196f640 1 -- 172.21.15.31:0/1663672926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 msgr2=0x7f669c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.035+0000 7f66a196f640 1 --2- 172.21.15.31:0/1663672926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c102380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f668c009980 tx=0x7f668c02e820 comp rx=0 tx=0).stop 2024-09-21T02:53:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 -- 172.21.15.31:0/1663672926 shutdown_connections 2024-09-21T02:53:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 --2- 172.21.15.31:0/1663672926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 0x7f669c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 --2- 172.21.15.31:0/1663672926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f669c109e00 0x7f669c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 --2- 172.21.15.31:0/1663672926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c102380 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 -- 172.21.15.31:0/1663672926 >> 172.21.15.31:0/1663672926 conn(0x7f669c077c60 msgr2=0x7f669c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 -- 172.21.15.31:0/1663672926 shutdown_connections 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.036+0000 7f66a196f640 1 -- 172.21.15.31:0/1663672926 wait complete. 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 Processor -- start 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 -- start start 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 0x7f669c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f669c109e00 0x7f669c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f669c1118d0 con 0x7f669c101f70 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f669c111750 con 0x7f669c109e00 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f66a196f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f669c111a50 con 0x7f669c1094b0 2024-09-21T02:53:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37736/0 (socket says 172.21.15.31:37736) 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.037+0000 7f669affd640 1 -- 172.21.15.31:0/3792827708 learned_addr learned my addr 172.21.15.31:0/3792827708 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669a7fc640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 0x7f669c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f669c109e00 0x7f669c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669affd640 1 -- 172.21.15.31:0/3792827708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 msgr2=0x7f669c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669affd640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 0x7f669c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669affd640 1 -- 172.21.15.31:0/3792827708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f669c109e00 msgr2=0x7f669c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669affd640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f669c109e00 0x7f669c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669affd640 1 -- 172.21.15.31:0/3792827708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c19e1d0 con 0x7f669c101f70 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669a7fc640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 0x7f669c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f669affd640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c198260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f668c004770 tx=0x7f668c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.038+0000 7f66a096d640 1 -- 172.21.15.31:0/3792827708 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c045020 con 0x7f669c101f70 2024-09-21T02:53:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.039+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c19e460 con 0x7f669c101f70 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.039+0000 7f66a096d640 1 -- 172.21.15.31:0/3792827708 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c0044d0 con 0x7f669c101f70 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.039+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c1a4c20 con 0x7f669c101f70 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.039+0000 7f66a096d640 1 -- 172.21.15.31:0/3792827708 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c040a80 con 0x7f669c101f70 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 Processor -- start 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 -- start start 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a096d640 1 -- 172.21.15.31:0/3792827708 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f668c048020 con 0x7f669c101f70 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561281429ea0 con 0x5612810e31b0 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5612814192d0 con 0x56128110c070 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56128141d0e0 con 0x5612810eb390 2024-09-21T02:53:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37742/0 (socket says 172.21.15.31:37742) 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.043+0000 7f669affd640 1 -- 172.21.15.31:0/2510242355 learned_addr learned my addr 172.21.15.31:0/2510242355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669b7fe640 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669a7fc640 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669affd640 1 -- 172.21.15.31:0/2510242355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 msgr2=0x5612814b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669affd640 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669affd640 1 -- 172.21.15.31:0/2510242355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 msgr2=0x5612814b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669affd640 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669affd640 1 -- 172.21.15.31:0/2510242355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561281177650 con 0x5612810e31b0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f66a096d640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f66840b6680 0x7f66840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669affd640 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814b3be0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f668c108930 tx=0x7f668c113950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adfe69ad7bce79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.044+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f66840b6680 0x7f66840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.045+0000 7f6681ffb640 1 -- 172.21.15.31:0/2510242355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0d0f40 con 0x5612810e31b0 2024-09-21T02:53:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.045+0000 7f6681ffb640 1 -- 172.21.15.31:0/2510242355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c0d0c80 con 0x5612810e31b0 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.045+0000 7f6681ffb640 1 -- 172.21.15.31:0/2510242355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0d08e0 con 0x5612810e31b0 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.045+0000 7f66a096d640 1 -- 172.21.15.31:0/3792827708 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f668c10b250 con 0x7f669c101f70 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 -- 172.21.15.31:0/2510242355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 msgr2=0x5612814b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814b3be0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f668c108930 tx=0x7f668c113950 comp rx=0 tx=0).stop 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f66840b6680 0x7f66840b8b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6694005fd0 tx=0x7f6694009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 -- 172.21.15.31:0/2510242355 shutdown_connections 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814b3be0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/2510242355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 -- 172.21.15.31:0/2510242355 >> 172.21.15.31:0/2510242355 conn(0x561281100460 msgr2=0x5612814b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 -- 172.21.15.31:0/2510242355 shutdown_connections 2024-09-21T02:53:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 -- 172.21.15.31:0/2510242355 wait complete. 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.046+0000 7f66a7ee3740 1 -- start start 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 Processor -- start 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56128127a700 con 0x5612810e31b0 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5612813b4160 con 0x56128110c070 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561281429ea0 con 0x5612810eb390 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f66a7ee3740 10 client.0 _ll_register_callbacks cb 0x5612813cd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.047+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814beda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37746/0 (socket says 172.21.15.31:37746) 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 -- 172.21.15.31:0/318164147 learned_addr learned my addr 172.21.15.31:0/318164147 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669a7fc640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669b7fe640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 -- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 msgr2=0x5612814bf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814bf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 -- 172.21.15.31:0/318164147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 msgr2=0x5612814cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 -- 172.21.15.31:0/318164147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612813e1c90 con 0x5612810e31b0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f669affd640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814beda0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f668c000c00 tx=0x7f668c040680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4f644edb4a27b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f667affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f667affd640 1 -- 172.21.15.31:0/318164147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0d0ac0 con 0x5612810e31b0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.048+0000 7f667affd640 1 -- 172.21.15.31:0/318164147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f668c040c20 con 0x5612810e31b0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668c0d0c80 con 0x5612810e31b0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f667affd640 1 -- 172.21.15.31:0/318164147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f668c0d08e0 con 0x5612810e31b0 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612814cd7b0 con 0x5612810e31b0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f667affd640 1 -- 172.21.15.31:0/318164147 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c0daae0 con 0x5612810e31b0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f667affd640 1 -- 172.21.15.31:0/318164147 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f668c0ccac0 con 0x5612810e31b0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 10 client.15690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 20 client.15690 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5612810ca9f0 0x5612814d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 4 client.15690 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.049+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f668c0d0c80 con 0x5612810ca9f0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.050+0000 7f669a7fc640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5612810ca9f0 0x5612814d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.050+0000 7f669a7fc640 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5612810ca9f0 0x5612814d0b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56fbeac60df68f0e server_cookie=24c1b2e63e8a117c in_seq=0 out_seq=0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.050+0000 7f667affd640 10 client.15690 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f667affd640 1 -- 172.21.15.31:0/318164147 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f668c0d0c80 con 0x5612810ca9f0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f667affd640 10 client.15690 handle_command_reply: tid=0 2024-09-21T02:53:47.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 client.15690 shutdown 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 msgr2=0x5612814beda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814beda0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f668c000c00 tx=0x7f668c040680 comp rx=0 tx=0).stop 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 shutdown_connections 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5612810ca9f0 0x5612814d0b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f667affd640 0 client.15690 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810eb390 0x5612814bf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56128110c070 0x5612814cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/318164147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612810e31b0 0x5612814beda0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 >> 172.21.15.31:0/318164147 conn(0x561281100460 msgr2=0x5612814b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 shutdown_connections 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.051+0000 7f66a7ee3740 1 -- 172.21.15.31:0/318164147 wait complete. 2024-09-21T02:53:47.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.052+0000 7f66a7ee3740 20 client.15690 trim_cache size 0 max 16384 2024-09-21T02:53:47.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.065+0000 7f66a7ee3740 1 Processor -- start 2024-09-21T02:53:47.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.065+0000 7f66a7ee3740 1 -- start start 2024-09-21T02:53:47.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612814bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f668800e790 con 0x561281100460 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66880102d0 con 0x5612810eb390 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f668800ee60 con 0x5612810ca9f0 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612814bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f669a7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f669affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612814bdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37750/0 (socket says 172.21.15.31:37750) 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f669affd640 1 -- 172.21.15.31:0/3964006083 learned_addr learned my addr 172.21.15.31:0/3964006083 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:47.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.066+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f669b7fe640 1 -- 172.21.15.31:0/3964006083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 msgr2=0x5612814b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f669b7fe640 1 -- 172.21.15.31:0/3964006083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 msgr2=0x5612814bdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612814bdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f669b7fe640 1 -- 172.21.15.31:0/3964006083 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612814b5d90 con 0x5612810eb390 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f669b7fe640 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814bbf90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6694013d80 tx=0x7f6694016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4418ec0a7028b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f66817fa640 1 -- 172.21.15.31:0/3964006083 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694011560 con 0x5612810eb390 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f66817fa640 1 -- 172.21.15.31:0/3964006083 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6694025070 con 0x5612810eb390 2024-09-21T02:53:47.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.067+0000 7f66817fa640 1 -- 172.21.15.31:0/3964006083 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402aaa0 con 0x5612810eb390 2024-09-21T02:53:47.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 -- 172.21.15.31:0/3964006083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 msgr2=0x5612814bbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814bbf90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6694013d80 tx=0x7f6694016000 comp rx=0 tx=0).stop 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 -- 172.21.15.31:0/3964006083 shutdown_connections 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814bbf90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/3964006083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612814bdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.068+0000 7f66a7ee3740 1 -- 172.21.15.31:0/3964006083 >> 172.21.15.31:0/3964006083 conn(0x5612814cb230 msgr2=0x5612814b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.069+0000 7f66a7ee3740 1 -- 172.21.15.31:0/3964006083 shutdown_connections 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.069+0000 7f66a7ee3740 1 -- 172.21.15.31:0/3964006083 wait complete. 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.069+0000 7f66a7ee3740 1 -- start start 2024-09-21T02:53:47.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.069+0000 7f66a7ee3740 1 Processor -- start 2024-09-21T02:53:47.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612815344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f668800fc00 con 0x561281100460 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f668800f530 con 0x5612810eb390 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f668800c770 con 0x5612810ca9f0 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f66a7ee3740 10 client.0 _ll_register_callbacks cb 0x7f6694009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612815344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669affd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612815344a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37758/0 (socket says 172.21.15.31:37758) 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669b7fe640 1 -- 172.21.15.31:0/990038838 learned_addr learned my addr 172.21.15.31:0/990038838 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669a7fc640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669affd640 1 -- 172.21.15.31:0/990038838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 msgr2=0x5612814c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669affd640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669affd640 1 -- 172.21.15.31:0/990038838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 msgr2=0x5612815344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:47.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669affd640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612815344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.070+0000 7f669affd640 1 -- 172.21.15.31:0/990038838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561281534b10 con 0x5612810ca9f0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f669a7fc640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f669affd640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814c36d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b8040 tx=0x7f668c0407f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d167c712a2e16a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f667affd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f667affd640 1 -- 172.21.15.31:0/990038838 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0d0f40 con 0x5612810ca9f0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f667affd640 1 -- 172.21.15.31:0/990038838 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f668c0d03d0 con 0x5612810ca9f0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694025070 con 0x5612810ca9f0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f667affd640 1 -- 172.21.15.31:0/990038838 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f668c0d0ac0 con 0x5612810ca9f0 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f66a7ee3740 10 client.25372 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:47.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.071+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561281534b10 con 0x5612810ca9f0 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f667affd640 1 -- 172.21.15.31:0/990038838 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6694025070 con 0x5612810ca9f0 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f667affd640 1 -- 172.21.15.31:0/990038838 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f668c0e9080 con 0x5612810ca9f0 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f66a7ee3740 10 client.25372 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f66a7ee3740 10 client.25372 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f66a7ee3740 20 client.25372 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x56128110c070 0x56128153e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f66a7ee3740 4 client.25372 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56128142bdf0 con 0x56128110c070 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.072+0000 7f669a7fc640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x56128110c070 0x56128153e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:47.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f667affd640 10 client.25372 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f669a7fc640 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x56128110c070 0x56128153e300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13fa253d475d926c server_cookie=f0c283112150240 in_seq=0 out_seq=0 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f667affd640 1 -- 172.21.15.31:0/990038838 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56128142bdf0 con 0x56128110c070 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f667affd640 10 client.25372 handle_command_reply: tid=0 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 client.25372 shutdown 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 msgr2=0x5612814c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814c36d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b8040 tx=0x7f668c0407f0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 shutdown_connections 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x56128110c070 0x56128153e300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f667affd640 0 client.25372 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561281100460 0x5612815344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612810eb390 0x5612814c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 --2- 172.21.15.31:0/990038838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612810ca9f0 0x5612814c36d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.073+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 >> 172.21.15.31:0/990038838 conn(0x5612814cb230 msgr2=0x5612814b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:47.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.074+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 shutdown_connections 2024-09-21T02:53:47.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.074+0000 7f66a7ee3740 1 -- 172.21.15.31:0/990038838 wait complete. 2024-09-21T02:53:47.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.074+0000 7f66a7ee3740 20 client.25372 trim_cache size 0 max 16384 2024-09-21T02:53:47.078 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.074+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f66840b6680 msgr2=0x7f66840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.074+0000 7f66a196f640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f66840b6680 0x7f66840b8b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6694005fd0 tx=0x7f6694009600 comp rx=0 tx=0).stop 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 msgr2=0x7f669c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c198260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f668c004770 tx=0x7f668c0042e0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 shutdown_connections 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f66840b6680 0x7f66840b8b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f669c1094b0 0x7f669c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f669c109e00 0x7f669c19daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 --2- 172.21.15.31:0/3792827708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f669c101f70 0x7f669c198260 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:47.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 >> 172.21.15.31:0/3792827708 conn(0x7f669c077c60 msgr2=0x7f669c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:47.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 shutdown_connections 2024-09-21T02:53:47.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:47.075+0000 7f66a196f640 1 -- 172.21.15.31:0/3792827708 wait complete. 2024-09-21T02:53:47.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:53:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:48.548 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:48 smithi031 ceph-mon[31209]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T02:53:48.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 -- 172.21.15.31:0/4249856152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c106c00 msgr2=0x7f721c110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 --2- 172.21.15.31:0/4249856152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c106c00 0x7f721c110690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7214009ca0 tx=0x7f721402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 -- 172.21.15.31:0/4249856152 shutdown_connections 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 --2- 172.21.15.31:0/4249856152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c106c00 0x7f721c110690 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 --2- 172.21.15.31:0/4249856152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 0x7f721c1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 --2- 172.21.15.31:0/4249856152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c102060 0x7f721c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.645+0000 7f7222733640 1 -- 172.21.15.31:0/4249856152 >> 172.21.15.31:0/4249856152 conn(0x7f721c077cc0 msgr2=0x7f721c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.646+0000 7f7222733640 1 -- 172.21.15.31:0/4249856152 shutdown_connections 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.646+0000 7f7222733640 1 -- 172.21.15.31:0/4249856152 wait complete. 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.647+0000 7f7222733640 1 Processor -- start 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.647+0000 7f7222733640 1 -- start start 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.647+0000 7f7222733640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c102060 0x7f721c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.647+0000 7f7222733640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 0x7f721c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f7222733640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c106c00 0x7f721c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f7222733640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f721c113ad0 con 0x7f721c102060 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f7222733640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f721c113950 con 0x7f721c1062b0 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f7222733640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f721c113c50 con 0x7f721c106c00 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c102060 0x7f721c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 0x7f721c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c102060 0x7f721c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37784/0 (socket says 172.21.15.31:37784) 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721bfff640 1 -- 172.21.15.31:0/2034182932 learned_addr learned my addr 172.21.15.31:0/2034182932 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f7220cd0640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c106c00 0x7f721c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721b7fe640 1 -- 172.21.15.31:0/2034182932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c106c00 msgr2=0x7f721c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721b7fe640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c106c00 0x7f721c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721b7fe640 1 -- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c102060 msgr2=0x7f721c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721b7fe640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c102060 0x7f721c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.648+0000 7f721b7fe640 1 -- 172.21.15.31:0/2034182932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c1a03b0 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.649+0000 7f721b7fe640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 0x7f721c19a9e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f721c075b40 tx=0x7f721000db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.649+0000 7f72197fa640 1 -- 172.21.15.31:0/2034182932 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210017070 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.649+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c1a06a0 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.649+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c102ee0 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.649+0000 7f72197fa640 1 -- 172.21.15.31:0/2034182932 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7210009820 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.649+0000 7f72197fa640 1 -- 172.21.15.31:0/2034182932 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210020060 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 Processor -- start 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 -- start start 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f72197fa640 1 -- 172.21.15.31:0/2034182932 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7f7210004b60 con 0x7f721c1062b0 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46d3070 0x55d9a4a82ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a83010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46b2390 0x55d9a4a83550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d9a49f0ea0 con 0x55d9a46b2390 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d9a49e02d0 con 0x55d9a46aa1b0 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.653+0000 7f7228ca7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d9a49e40e0 con 0x55d9a46d3070 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a83010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46b2390 0x55d9a4a83550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a83010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:56762/0 (socket says 172.21.15.31:56762) 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 -- 172.21.15.31:0/2738123735 learned_addr learned my addr 172.21.15.31:0/2738123735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f7220cd0640 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46d3070 0x55d9a4a82ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 -- 172.21.15.31:0/2738123735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46d3070 msgr2=0x55d9a4a82ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46d3070 0x55d9a4a82ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 -- 172.21.15.31:0/2738123735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46b2390 msgr2=0x55d9a4a83550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46b2390 0x55d9a4a83550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 -- 172.21.15.31:0/2738123735 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a473e650 con 0x55d9a46aa1b0 2024-09-21T02:53:48.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721b7fe640 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46b2390 0x55d9a4a83550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.654+0000 7f721bfff640 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a83010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f720c00d470 tx=0x7f720c00d930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6dca5444492f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.655+0000 7f72027fc640 1 -- 172.21.15.31:0/2738123735 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c004220 con 0x55d9a46aa1b0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.655+0000 7f72027fc640 1 -- 172.21.15.31:0/2738123735 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c0043c0 con 0x55d9a46aa1b0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.655+0000 7f72027fc640 1 -- 172.21.15.31:0/2738123735 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c018870 con 0x55d9a46aa1b0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.655+0000 7f72197fa640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72040b6780 0x7f72040b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.655+0000 7f72197fa640 1 -- 172.21.15.31:0/2034182932 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f72100e6d70 con 0x7f721c1062b0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.655+0000 7f7220cd0640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72040b6780 0x7f72040b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.656+0000 7f7228ca7740 1 -- 172.21.15.31:0/2738123735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 msgr2=0x55d9a4a83010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.656+0000 7f7228ca7740 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a83010 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f720c00d470 tx=0x7f720c00d930 comp rx=0 tx=0).stop 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7220cd0640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72040b6780 0x7f72040b8c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f7214009420 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 -- 172.21.15.31:0/2738123735 shutdown_connections 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46b2390 0x55d9a4a83550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a83010 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 --2- 172.21.15.31:0/2738123735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46d3070 0x55d9a4a82ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 -- 172.21.15.31:0/2738123735 >> 172.21.15.31:0/2738123735 conn(0x55d9a46c7460 msgr2=0x55d9a4a809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 -- 172.21.15.31:0/2738123735 shutdown_connections 2024-09-21T02:53:48.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.657+0000 7f7228ca7740 1 -- 172.21.15.31:0/2738123735 wait complete. 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 -- start start 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 Processor -- start 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46b2390 0x55d9a4a8e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46d3070 0x55d9a4a8ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4841700 con 0x55d9a46d3070 2024-09-21T02:53:48.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d9a497b160 con 0x55d9a46aa1b0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.658+0000 7f7228ca7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d9a4a912b0 con 0x55d9a46b2390 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f7228ca7740 10 client.0 _ll_register_callbacks cb 0x55d9a49949e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:56770/0 (socket says 172.21.15.31:56770) 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 -- 172.21.15.31:0/275309357 learned_addr learned my addr 172.21.15.31:0/275309357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721b7fe640 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46b2390 0x55d9a4a8e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 -- 172.21.15.31:0/275309357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46b2390 msgr2=0x55d9a4a8e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46b2390 0x55d9a4a8e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 -- 172.21.15.31:0/275309357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46d3070 msgr2=0x55d9a4a8ec20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46d3070 0x55d9a4a8ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 -- 172.21.15.31:0/275309357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a49a8c90 con 0x55d9a46aa1b0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f721bfff640 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f720c004230 tx=0x7f720c004260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c51b4832efd8526a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.659+0000 7f71fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:53:48.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f71fb7fe640 1 -- 172.21.15.31:0/275309357 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c026020 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f71fb7fe640 1 -- 172.21.15.31:0/275309357 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f720c005450 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720c0043c0 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f71fb7fe640 1 -- 172.21.15.31:0/275309357 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f720c031050 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f7228ca7740 10 client.25337 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9a4a9cfe0 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.660+0000 7f71fb7fe640 1 -- 172.21.15.31:0/275309357 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c003aa0 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f71fb7fe640 1 -- 172.21.15.31:0/275309357 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f720c004c50 con 0x55d9a46aa1b0 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f7228ca7740 10 client.25337 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f7228ca7740 10 client.25337 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f7228ca7740 20 client.25337 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f7228ca7740 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46919f0 0x55d9a4aa0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f7228ca7740 4 client.25337 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:48.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f720c0043c0 con 0x55d9a46919f0 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f721b7fe640 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46919f0 0x55d9a4aa0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.661+0000 7f71fb7fe640 10 client.25337 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.662+0000 7f721b7fe640 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46919f0 0x55d9a4aa0340 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e70a502d1008c55 server_cookie=f4117772fd302cbb in_seq=0 out_seq=0 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f71fb7fe640 1 -- 172.21.15.31:0/275309357 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f720c0043c0 con 0x55d9a46919f0 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f71fb7fe640 10 client.25337 handle_command_reply: tid=0 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 client.25337 shutdown 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 msgr2=0x55d9a4a8e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f720c004230 tx=0x7f720c004260 comp rx=0 tx=0).stop 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 shutdown_connections 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46919f0 0x55d9a4aa0340 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f71fb7fe640 0 client.25337 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46b2390 0x55d9a4a8e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46d3070 0x55d9a4a8ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 --2- 172.21.15.31:0/275309357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8e160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.663+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 >> 172.21.15.31:0/275309357 conn(0x55d9a46c7460 msgr2=0x55d9a4a85c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.664+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 shutdown_connections 2024-09-21T02:53:48.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.664+0000 7f7228ca7740 1 -- 172.21.15.31:0/275309357 wait complete. 2024-09-21T02:53:48.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.664+0000 7f7228ca7740 20 client.25337 trim_cache size 0 max 16384 2024-09-21T02:53:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.678+0000 7f7228ca7740 1 Processor -- start 2024-09-21T02:53:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.678+0000 7f7228ca7740 1 -- start start 2024-09-21T02:53:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 0x55d9a4a84210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4a8b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 0x55d9a4a84210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37800/0 (socket says 172.21.15.31:37800) 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 0x55d9a4a84210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:40888/0 (socket says 172.21.15.31:40888) 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721b7fe640 1 -- 172.21.15.31:0/4280530488 learned_addr learned my addr 172.21.15.31:0/4280530488 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f720800e790 con 0x55d9a46aa1b0 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72080102d0 con 0x55d9a46b2390 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f720800ee60 con 0x55d9a46919f0 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f7220cd0640 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4a8b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 -- 172.21.15.31:0/4280530488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 msgr2=0x55d9a4a8b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4a8b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 -- 172.21.15.31:0/4280530488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 msgr2=0x55d9a4a8b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721bfff640 1 -- 172.21.15.31:0/4280530488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a4a85310 con 0x55d9a46919f0 2024-09-21T02:53:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.679+0000 7f721b7fe640 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.680+0000 7f721bfff640 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 0x55d9a4a84210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f720c004610 tx=0x7f720c001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec555a1f7dcdf029 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:48.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.680+0000 7f7201ffb640 1 -- 172.21.15.31:0/4280530488 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c039070 con 0x55d9a46919f0 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.680+0000 7f7201ffb640 1 -- 172.21.15.31:0/4280530488 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c005700 con 0x55d9a46919f0 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.680+0000 7f7201ffb640 1 -- 172.21.15.31:0/4280530488 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c004680 con 0x55d9a46919f0 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 msgr2=0x55d9a4a84210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 0x55d9a4a84210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f720c004610 tx=0x7f720c001e50 comp rx=0 tx=0).stop 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 shutdown_connections 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4a8b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 --2- 172.21.15.31:0/4280530488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46919f0 0x55d9a4a84210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 >> 172.21.15.31:0/4280530488 conn(0x55d9a46c7460 msgr2=0x55d9a4a83b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 shutdown_connections 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 -- 172.21.15.31:0/4280530488 wait complete. 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.681+0000 7f7228ca7740 1 -- start start 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 Processor -- start 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 0x55d9a4a928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a92df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4b09ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f720800fc00 con 0x55d9a46919f0 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f720800f530 con 0x55d9a46b2390 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.682+0000 7f7228ca7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f720800c770 con 0x55d9a46aa1b0 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f7228ca7740 10 client.0 _ll_register_callbacks cb 0x7f72100ea670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 0x55d9a4a928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 0x55d9a4a928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37802/0 (socket says 172.21.15.31:37802) 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 -- 172.21.15.31:0/513773164 learned_addr learned my addr 172.21.15.31:0/513773164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721b7fe640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a92df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 -- 172.21.15.31:0/513773164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46aa1b0 msgr2=0x55d9a4a92df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a92df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f7220cd0640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4b09ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 -- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 msgr2=0x55d9a4b09ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4b09ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.683+0000 7f721bfff640 1 -- 172.21.15.31:0/513773164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a4a936c0 con 0x55d9a46919f0 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.684+0000 7f721bfff640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 0x55d9a4a928b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f720c022a20 tx=0x7f720c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0f428e1f7aa6c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.684+0000 7f71fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.684+0000 7f71fb7fe640 1 -- 172.21.15.31:0/513773164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c040020 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.684+0000 7f71fb7fe640 1 -- 172.21.15.31:0/513773164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f720c005250 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.684+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9a4a80c10 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.685+0000 7f71fb7fe640 1 -- 172.21.15.31:0/513773164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f720c04b050 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.685+0000 7f7228ca7740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.685+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9a4b0b0a0 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.685+0000 7f71fb7fe640 1 -- 172.21.15.31:0/513773164 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720c004680 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.685+0000 7f71fb7fe640 1 -- 172.21.15.31:0/513773164 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f720c0035f0 con 0x55d9a46919f0 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f7228ca7740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f7228ca7740 10 client.15711 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f7228ca7740 20 client.15711 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f7228ca7740 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46d3070 0x55d9a4b0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:48.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f7228ca7740 4 client.15711 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9a4a80c10 con 0x55d9a46d3070 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.686+0000 7f721b7fe640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46d3070 0x55d9a4b0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.687+0000 7f71fb7fe640 10 client.15711 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.687+0000 7f721b7fe640 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46d3070 0x55d9a4b0d7f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7c44fc39393e94b server_cookie=2ea20f4ce8ad8664 in_seq=0 out_seq=0 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.687+0000 7f71fb7fe640 1 -- 172.21.15.31:0/513773164 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9a4a80c10 con 0x55d9a46d3070 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.687+0000 7f71fb7fe640 10 client.15711 handle_command_reply: tid=0 2024-09-21T02:53:48.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.687+0000 7f7228ca7740 1 client.15711 shutdown 2024-09-21T02:53:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 msgr2=0x55d9a4a928b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 0x55d9a4a928b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f720c022a20 tx=0x7f720c0042e0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 shutdown_connections 2024-09-21T02:53:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55d9a46d3070 0x55d9a4b0d7f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d9a46aa1b0 0x55d9a4a92df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a46b2390 0x55d9a4b09ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.692 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:53:48.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f71fb7fe640 0 client.15711 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:53:48.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 --2- 172.21.15.31:0/513773164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9a46919f0 0x55d9a4a928b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 >> 172.21.15.31:0/513773164 conn(0x55d9a46c7460 msgr2=0x55d9a4a99520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 shutdown_connections 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 1 -- 172.21.15.31:0/513773164 wait complete. 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.688+0000 7f7228ca7740 20 client.15711 trim_cache size 0 max 16384 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72040b6780 msgr2=0x7f72040b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72040b6780 0x7f72040b8c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f7214009420 comp rx=0 tx=0).stop 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 msgr2=0x7f721c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 0x7f721c19a9e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f721c075b40 tx=0x7f721000db30 comp rx=0 tx=0).stop 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 shutdown_connections 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f72040b6780 0x7f72040b8c90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f721c106c00 0x7f721c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f721c1062b0 0x7f721c19a9e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 --2- 172.21.15.31:0/2034182932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f721c102060 0x7f721c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 >> 172.21.15.31:0/2034182932 conn(0x7f721c077cc0 msgr2=0x7f721c1074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 shutdown_connections 2024-09-21T02:53:48.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:48.689+0000 7f7222733640 1 -- 172.21.15.31:0/2034182932 wait complete. 2024-09-21T02:53:48.695 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:48 smithi165 ceph-mon[34818]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T02:53:48.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:48 smithi178 ceph-mon[34759]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T02:53:49.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:53:49.685 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.278+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3585279879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e4107b30 msgr2=0x7fb6e4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.278+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3585279879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e4107b30 0x7fb6e4107f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009900 tx=0x7fb6d402a410 comp rx=0 tx=0).stop 2024-09-21T02:53:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.278+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3585279879 shutdown_connections 2024-09-21T02:53:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.278+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3585279879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4108480 0x7fb6e410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.278+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3585279879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e4107b30 0x7fb6e4107f40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.279+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3585279879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e41021f0 0x7fb6e4102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.279+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3585279879 >> 172.21.15.31:0/3585279879 conn(0x7fb6e4077c60 msgr2=0x7fb6e40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.279+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3585279879 shutdown_connections 2024-09-21T02:53:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.279+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3585279879 wait complete. 2024-09-21T02:53:50.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 Processor -- start 2024-09-21T02:53:50.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 -- start start 2024-09-21T02:53:50.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 0x7fb6e4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4107b30 0x7fb6e406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4108480 0x7fb6e406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6e9d35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 0x7fb6e4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6e9d35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 0x7fb6e4074f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43450/0 (socket says 172.21.15.31:43450) 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4111910 con 0x7fb6e41021f0 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4111790 con 0x7fb6e4107b30 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3907359820 learned_addr learned my addr 172.21.15.31:0/3907359820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4111a90 con 0x7fb6e4108480 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6e9534640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4107b30 0x7fb6e406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.280+0000 7fb6ea536640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4108480 0x7fb6e406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3907359820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4108480 msgr2=0x7fb6e406e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4108480 0x7fb6e406e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3907359820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4107b30 msgr2=0x7fb6e406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4107b30 0x7fb6e406e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3907359820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e4073b90 con 0x7fb6e41021f0 2024-09-21T02:53:50.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 0x7fb6e4074f20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000ea40 tx=0x7fb6e000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6ea536640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4108480 0x7fb6e406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6daffd640 1 -- 172.21.15.31:0/3907359820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0005020 con 0x7fb6e41021f0 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6daffd640 1 -- 172.21.15.31:0/3907359820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e00047a0 con 0x7fb6e41021f0 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.281+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e4073e20 con 0x7fb6e41021f0 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.282+0000 7fb6daffd640 1 -- 172.21.15.31:0/3907359820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00137d0 con 0x7fb6e41021f0 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.282+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e4074310 con 0x7fb6e41021f0 2024-09-21T02:53:50.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.282+0000 7fb6e9534640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4107b30 0x7fb6e406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:50.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.285+0000 7fb6daffd640 1 -- 172.21.15.31:0/3907359820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7fb6e00051c0 con 0x7fb6e41021f0 2024-09-21T02:53:50.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.285+0000 7fb6f250d740 1 Processor -- start 2024-09-21T02:53:50.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.286+0000 7fb6daffd640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6700 0x7fb6d00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.286+0000 7fb6daffd640 1 -- 172.21.15.31:0/3907359820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb6e00ebbf0 con 0x7fb6e41021f0 2024-09-21T02:53:50.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.286+0000 7fb6ea536640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6700 0x7fb6d00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.286+0000 7fb6f250d740 1 -- start start 2024-09-21T02:53:50.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db56eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db57930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6f250d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed0dac4ea0 con 0x55ed0d7a7070 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6f250d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed0dab42d0 con 0x55ed0d786390 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6f250d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed0dab80e0 con 0x55ed0d77e1b0 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6ea536640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6700 0x7fb6d00b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009370 tx=0x7fb6dc009300 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6e9d35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db56eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6e9d35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db56eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43460/0 (socket says 172.21.15.31:43460) 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6e9d35640 1 -- 172.21.15.31:0/2445328169 learned_addr learned my addr 172.21.15.31:0/2445328169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6e9534640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.287+0000 7fb6e9d35640 1 -- 172.21.15.31:0/2445328169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 msgr2=0x55ed0db573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6e9d35640 1 -- 172.21.15.31:0/2445328169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 msgr2=0x55ed0db57930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6ea536640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db57930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db57930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6e9d35640 1 -- 172.21.15.31:0/2445328169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0d812650 con 0x55ed0d7a7070 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6e9534640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6ea536640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db57930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db56eb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00adfd0 tx=0x7fb6e00ae0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa2ddfceaaefbe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6cffff640 1 -- 172.21.15.31:0/2445328169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00b2100 con 0x55ed0d7a7070 2024-09-21T02:53:50.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6cffff640 1 -- 172.21.15.31:0/2445328169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e00b64e0 con 0x55ed0d7a7070 2024-09-21T02:53:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.288+0000 7fb6cffff640 1 -- 172.21.15.31:0/2445328169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00ade00 con 0x55ed0d7a7070 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.289+0000 7fb6f250d740 1 -- 172.21.15.31:0/2445328169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 msgr2=0x55ed0db56eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.289+0000 7fb6f250d740 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db56eb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00adfd0 tx=0x7fb6e00ae0d0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.289+0000 7fb6f250d740 1 -- 172.21.15.31:0/2445328169 shutdown_connections 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.289+0000 7fb6f250d740 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.289+0000 7fb6f250d740 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db57930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.290+0000 7fb6f250d740 1 --2- 172.21.15.31:0/2445328169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db56eb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.290+0000 7fb6f250d740 1 -- 172.21.15.31:0/2445328169 >> 172.21.15.31:0/2445328169 conn(0x55ed0d79b460 msgr2=0x55ed0db54da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.290+0000 7fb6f250d740 1 -- 172.21.15.31:0/2445328169 shutdown_connections 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.290+0000 7fb6f250d740 1 -- 172.21.15.31:0/2445328169 wait complete. 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.290+0000 7fb6f250d740 1 -- start start 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.290+0000 7fb6f250d740 1 Processor -- start 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db62ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed0d915700 con 0x55ed0d7a7070 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed0da4f160 con 0x55ed0d786390 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed0db65690 con 0x55ed0d77e1b0 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6f250d740 10 client.0 _ll_register_callbacks cb 0x55ed0da689e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6ea536640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6e9534640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db62ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6e9d35640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6e9534640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db62ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:34082/0 (socket says 172.21.15.31:34082) 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.291+0000 7fb6e9534640 1 -- 172.21.15.31:0/3073824672 learned_addr learned my addr 172.21.15.31:0/3073824672 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 msgr2=0x55ed0db62ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db62ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3073824672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 msgr2=0x55ed0db6e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9d35640 1 -- 172.21.15.31:0/3073824672 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0da7cc90 con 0x55ed0d77e1b0 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9534640 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db62ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db62660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00aec70 tx=0x7fb6e00add00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f64854ca37e4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/3073824672 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00e9790 con 0x55ed0d77e1b0 2024-09-21T02:53:50.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.292+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/3073824672 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb6e00bb190 con 0x55ed0d77e1b0 2024-09-21T02:53:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.293+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e00b64e0 con 0x55ed0d77e1b0 2024-09-21T02:53:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.293+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/3073824672 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6e00adde0 con 0x55ed0d77e1b0 2024-09-21T02:53:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.293+0000 7fb6f250d740 10 client.25399 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.293+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed0db71400 con 0x55ed0d77e1b0 2024-09-21T02:53:50.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.293+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/3073824672 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e00b64e0 con 0x55ed0d77e1b0 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/3073824672 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e000b040 con 0x55ed0d77e1b0 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6f250d740 10 client.25399 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6f250d740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6f250d740 20 client.25399 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6f250d740 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d7659f0 0x55ed0db74760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6f250d740 4 client.25399 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.294+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed0dac6df0 con 0x55ed0d7659f0 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.295+0000 7fb6e9534640 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d7659f0 0x55ed0db74760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.296+0000 7fb6e9534640 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d7659f0 0x55ed0db74760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b89521fdaf3c86 server_cookie=216e4ca41b1ab8ff in_seq=0 out_seq=0 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.296+0000 7fb6cd7fa640 10 client.25399 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:50.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/3073824672 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed0dac6df0 con 0x55ed0d7659f0 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6cd7fa640 10 client.25399 handle_command_reply: tid=0 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 client.25399 shutdown 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 msgr2=0x55ed0db62660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db62660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00aec70 tx=0x7fb6e00add00 comp rx=0 tx=0).stop 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 shutdown_connections 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d7659f0 0x55ed0db74760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d786390 0x55ed0db62ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 --2- 172.21.15.31:0/3073824672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d77e1b0 0x55ed0db62660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6cd7fa640 0 client.25399 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 >> 172.21.15.31:0/3073824672 conn(0x55ed0d79b460 msgr2=0x55ed0db5a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 shutdown_connections 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 1 -- 172.21.15.31:0/3073824672 wait complete. 2024-09-21T02:53:50.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.297+0000 7fb6f250d740 20 client.25399 trim_cache size 0 max 16384 2024-09-21T02:53:50.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.314+0000 7fb6f250d740 1 Processor -- start 2024-09-21T02:53:50.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.314+0000 7fb6f250d740 1 -- start start 2024-09-21T02:53:50.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.314+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d77e1b0 0x55ed0db589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.314+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db64290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.314+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6f250d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a4e20 con 0x55ed0d77e1b0 2024-09-21T02:53:50.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6f250d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a6960 con 0x55ed0d7a7070 2024-09-21T02:53:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6f250d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a54f0 con 0x55ed0d786390 2024-09-21T02:53:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db64290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db64290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:43052/0 (socket says 172.21.15.31:43052) 2024-09-21T02:53:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 -- 172.21.15.31:0/1276042857 learned_addr learned my addr 172.21.15.31:0/1276042857 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6ea536640 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 -- 172.21.15.31:0/1276042857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d7a7070 msgr2=0x55ed0db6e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 -- 172.21.15.31:0/1276042857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d77e1b0 msgr2=0x55ed0db589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d77e1b0 0x55ed0db589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d77e1b0 0x55ed0db589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9534640 1 -- 172.21.15.31:0/1276042857 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed0db651f0 con 0x55ed0d786390 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.315+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d77e1b0 0x55ed0db589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.316+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db64290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6d402a930 tx=0x7fb6d4005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c9aea5d8acc87a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.316+0000 7fb6cf7fe640 1 -- 172.21.15.31:0/1276042857 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4035040 con 0x55ed0d786390 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.316+0000 7fb6cf7fe640 1 -- 172.21.15.31:0/1276042857 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d4048030 con 0x55ed0d786390 2024-09-21T02:53:50.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.316+0000 7fb6cf7fe640 1 -- 172.21.15.31:0/1276042857 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4003a80 con 0x55ed0d786390 2024-09-21T02:53:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 -- 172.21.15.31:0/1276042857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 msgr2=0x55ed0db64290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db64290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6d402a930 tx=0x7fb6d4005e60 comp rx=0 tx=0).stop 2024-09-21T02:53:50.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 -- 172.21.15.31:0/1276042857 shutdown_connections 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db64290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d7a7070 0x55ed0db6e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1276042857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d77e1b0 0x55ed0db589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 -- 172.21.15.31:0/1276042857 >> 172.21.15.31:0/1276042857 conn(0x55ed0d79b460 msgr2=0x55ed0db63b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 -- 172.21.15.31:0/1276042857 shutdown_connections 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.317+0000 7fb6f250d740 1 -- 172.21.15.31:0/1276042857 wait complete. 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.318+0000 7fb6f250d740 1 -- start start 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.318+0000 7fb6f250d740 1 Processor -- start 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d77e1b0 0x55ed0db78df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6f250d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db74480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6f250d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a6290 con 0x55ed0d7a7070 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6f250d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a5bc0 con 0x55ed0d77e1b0 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6f250d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41a2e00 con 0x55ed0d786390 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6ea536640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db74480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6ea536640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db74480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43480/0 (socket says 172.21.15.31:43480) 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6ea536640 1 -- 172.21.15.31:0/1260454937 learned_addr learned my addr 172.21.15.31:0/1260454937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d77e1b0 0x55ed0db78df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6e9534640 1 -- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d77e1b0 msgr2=0x55ed0db78df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d77e1b0 0x55ed0db78df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.319+0000 7fb6e9534640 1 -- 172.21.15.31:0/1260454937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 msgr2=0x55ed0db74480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db74480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6f250d740 10 client.0 _ll_register_callbacks cb 0x7fb6e00b1880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6e9534640 1 -- 172.21.15.31:0/1260454937 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed0db74c10 con 0x55ed0d786390 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d4048030 con 0x55ed0d786390 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6ea536640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db74480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6e9534640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db793b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4006010 tx=0x7fb6d4003630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8876760b90a85958 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6cd7fa640 10 client.25405 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:53:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/1260454937 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d405f050 con 0x55ed0d786390 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/1260454937 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb6d4003800 con 0x55ed0d786390 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/1260454937 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6d4055070 con 0x55ed0d786390 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6f250d740 10 client.25405 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.320+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55ed0dbde7e0 con 0x55ed0d786390 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/1260454937 <== mon.1 v2:172.21.15.178:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb6d4003a00 con 0x55ed0d786390 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6f250d740 10 client.25405 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6f250d740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6f250d740 20 client.25405 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d73ada0 0x55ed0dbe0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6f250d740 4 client.25405 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed0dac28a0 con 0x55ed0d73ada0 2024-09-21T02:53:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.321+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d73ada0 0x55ed0dbe0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6e9d35640 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d73ada0 0x55ed0dbe0e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2233b28dd750abc8 server_cookie=b5338629be8f0d33 in_seq=0 out_seq=0 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6cd7fa640 10 client.25405 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6cd7fa640 1 -- 172.21.15.31:0/1260454937 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed0dac28a0 con 0x55ed0d73ada0 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6cd7fa640 10 client.25405 handle_command_reply: tid=0 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 client.25405 shutdown 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 msgr2=0x55ed0db793b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db793b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4006010 tx=0x7fb6d4003630 comp rx=0 tx=0).stop 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 shutdown_connections 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ed0d73ada0 0x55ed0dbe0e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed0d786390 0x55ed0db793b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6cd7fa640 0 client.25405 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed0d7a7070 0x55ed0db74480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 --2- 172.21.15.31:0/1260454937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed0d77e1b0 0x55ed0db78df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 >> 172.21.15.31:0/1260454937 conn(0x55ed0d79b460 msgr2=0x55ed0db64480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 shutdown_connections 2024-09-21T02:53:50.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.322+0000 7fb6f250d740 1 -- 172.21.15.31:0/1260454937 wait complete. 2024-09-21T02:53:50.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.323+0000 7fb6f250d740 20 client.25405 trim_cache size 0 max 16384 2024-09-21T02:53:50.328 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.323+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6700 msgr2=0x7fb6d00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.323+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6700 0x7fb6d00b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009370 tx=0x7fb6dc009300 comp rx=0 tx=0).stop 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.323+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 msgr2=0x7fb6e4074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.323+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 0x7fb6e4074f20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000ea40 tx=0x7fb6e000ef00 comp rx=0 tx=0).stop 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 shutdown_connections 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb6d00b6700 0x7fb6d00b8c10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6e4108480 0x7fb6e406e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6e4107b30 0x7fb6e406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 --2- 172.21.15.31:0/3907359820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e41021f0 0x7fb6e4074f20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 >> 172.21.15.31:0/3907359820 conn(0x7fb6e4077c60 msgr2=0x7fb6e4103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 shutdown_connections 2024-09-21T02:53:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:50.324+0000 7fb6ebf99640 1 -- 172.21.15.31:0/3907359820 wait complete. 2024-09-21T02:53:50.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:50 smithi031 ceph-mon[31209]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T02:53:50.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:50 smithi165 ceph-mon[34818]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T02:53:50.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:50 smithi178 ceph-mon[34759]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T02:53:52.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:53:52.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:53:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:52.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:52 smithi031 ceph-mon[31209]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T02:53:52.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:52 smithi165 ceph-mon[34818]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T02:53:52.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:52 smithi178 ceph-mon[34759]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T02:53:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 -- 172.21.15.31:0/3471218146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 msgr2=0x7ff9a01087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 --2- 172.21.15.31:0/3471218146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01087f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff99000d910 tx=0x7ff99002c380 comp rx=0 tx=0).stop 2024-09-21T02:53:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 -- 172.21.15.31:0/3471218146 shutdown_connections 2024-09-21T02:53:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 --2- 172.21.15.31:0/3471218146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 0x7ff9a010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 --2- 172.21.15.31:0/3471218146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01087f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 --2- 172.21.15.31:0/3471218146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a0102aa0 0x7ff9a0102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.028+0000 7ff9a7923640 1 -- 172.21.15.31:0/3471218146 >> 172.21.15.31:0/3471218146 conn(0x7ff9a00797e0 msgr2=0x7ff9a0079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.029+0000 7ff9a7923640 1 -- 172.21.15.31:0/3471218146 shutdown_connections 2024-09-21T02:53:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.029+0000 7ff9a7923640 1 -- 172.21.15.31:0/3471218146 wait complete. 2024-09-21T02:53:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.029+0000 7ff9a7923640 1 Processor -- start 2024-09-21T02:53:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 -- start start 2024-09-21T02:53:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a0102aa0 0x7ff9a019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 0x7ff9a01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01118e0 con 0x7ff9a01083e0 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0111760 con 0x7ff9a0108d30 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a7923640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0111a60 con 0x7ff9a0102aa0 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a4ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a4ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43496/0 (socket says 172.21.15.31:43496) 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a4ebe640 1 -- 172.21.15.31:0/608803151 learned_addr learned my addr 172.21.15.31:0/608803151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.030+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a0102aa0 0x7ff9a019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 0x7ff9a01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/608803151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a0102aa0 msgr2=0x7ff9a019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a0102aa0 0x7ff9a019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 msgr2=0x7ff9a01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/608803151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a01acc10 con 0x7ff9a0108d30 2024-09-21T02:53:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.031+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 0x7ff9a01987e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff998009fd0 tx=0x7ff99800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.032+0000 7ff9967fc640 1 -- 172.21.15.31:0/608803151 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99800ce20 con 0x7ff9a0108d30 2024-09-21T02:53:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.032+0000 7ff9967fc640 1 -- 172.21.15.31:0/608803151 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff998005020 con 0x7ff9a0108d30 2024-09-21T02:53:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.032+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a01ace40 con 0x7ff9a0108d30 2024-09-21T02:53:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.032+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01ad240 con 0x7ff9a0108d30 2024-09-21T02:53:53.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.032+0000 7ff9967fc640 1 -- 172.21.15.31:0/608803151 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998010690 con 0x7ff9a0108d30 2024-09-21T02:53:53.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.036+0000 7ff9ade97740 1 Processor -- start 2024-09-21T02:53:53.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.036+0000 7ff9967fc640 1 -- 172.21.15.31:0/608803151 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7ff998010830 con 0x7ff9a0108d30 2024-09-21T02:53:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9967fc640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff98c0b6700 0x7ff98c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 -- start start 2024-09-21T02:53:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9967fc640 1 -- 172.21.15.31:0/608803151 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff9980e7de0 con 0x7ff9a0108d30 2024-09-21T02:53:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9a4ebe640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff98c0b6700 0x7ff98c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e536430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e536970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e536eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b65e4ace60 con 0x55b65e166490 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b65e49c290 con 0x55b65e16e670 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9ade97740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b65e4a00a0 con 0x55b65e18f350 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9a5ec0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e536eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.038+0000 7ff9a5ec0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e536eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:34138/0 (socket says 172.21.15.31:34138) 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2425733913 learned_addr learned my addr 172.21.15.31:0/2425733913 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e536430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2425733913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 msgr2=0x55b65e536430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e536430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2425733913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 msgr2=0x55b65e536970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e536970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2425733913 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65e1fa930 con 0x55b65e16e670 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e536eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ade10 tx=0x7ff9980adc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ca0d87e5563405 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff9a4ebe640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff98c0b6700 0x7ff98c0b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff990006010 tx=0x7ff99000d1b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.039+0000 7ff98b7fe640 1 -- 172.21.15.31:0/2425733913 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980b7b30 con 0x55b65e16e670 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.040+0000 7ff98b7fe640 1 -- 172.21.15.31:0/2425733913 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980e57d0 con 0x55b65e16e670 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.040+0000 7ff98b7fe640 1 -- 172.21.15.31:0/2425733913 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980056a0 con 0x55b65e16e670 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 -- 172.21.15.31:0/2425733913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 msgr2=0x55b65e536eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e536eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ade10 tx=0x7ff9980adc20 comp rx=0 tx=0).stop 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 -- 172.21.15.31:0/2425733913 shutdown_connections 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e536eb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e536970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2425733913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e536430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 -- 172.21.15.31:0/2425733913 >> 172.21.15.31:0/2425733913 conn(0x55b65e183740 msgr2=0x55b65e534320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 -- 172.21.15.31:0/2425733913 shutdown_connections 2024-09-21T02:53:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.041+0000 7ff9ade97740 1 -- 172.21.15.31:0/2425733913 wait complete. 2024-09-21T02:53:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.042+0000 7ff9ade97740 1 -- start start 2024-09-21T02:53:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.042+0000 7ff9ade97740 1 Processor -- start 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.042+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e542060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.042+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e542600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e54dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9ade97740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b65e2fd890 con 0x55b65e166490 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9ade97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b65e436d40 con 0x55b65e16e670 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9ade97740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b65e544bb0 con 0x55b65e18f350 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9ade97740 10 client.0 _ll_register_callbacks cb 0x55b65e49ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a56bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e542060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a5ec0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e54dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a5ec0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e54dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:43092/0 (socket says 172.21.15.31:43092) 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2458054510 learned_addr learned my addr 172.21.15.31:0/2458054510 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a4ebe640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e542600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 msgr2=0x55b65e542600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e542600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.043+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2458054510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 msgr2=0x55b65e542060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e542060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/2458054510 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65e54e4e0 con 0x55b65e18f350 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e542060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e54dea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9980d1290 tx=0x7ff9980d12c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4d46935777b926 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff988ff9640 1 -- 172.21.15.31:0/2458054510 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998099070 con 0x55b65e18f350 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff988ff9640 1 -- 172.21.15.31:0/2458054510 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff9980ad580 con 0x55b65e18f350 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.044+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9980e57d0 con 0x55b65e18f350 2024-09-21T02:53:53.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.045+0000 7ff988ff9640 1 -- 172.21.15.31:0/2458054510 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9980b7b30 con 0x55b65e18f350 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.045+0000 7ff9ade97740 10 client.25417 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.045+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b65e550e10 con 0x55b65e18f350 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.045+0000 7ff988ff9640 1 -- 172.21.15.31:0/2458054510 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980e57d0 con 0x55b65e18f350 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff988ff9640 1 -- 172.21.15.31:0/2458054510 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff9980a9af0 con 0x55b65e18f350 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff9ade97740 10 client.25417 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff9ade97740 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff9ade97740 20 client.25417 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e14dcd0 0x55b65e5543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff9ade97740 4 client.25417 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:53.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.046+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b65e4aedb0 con 0x55b65e14dcd0 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.047+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e14dcd0 0x55b65e5543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.047+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e14dcd0 0x55b65e5543f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e27127655bfb2fd server_cookie=aaa04df65a3e68f7 in_seq=0 out_seq=0 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.047+0000 7ff988ff9640 10 client.25417 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff988ff9640 1 -- 172.21.15.31:0/2458054510 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b65e4aedb0 con 0x55b65e14dcd0 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff988ff9640 10 client.25417 handle_command_reply: tid=0 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 client.25417 shutdown 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 msgr2=0x55b65e54dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e54dea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9980d1290 tx=0x7ff9980d12c0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 shutdown_connections 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e14dcd0 0x55b65e5543f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e54dea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e16e670 0x55b65e542600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 --2- 172.21.15.31:0/2458054510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e166490 0x55b65e542060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 >> 172.21.15.31:0/2458054510 conn(0x55b65e183740 msgr2=0x55b65e5395c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.048+0000 7ff988ff9640 0 client.25417 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:53.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.049+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 shutdown_connections 2024-09-21T02:53:53.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.049+0000 7ff9ade97740 1 -- 172.21.15.31:0/2458054510 wait complete. 2024-09-21T02:53:53.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.049+0000 7ff9ade97740 20 client.25417 trim_cache size 0 max 16384 2024-09-21T02:53:53.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.084+0000 7ff9ade97740 1 Processor -- start 2024-09-21T02:53:53.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.084+0000 7ff9ade97740 1 -- start start 2024-09-21T02:53:53.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.084+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.084+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e53b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e53ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9ade97740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b5c30 con 0x55b65e16e670 2024-09-21T02:53:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9ade97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b7770 con 0x55b65e166490 2024-09-21T02:53:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a4ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e53b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a4ebe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e53b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43524/0 (socket says 172.21.15.31:43524) 2024-09-21T02:53:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a4ebe640 1 -- 172.21.15.31:0/3371880754 learned_addr learned my addr 172.21.15.31:0/3371880754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9ade97740 1 -- 172.21.15.31:0/3371880754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b6300 con 0x55b65e18f350 2024-09-21T02:53:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e53ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a56bf640 1 -- 172.21.15.31:0/3371880754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 msgr2=0x55b65e53ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e53ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a56bf640 1 -- 172.21.15.31:0/3371880754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 msgr2=0x55b65e53b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e53b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.085+0000 7ff9a56bf640 1 -- 172.21.15.31:0/3371880754 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65e54e2e0 con 0x55b65e166490 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.086+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54dda0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff99c004700 tx=0x7ff99c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1735eaad4cacf81d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.086+0000 7ff98affd640 1 -- 172.21.15.31:0/3371880754 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c004030 con 0x55b65e166490 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.086+0000 7ff98affd640 1 -- 172.21.15.31:0/3371880754 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99c00f870 con 0x55b65e166490 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.086+0000 7ff98affd640 1 -- 172.21.15.31:0/3371880754 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0039e0 con 0x55b65e166490 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 -- 172.21.15.31:0/3371880754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 msgr2=0x55b65e54dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54dda0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff99c004700 tx=0x7ff99c005f90 comp rx=0 tx=0).stop 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 -- 172.21.15.31:0/3371880754 shutdown_connections 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e53ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e53b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 --2- 172.21.15.31:0/3371880754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54dda0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.087+0000 7ff9ade97740 1 -- 172.21.15.31:0/3371880754 >> 172.21.15.31:0/3371880754 conn(0x55b65e183740 msgr2=0x55b65e5406e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.088+0000 7ff9ade97740 1 -- 172.21.15.31:0/3371880754 shutdown_connections 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.088+0000 7ff9ade97740 1 -- 172.21.15.31:0/3371880754 wait complete. 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.088+0000 7ff9ade97740 1 -- start start 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.088+0000 7ff9ade97740 1 Processor -- start 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e5597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e559d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b70a0 con 0x55b65e16e670 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b69d0 con 0x55b65e166490 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b3c10 con 0x55b65e18f350 2024-09-21T02:53:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9ade97740 10 client.0 _ll_register_callbacks cb 0x7ff99c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9a5ec0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e559d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9a5ec0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e559d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:43104/0 (socket says 172.21.15.31:43104) 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/563898297 learned_addr learned my addr 172.21.15.31:0/563898297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.089+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a4ebe640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e5597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 msgr2=0x55b65e54a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/563898297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 msgr2=0x55b65e5597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e5597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a5ec0640 1 -- 172.21.15.31:0/563898297 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65e55a460 con 0x55b65e18f350 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a4ebe640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e5597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff9a5ec0640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e559d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9980d1260 tx=0x7ff9980a9fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d787722e9d18b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff988ff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:53:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff988ff9640 1 -- 172.21.15.31:0/563898297 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998099040 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.090+0000 7ff988ff9640 1 -- 172.21.15.31:0/563898297 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff9980aceb0 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.091+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff99c00f870 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.091+0000 7ff988ff9640 1 -- 172.21.15.31:0/563898297 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9980aa760 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.091+0000 7ff9ade97740 10 client.25423 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.091+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b65e5bedf0 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff988ff9640 1 -- 172.21.15.31:0/563898297 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99808b020 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff988ff9640 1 -- 172.21.15.31:0/563898297 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff998004a40 con 0x55b65e18f350 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9ade97740 10 client.25423 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9ade97740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9ade97740 20 client.25423 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9ade97740 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e123080 0x55b65e5c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9ade97740 4 client.25423 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:53:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b65e4aa860 con 0x55b65e123080 2024-09-21T02:53:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.092+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e123080 0x55b65e5c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.093+0000 7ff988ff9640 10 client.25423 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.093+0000 7ff9a56bf640 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e123080 0x55b65e5c12c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba4c93e889d586fe server_cookie=8544f96751922104 in_seq=0 out_seq=0 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff988ff9640 1 -- 172.21.15.31:0/563898297 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b65e4aa860 con 0x55b65e123080 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff988ff9640 10 client.25423 handle_command_reply: tid=0 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 client.25423 shutdown 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 msgr2=0x55b65e559d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e559d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9980d1260 tx=0x7ff9980a9fb0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 shutdown_connections 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b65e123080 0x55b65e5c12c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65e18f350 0x55b65e559d20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff988ff9640 0 client.25423 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e16e670 0x55b65e5597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 --2- 172.21.15.31:0/563898297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65e166490 0x55b65e54a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 >> 172.21.15.31:0/563898297 conn(0x55b65e183740 msgr2=0x55b65e539530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.095+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 shutdown_connections 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.096+0000 7ff9ade97740 1 -- 172.21.15.31:0/563898297 wait complete. 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.096+0000 7ff9ade97740 20 client.25423 trim_cache size 0 max 16384 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21", 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:53:53.101 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:53:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.096+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff98c0b6700 msgr2=0x7ff98c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff98c0b6700 0x7ff98c0b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff990006010 tx=0x7ff99000d1b0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 msgr2=0x7ff9a01987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 0x7ff9a01987e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff998009fd0 tx=0x7ff99800ef10 comp rx=0 tx=0).stop 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 shutdown_connections 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff98c0b6700 0x7ff98c0b8c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9a0108d30 0x7ff9a01987e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9a01083e0 0x7ff9a01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 --2- 172.21.15.31:0/608803151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9a0102aa0 0x7ff9a019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 >> 172.21.15.31:0/608803151 conn(0x7ff9a00797e0 msgr2=0x7ff9a01043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 shutdown_connections 2024-09-21T02:53:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:53.097+0000 7ff9a7923640 1 -- 172.21.15.31:0/608803151 wait complete. 2024-09-21T02:53:53.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:53 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:53:53.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:53 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:53.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:53 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:53:53.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:53 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:53.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:53 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:53:53.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:53 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:53.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:53 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:53:53.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:53 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:53.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:53 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:53:53.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:53 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:53.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:53 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:53:53.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:53 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:53:53.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:53:54.221 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:54.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:54 smithi031 ceph-mon[31209]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T02:53:54.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:54 smithi165 ceph-mon[34818]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T02:53:54.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:54 smithi178 ceph-mon[34759]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T02:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.788+0000 7efcd54cc640 1 -- 172.21.15.31:0/42715501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd01021f0 msgr2=0x7efcd0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.788+0000 7efcd54cc640 1 --2- 172.21.15.31:0/42715501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd01021f0 0x7efcd0102600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009900 tx=0x7efcc002b520 comp rx=0 tx=0).stop 2024-09-21T02:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 -- 172.21.15.31:0/42715501 shutdown_connections 2024-09-21T02:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 --2- 172.21.15.31:0/42715501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 0x7efcd010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 --2- 172.21.15.31:0/42715501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 --2- 172.21.15.31:0/42715501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd01021f0 0x7efcd0102600 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 -- 172.21.15.31:0/42715501 >> 172.21.15.31:0/42715501 conn(0x7efcd0077c60 msgr2=0x7efcd00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 -- 172.21.15.31:0/42715501 shutdown_connections 2024-09-21T02:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.789+0000 7efcd54cc640 1 -- 172.21.15.31:0/42715501 wait complete. 2024-09-21T02:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.790+0000 7efcd54cc640 1 Processor -- start 2024-09-21T02:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.791+0000 7efcd54cc640 1 -- start start 2024-09-21T02:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.791+0000 7efcd54cc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.791+0000 7efcd54cc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 0x7efcd0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.791+0000 7efcd54cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd019dc10 0x7efcd019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd019dc10 0x7efcd019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 0x7efcd0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd019dc10 0x7efcd019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43570/0 (socket says 172.21.15.31:43570) 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 0x7efcd0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:43128/0 (socket says 172.21.15.31:43128) 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:34194/0 (socket says 172.21.15.31:34194) 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 -- 172.21.15.31:0/2400458625 learned_addr learned my addr 172.21.15.31:0/2400458625 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01118c0 con 0x7efcd019dc10 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0111740 con 0x7efcd0107b30 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0111a40 con 0x7efcd0108480 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 -- 172.21.15.31:0/2400458625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 msgr2=0x7efcd0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 0x7efcd0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 -- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd019dc10 msgr2=0x7efcd019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd019dc10 0x7efcd019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.792+0000 7efcceffd640 1 -- 172.21.15.31:0/2400458625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019e7f0 con 0x7efcd0107b30 2024-09-21T02:53:54.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.793+0000 7efcceffd640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0198490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006010 tx=0x7efcc00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:54.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.793+0000 7efcb7fff640 1 -- 172.21.15.31:0/2400458625 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc003f030 con 0x7efcd0107b30 2024-09-21T02:53:54.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.793+0000 7efcb7fff640 1 -- 172.21.15.31:0/2400458625 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc0003cb0 con 0x7efcd0107b30 2024-09-21T02:53:54.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.793+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd01a4d90 con 0x7efcd0107b30 2024-09-21T02:53:54.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.793+0000 7efcb7fff640 1 -- 172.21.15.31:0/2400458625 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0036070 con 0x7efcd0107b30 2024-09-21T02:53:54.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.793+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd01a5130 con 0x7efcd0107b30 2024-09-21T02:53:54.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 Processor -- start 2024-09-21T02:53:54.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 -- start start 2024-09-21T02:53:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed70a390 0x5586edad2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5586eda48e50 con 0x5586ed72b070 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5586eda38280 con 0x5586ed7021b0 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.797+0000 7efcdba40740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5586eda3c090 con 0x5586ed70a390 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43578/0 (socket says 172.21.15.31:43578) 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 -- 172.21.15.31:0/2759688695 learned_addr learned my addr 172.21.15.31:0/2759688695 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed70a390 0x5586edad2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 -- 172.21.15.31:0/2759688695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed70a390 msgr2=0x5586edad2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed70a390 0x5586edad2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 -- 172.21.15.31:0/2759688695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 msgr2=0x5586edad27b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:53:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad27b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 -- 172.21.15.31:0/2759688695 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ed796650 con 0x5586ed72b070 2024-09-21T02:53:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efcb7fff640 1 -- 172.21.15.31:0/2400458625 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 25) ==== 154637+0+0 (secure 0 0 0) 0x7efcc0042030 con 0x7efcd0107b30 2024-09-21T02:53:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.798+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad2270 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f70 tx=0x7efcc8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bc92499f5e3997 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.799+0000 7efcb57fa640 1 -- 172.21.15.31:0/2759688695 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8016070 con 0x5586ed72b070 2024-09-21T02:53:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.799+0000 7efcb57fa640 1 -- 172.21.15.31:0/2759688695 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc8004d20 con 0x5586ed72b070 2024-09-21T02:53:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.799+0000 7efcb57fa640 1 -- 172.21.15.31:0/2759688695 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8012790 con 0x5586ed72b070 2024-09-21T02:53:54.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.799+0000 7efcb7fff640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efcb80b6780 0x7efcb80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcb7fff640 1 -- 172.21.15.31:0/2400458625 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7efcc0107b60 con 0x7efcd0107b30 2024-09-21T02:53:54.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcceffd640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efcb80b6780 0x7efcb80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 -- 172.21.15.31:0/2759688695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 msgr2=0x5586edad2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad2270 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f70 tx=0x7efcc8010800 comp rx=0 tx=0).stop 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 -- 172.21.15.31:0/2759688695 shutdown_connections 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed70a390 0x5586edad2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 --2- 172.21.15.31:0/2759688695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad2270 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 -- 172.21.15.31:0/2759688695 >> 172.21.15.31:0/2759688695 conn(0x5586ed71f460 msgr2=0x5586edad0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.800+0000 7efcdba40740 1 -- 172.21.15.31:0/2759688695 shutdown_connections 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.801+0000 7efcceffd640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efcb80b6780 0x7efcb80b8c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5586edad0ef0 tx=0x7efcc0005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.801+0000 7efcdba40740 1 -- 172.21.15.31:0/2759688695 wait complete. 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.801+0000 7efcdba40740 1 -- start start 2024-09-21T02:53:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.801+0000 7efcdba40740 1 Processor -- start 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 0x5586edad83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed72b070 0x5586edadd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5586ed899700 con 0x5586ed70a390 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5586ed9d3110 con 0x5586ed7021b0 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5586edade960 con 0x5586ed72b070 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcdba40740 10 client.0 _ll_register_callbacks cb 0x5586ed9ec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 0x5586edad83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 0x5586edad83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43586/0 (socket says 172.21.15.31:43586) 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efcce7fc640 1 -- 172.21.15.31:0/2120452340 learned_addr learned my addr 172.21.15.31:0/2120452340 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.802+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed72b070 0x5586edadd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcce7fc640 1 -- 172.21.15.31:0/2120452340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed72b070 msgr2=0x5586edadd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed72b070 0x5586edadd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcce7fc640 1 -- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 msgr2=0x5586edad7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcce7fc640 1 -- 172.21.15.31:0/2120452340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586eda00c40 con 0x5586ed70a390 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 0x5586edad83e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b8a0 tx=0x7efcc400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00f6b90288278ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:54.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2120452340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc401e060 con 0x5586ed70a390 2024-09-21T02:53:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2120452340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7efcc4019480 con 0x5586ed70a390 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.803+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc8004d20 con 0x5586ed70a390 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.804+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2120452340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efcc4019680 con 0x5586ed70a390 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.804+0000 7efcdba40740 10 client.15750 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.804+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586edaf50b0 con 0x5586ed70a390 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.804+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2120452340 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc8004d20 con 0x5586ed70a390 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2120452340 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efcc4018500 con 0x5586ed70a390 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcdba40740 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcdba40740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcdba40740 20 client.15750 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcdba40740 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6e99f0 0x5586edaf85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcdba40740 4 client.15750 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:53:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.805+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586eda4ada0 con 0x5586ed6e99f0 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.806+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6e99f0 0x5586edaf85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.806+0000 7efcb2ff5640 10 client.15750 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.806+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6e99f0 0x5586edaf85c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d08432efec5ce33 server_cookie=6edf70e6e35e91bf in_seq=0 out_seq=0 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2120452340 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586eda4ada0 con 0x5586ed6e99f0 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcb2ff5640 10 client.15750 handle_command_reply: tid=0 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 client.15750 shutdown 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 msgr2=0x5586edad83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 0x5586edad83e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efcc400b8a0 tx=0x7efcc400bd60 comp rx=0 tx=0).stop 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 shutdown_connections 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6e99f0 0x5586edaf85c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcb2ff5640 0 client.15750 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed72b070 0x5586edadd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed70a390 0x5586edad83e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 --2- 172.21.15.31:0/2120452340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.807+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 >> 172.21.15.31:0/2120452340 conn(0x5586ed71f460 msgr2=0x5586edad5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.808+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 shutdown_connections 2024-09-21T02:53:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.808+0000 7efcdba40740 1 -- 172.21.15.31:0/2120452340 wait complete. 2024-09-21T02:53:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.808+0000 7efcdba40740 20 client.15750 trim_cache size 0 max 16384 2024-09-21T02:53:54.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 Processor -- start 2024-09-21T02:53:54.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 -- start start 2024-09-21T02:53:54.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edae18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01adb90 con 0x5586ed72b070 2024-09-21T02:53:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01af6d0 con 0x5586ed7021b0 2024-09-21T02:53:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.844+0000 7efcdba40740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01ae260 con 0x5586ed6e99f0 2024-09-21T02:53:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43592/0 (socket says 172.21.15.31:43592) 2024-09-21T02:53:54.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 -- 172.21.15.31:0/2118358698 learned_addr learned my addr 172.21.15.31:0/2118358698 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:54.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edae18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 -- 172.21.15.31:0/2118358698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 msgr2=0x5586edad01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 -- 172.21.15.31:0/2118358698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 msgr2=0x5586edad5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efccf7fe640 1 -- 172.21.15.31:0/2118358698 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586edaf3320 con 0x5586ed6e99f0 2024-09-21T02:53:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.845+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:53:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.846+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edae18c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efcc8010d00 tx=0x7efcc8010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e98dc00a8c42a43d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.846+0000 7efcb4ff9640 1 -- 172.21.15.31:0/2118358698 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8037070 con 0x5586ed6e99f0 2024-09-21T02:53:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.846+0000 7efcb4ff9640 1 -- 172.21.15.31:0/2118358698 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc80055b0 con 0x5586ed6e99f0 2024-09-21T02:53:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.846+0000 7efcb4ff9640 1 -- 172.21.15.31:0/2118358698 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8016040 con 0x5586ed6e99f0 2024-09-21T02:53:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 -- 172.21.15.31:0/2118358698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 msgr2=0x5586edae18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edae18c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efcc8010d00 tx=0x7efcc8010fd0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 -- 172.21.15.31:0/2118358698 shutdown_connections 2024-09-21T02:53:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 --2- 172.21.15.31:0/2118358698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edae18c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 -- 172.21.15.31:0/2118358698 >> 172.21.15.31:0/2118358698 conn(0x5586ed71f460 msgr2=0x5586edacfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 -- 172.21.15.31:0/2118358698 shutdown_connections 2024-09-21T02:53:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.847+0000 7efcdba40740 1 -- 172.21.15.31:0/2118358698 wait complete. 2024-09-21T02:53:54.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.848+0000 7efcdba40740 1 -- start start 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.848+0000 7efcdba40740 1 Processor -- start 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.848+0000 7efcdba40740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edaf2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.848+0000 7efcdba40740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcdba40740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcdba40740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01af000 con 0x5586ed72b070 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcdba40740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01ae930 con 0x5586ed7021b0 2024-09-21T02:53:54.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcdba40740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01abb70 con 0x5586ed6e99f0 2024-09-21T02:53:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcdba40740 10 client.0 _ll_register_callbacks cb 0x7efcc4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:53:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43594/0 (socket says 172.21.15.31:43594) 2024-09-21T02:53:54.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 -- 172.21.15.31:0/2036816469 learned_addr learned my addr 172.21.15.31:0/2036816469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:53:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edaf2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 -- 172.21.15.31:0/2036816469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 msgr2=0x5586edaf2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edaf2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 -- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 msgr2=0x5586edad5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:53:54.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 -- 172.21.15.31:0/2036816469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586edad69e0 con 0x5586ed72b070 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efccf7fe640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad6420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000f80 tx=0x7efcc8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34332a9145937b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.849+0000 7efcb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2036816469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8037040 con 0x5586ed72b070 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2036816469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7efcc802b9a0 con 0x5586ed72b070 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc80055b0 con 0x5586ed72b070 2024-09-21T02:53:54.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2036816469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efcc8058050 con 0x5586ed72b070 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcdba40740 10 client.15756 fetch_fsmap learned FSMap version 27 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5586edaf2350 con 0x5586ed72b070 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.850+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2036816469 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc802bba0 con 0x5586ed72b070 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2036816469 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efcc8028040 con 0x5586ed72b070 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcdba40740 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcdba40740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcdba40740 20 client.15756 populate_metadata read hostname 'smithi031' 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcdba40740 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6beda0 0x5586edb651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcdba40740 4 client.15756 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7efcc80055b0 con 0x5586ed6beda0 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6beda0 0x5586edb651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcb2ff5640 10 client.15756 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:54.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.851+0000 7efcce7fc640 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6beda0 0x5586edb651d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c0575cf84fc72 server_cookie=35f4c5c759f6fcd0 in_seq=0 out_seq=0 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcb2ff5640 1 -- 172.21.15.31:0/2036816469 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x7efcc80055b0 con 0x5586ed6beda0 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcb2ff5640 10 client.15756 handle_command_reply: tid=0 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 client.15756 shutdown 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 msgr2=0x5586edad6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad6420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000f80 tx=0x7efcc8009800 comp rx=0 tx=0).stop 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 shutdown_connections 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5586ed6beda0 0x5586edb651d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5586ed72b070 0x5586edad6420 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5586ed7021b0 0x5586edad5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 --2- 172.21.15.31:0/2036816469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5586ed6e99f0 0x5586edaf2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 >> 172.21.15.31:0/2036816469 conn(0x5586ed71f460 msgr2=0x5586edadb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:54.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcb2ff5640 0 client.15756 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 shutdown_connections 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 1 -- 172.21.15.31:0/2036816469 wait complete. 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.852+0000 7efcdba40740 20 client.15756 trim_cache size 0 max 16384 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:53:54.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21", 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:53:54.874 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:53:54.875 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efcb80b6780 msgr2=0x7efcb80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efcb80b6780 0x7efcb80b8c90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5586edad0ef0 tx=0x7efcc0005d50 comp rx=0 tx=0).stop 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 msgr2=0x7efcd0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0198490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006010 tx=0x7efcc00042f0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 shutdown_connections 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efcb80b6780 0x7efcb80b8c90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcd0108480 0x7efcd0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd019dc10 0x7efcd019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 --2- 172.21.15.31:0/2400458625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efcd0107b30 0x7efcd0198490 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.853+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 >> 172.21.15.31:0/2400458625 conn(0x7efcd0077c60 msgr2=0x7efcd0105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.854+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 shutdown_connections 2024-09-21T02:53:54.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:53:54.854+0000 7efcd54cc640 1 -- 172.21.15.31:0/2400458625 wait complete. 2024-09-21T02:53:55.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:55 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:53:55.610 INFO:tasks.cephfs.filesystem:scrub status for tag:85a5ac9c-d6e6-477d-aeae-fbd2fa198d21 - {'path': '/', 'tag': '85a5ac9c-d6e6-477d-aeae-fbd2fa198d21', 'options': 'recursive,force'} 2024-09-21T02:53:55.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:55 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:53:55.731 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:55 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:53:56.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:56 smithi031 ceph-mon[31209]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:53:56.592 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:56 smithi165 ceph-mon[34818]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:53:56.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:56 smithi178 ceph-mon[34759]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:53:58.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:53:58 smithi031 ceph-mon[31209]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T02:53:58.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:53:58 smithi165 ceph-mon[34818]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T02:53:58.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:53:58 smithi178 ceph-mon[34759]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T02:54:00.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:00 smithi031 ceph-mon[31209]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T02:54:00.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:00 smithi165 ceph-mon[34818]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T02:54:00.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:00 smithi178 ceph-mon[34759]: pgmap v358: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T02:54:01.084 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:54:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:54:00.695+0000 7f08c84fe640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:54:01.084 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:54:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:54:00.695+0000 7f08c84fe640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:54:01.084 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:54:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:54:00.695+0000 7f08c84fe640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:54:01.085 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:54:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:54:00.695+0000 7f08c84fe640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:54:01.085 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 02:54:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T02:54:00.695+0000 7f08c84fe640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:54:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:02 smithi031 ceph-mon[31209]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:54:02.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:02 smithi031 ceph-mon[31209]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T02:54:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:02 smithi165 ceph-mon[34818]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:54:02.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:02 smithi165 ceph-mon[34818]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T02:54:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:02 smithi178 ceph-mon[34759]: pgmap v359: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:54:02.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:02 smithi178 ceph-mon[34759]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T02:54:04.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:04 smithi031 ceph-mon[31209]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 24 op/s 2024-09-21T02:54:04.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:04 smithi165 ceph-mon[34818]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 24 op/s 2024-09-21T02:54:04.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:04 smithi178 ceph-mon[34759]: pgmap v360: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 24 op/s 2024-09-21T02:54:06.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:06 smithi165 ceph-mon[34818]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:54:06.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:06 smithi178 ceph-mon[34759]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:54:06.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:06 smithi031 ceph-mon[31209]: pgmap v361: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:54:08.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:08 smithi165 ceph-mon[34818]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T02:54:08.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:08 smithi178 ceph-mon[34759]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T02:54:08.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:08 smithi031 ceph-mon[31209]: pgmap v362: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T02:54:10.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:10 smithi165 ceph-mon[34818]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-21T02:54:10.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:10 smithi178 ceph-mon[34759]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-21T02:54:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:10 smithi031 ceph-mon[31209]: pgmap v363: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-21T02:54:11.341 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 90% (11475/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T02:54:11.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:11.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:11 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:11.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:11 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:11.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:11 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T02:54:11.992 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:11.993 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:54:11.996 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T02:54:12.030 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T02:54:12.032 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T02:54:12.033 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T02:54:12.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:12 smithi165 ceph-mon[34818]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T02:54:12.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:12.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:12.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:12.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:12 smithi178 ceph-mon[34759]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T02:54:12.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:12.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:12.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:12.752 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T02:54:12.829 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T02:54:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:12 smithi031 ceph-mon[31209]: pgmap v364: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T02:54:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:12.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:54:12.870 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T02:54:12.872 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T02:54:12.872 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T02:54:12.912 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T02:54:12.915 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T02:54:12.915 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T02:54:12.973 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-09-21T02:54:12.974 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-09-21T02:54:12.974 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-09-21T02:54:13.035 INFO:tasks.workunit.client.0.smithi031.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-09-21T02:54:13.067 INFO:tasks.workunit.client.0.smithi031.stderr:--2024-09-21 02:54:13-- http://download.ceph.com/qa/fsync-tester.c 2024-09-21T02:54:13.068 INFO:tasks.workunit.client.0.smithi031.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T02:54:13.089 INFO:tasks.workunit.client.0.smithi031.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T02:54:13.110 INFO:tasks.workunit.client.0.smithi031.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T02:54:13.110 INFO:tasks.workunit.client.0.smithi031.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-09-21T02:54:13.114 INFO:tasks.workunit.client.0.smithi031.stderr:Saving to: ‘fsync-tester.c’ 2024-09-21T02:54:13.114 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:13.114 INFO:tasks.workunit.client.0.smithi031.stderr: 0K .. 100% 551M=0s 2024-09-21T02:54:13.114 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:13.115 INFO:tasks.workunit.client.0.smithi031.stderr:2024-09-21 02:54:13 (551 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-09-21T02:54:13.115 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-21T02:54:13.115 INFO:tasks.workunit.client.0.smithi031.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-09-21T02:54:13.212 INFO:tasks.workunit.client.0.smithi031.stderr:+ ./fsync-tester 2024-09-21T02:54:14.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:14 smithi165 ceph-mon[34818]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T02:54:14.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:14 smithi178 ceph-mon[34759]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T02:54:14.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:14 smithi031 ceph-mon[31209]: pgmap v365: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T02:54:16.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:16 smithi165 ceph-mon[34818]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:54:16.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:16 smithi178 ceph-mon[34759]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:54:16.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:16 smithi031 ceph-mon[31209]: pgmap v366: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:54:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:18 smithi165 ceph-mon[34818]: pgmap v367: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 KiB/s wr, 13 op/s 2024-09-21T02:54:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:18 smithi165 ceph-mon[34818]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T02:54:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:18 smithi178 ceph-mon[34759]: pgmap v367: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 KiB/s wr, 13 op/s 2024-09-21T02:54:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:18 smithi178 ceph-mon[34759]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T02:54:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:18 smithi031 ceph-mon[31209]: pgmap v367: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 263 KiB/s wr, 13 op/s 2024-09-21T02:54:18.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:18 smithi031 ceph-mon[31209]: mgrmap e27: x(active, since 10m), standbys: z, y 2024-09-21T02:54:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:20 smithi165 ceph-mon[34818]: pgmap v368: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 263 KiB/s wr, 12 op/s 2024-09-21T02:54:20.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:20 smithi178 ceph-mon[34759]: pgmap v368: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 263 KiB/s wr, 12 op/s 2024-09-21T02:54:20.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:20 smithi031 ceph-mon[31209]: pgmap v368: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 263 KiB/s wr, 12 op/s 2024-09-21T02:54:22.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:22 smithi031 ceph-mon[31209]: pgmap v369: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 690 KiB/s wr, 12 op/s 2024-09-21T02:54:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:22 smithi165 ceph-mon[34818]: pgmap v369: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 690 KiB/s wr, 12 op/s 2024-09-21T02:54:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:22 smithi178 ceph-mon[34759]: pgmap v369: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 690 KiB/s wr, 12 op/s 2024-09-21T02:54:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:24 smithi031 ceph-mon[31209]: pgmap v370: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 694 KiB/s wr, 14 op/s 2024-09-21T02:54:24.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:24 smithi165 ceph-mon[34818]: pgmap v370: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 694 KiB/s wr, 14 op/s 2024-09-21T02:54:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:24 smithi178 ceph-mon[34759]: pgmap v370: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 694 KiB/s wr, 14 op/s 2024-09-21T02:54:25.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:54:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:26.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 -- 172.21.15.31:0/4109378748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48010de50 msgr2=0x7fd480110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 --2- 172.21.15.31:0/4109378748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48010de50 0x7fd480110340 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd478009ca0 tx=0x7fd47802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 -- 172.21.15.31:0/4109378748 shutdown_connections 2024-09-21T02:54:26.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 --2- 172.21.15.31:0/4109378748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd480105340 0x7fd48010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 --2- 172.21.15.31:0/4109378748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48010de50 0x7fd480110340 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 --2- 172.21.15.31:0/4109378748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 0x7fd480104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 -- 172.21.15.31:0/4109378748 >> 172.21.15.31:0/4109378748 conn(0x7fd4800797e0 msgr2=0x7fd480079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 -- 172.21.15.31:0/4109378748 shutdown_connections 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.401+0000 7fd485596640 1 -- 172.21.15.31:0/4109378748 wait complete. 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 Processor -- start 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 -- start start 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 0x7fd480102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 0x7fd480102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd48010de50 0x7fd4801031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113ab0 con 0x7fd480105340 2024-09-21T02:54:26.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113930 con 0x7fd4801049f0 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.402+0000 7fd485596640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113c30 con 0x7fd48010de50 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 0x7fd480102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 0x7fd480102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55456/0 (socket says 172.21.15.31:55456) 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3467595534 learned_addr learned my addr 172.21.15.31:0/3467595534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47effd640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 0x7fd480102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd48010de50 0x7fd4801031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3467595534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd48010de50 msgr2=0x7fd4801031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd48010de50 0x7fd4801031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3467595534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 msgr2=0x7fd480102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 0x7fd480102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3467595534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801a8740 con 0x7fd480105340 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47effd640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 0x7fd480102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.403+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 0x7fd480102c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd47400d950 tx=0x7fd47400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.404+0000 7fd467fff640 1 -- 172.21.15.31:0/3467595534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474017070 con 0x7fd480105340 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.404+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801a8a30 con 0x7fd480105340 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.404+0000 7fd467fff640 1 -- 172.21.15.31:0/3467595534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4740054d0 con 0x7fd480105340 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.404+0000 7fd467fff640 1 -- 172.21.15.31:0/3467595534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474004ca0 con 0x7fd480105340 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.404+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a8f70 con 0x7fd480105340 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 Processor -- start 2024-09-21T02:54:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 -- start start 2024-09-21T02:54:26.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd467fff640 1 -- 172.21.15.31:0/3467595534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd474004e40 con 0x7fd480105340 2024-09-21T02:54:26.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a309070 0x55d67a6b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d67a626e50 con 0x55d67a2e8390 2024-09-21T02:54:26.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d67a616280 con 0x55d67a2e01b0 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d67a61a090 con 0x55d67a309070 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd47e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd47e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55464/0 (socket says 172.21.15.31:55464) 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd47e7fc640 1 -- 172.21.15.31:0/1494767763 learned_addr learned my addr 172.21.15.31:0/1494767763 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.407+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a309070 0x55d67a6b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47effd640 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47f7fe640 1 -- 172.21.15.31:0/1494767763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 msgr2=0x55d67a6b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47f7fe640 1 -- 172.21.15.31:0/1494767763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 msgr2=0x55d67a6b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47f7fe640 1 -- 172.21.15.31:0/1494767763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67a374650 con 0x55d67a309070 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd467fff640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd4680b6680 0x7fd4680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a309070 0x55d67a6b0650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd478005e30 tx=0x7fd47802f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291977c4e9a71b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd47effd640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd4680b6680 0x7fd4680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd467fff640 1 -- 172.21.15.31:0/3467595534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd4740e6ea0 con 0x7fd480105340 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.408+0000 7fd4657fa640 1 -- 172.21.15.31:0/1494767763 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478031cf0 con 0x55d67a309070 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.409+0000 7fd4657fa640 1 -- 172.21.15.31:0/1494767763 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478036030 con 0x55d67a309070 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.409+0000 7fd4657fa640 1 -- 172.21.15.31:0/1494767763 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478003e60 con 0x55d67a309070 2024-09-21T02:54:26.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1494767763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a309070 msgr2=0x55d67a6b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a309070 0x55d67a6b0650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd478005e30 tx=0x7fd47802f4c0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd47effd640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd4680b6680 0x7fd4680b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55d67a6af2d0 tx=0x7fd470005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1494767763 shutdown_connections 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1494767763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a309070 0x55d67a6b0650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1494767763 >> 172.21.15.31:0/1494767763 conn(0x55d67a2fd460 msgr2=0x55d67a6ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:26.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1494767763 shutdown_connections 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1494767763 wait complete. 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 -- start start 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.410+0000 7fd48bb0a740 1 Processor -- start 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e01b0 0x55d67a6bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a6bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a309070 0x55d67a6c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d67a477700 con 0x55d67a2e01b0 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d67a5b1110 con 0x55d67a309070 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d67a6bec50 con 0x55d67a2e8390 2024-09-21T02:54:26.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd48bb0a740 10 client.0 _ll_register_callbacks cb 0x55d67a5ca990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a6bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a6bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:44620/0 (socket says 172.21.15.31:44620) 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3033876308 learned_addr learned my addr 172.21.15.31:0/3033876308 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a309070 0x55d67a6c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47effd640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e01b0 0x55d67a6bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a309070 msgr2=0x55d67a6c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a309070 0x55d67a6c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3033876308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e01b0 msgr2=0x55d67a6bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e01b0 0x55d67a6bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.411+0000 7fd47e7fc640 1 -- 172.21.15.31:0/3033876308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d67a5dec40 con 0x55d67a2e8390 2024-09-21T02:54:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd47effd640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e01b0 0x55d67a6bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a6bc1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4740acc40 tx=0x7fd47400b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db835891aa598db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd462ff5640 1 -- 172.21.15.31:0/3033876308 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a8b50 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd462ff5640 1 -- 172.21.15.31:0/3033876308 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd4740d02b0 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd478036030 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd462ff5640 1 -- 172.21.15.31:0/3033876308 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4740e4a40 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd48bb0a740 10 client.25459 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.412+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d67a6ca980 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd462ff5640 1 -- 172.21.15.31:0/3033876308 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478036030 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd462ff5640 1 -- 172.21.15.31:0/3033876308 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd4740a8600 con 0x55d67a2e8390 2024-09-21T02:54:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd48bb0a740 10 client.25459 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd48bb0a740 10 client.25459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd48bb0a740 20 client.25459 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a2c79f0 0x55d67a6cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd48bb0a740 4 client.25459 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d67a628da0 con 0x55d67a2c79f0 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.413+0000 7fd47effd640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a2c79f0 0x55d67a6cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.414+0000 7fd47effd640 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a2c79f0 0x55d67a6cddf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cacd3cf5a5889ffa server_cookie=4637b607ce41c34a in_seq=0 out_seq=0 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.414+0000 7fd462ff5640 10 client.25459 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd462ff5640 1 -- 172.21.15.31:0/3033876308 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d67a628da0 con 0x55d67a2c79f0 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd462ff5640 10 client.25459 handle_command_reply: tid=0 2024-09-21T02:54:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 client.25459 shutdown 2024-09-21T02:54:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 msgr2=0x55d67a6bc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a6bc1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4740acc40 tx=0x7fd47400b4b0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 shutdown_connections 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a2c79f0 0x55d67a6cddf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a6bc1e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a309070 0x55d67a6c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd462ff5640 0 client.25459 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/3033876308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e01b0 0x55d67a6bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 >> 172.21.15.31:0/3033876308 conn(0x55d67a2fd460 msgr2=0x55d67a6b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 shutdown_connections 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.415+0000 7fd48bb0a740 1 -- 172.21.15.31:0/3033876308 wait complete. 2024-09-21T02:54:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.416+0000 7fd48bb0a740 20 client.25459 trim_cache size 0 max 16384 2024-09-21T02:54:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.443+0000 7fd48bb0a740 1 Processor -- start 2024-09-21T02:54:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.443+0000 7fd48bb0a740 1 -- start start 2024-09-21T02:54:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2c79f0 0x55d67a6b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e01b0 0x55d67a6bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801b18d0 con 0x55d67a2e8390 2024-09-21T02:54:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801b3410 con 0x55d67a2c79f0 2024-09-21T02:54:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801b1fa0 con 0x55d67a2e01b0 2024-09-21T02:54:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55490/0 (socket says 172.21.15.31:55490) 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47f7fe640 1 -- 172.21.15.31:0/1146800878 learned_addr learned my addr 172.21.15.31:0/1146800878 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e01b0 0x55d67a6bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47effd640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2c79f0 0x55d67a6b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47e7fc640 1 -- 172.21.15.31:0/1146800878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2c79f0 msgr2=0x55d67a6b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2c79f0 0x55d67a6b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47e7fc640 1 -- 172.21.15.31:0/1146800878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 msgr2=0x55d67a6c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47e7fc640 1 -- 172.21.15.31:0/1146800878 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67a6ba1a0 con 0x55d67a2e01b0 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.444+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.445+0000 7fd47effd640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2c79f0 0x55d67a6b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.445+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e01b0 0x55d67a6bd8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd474009610 tx=0x7fd474009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca445f04746624c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.445+0000 7fd464ff9640 1 -- 172.21.15.31:0/1146800878 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a8420 con 0x55d67a2e01b0 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.445+0000 7fd464ff9640 1 -- 172.21.15.31:0/1146800878 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd474009c30 con 0x55d67a2e01b0 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.445+0000 7fd464ff9640 1 -- 172.21.15.31:0/1146800878 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740ac420 con 0x55d67a2e01b0 2024-09-21T02:54:26.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1146800878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e01b0 msgr2=0x55d67a6bd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e01b0 0x55d67a6bd8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd474009610 tx=0x7fd474009f20 comp rx=0 tx=0).stop 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1146800878 shutdown_connections 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e01b0 0x55d67a6bd8d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2e8390 0x55d67a6c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/1146800878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2c79f0 0x55d67a6b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1146800878 >> 172.21.15.31:0/1146800878 conn(0x55d67a2fd460 msgr2=0x55d67a6bd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1146800878 shutdown_connections 2024-09-21T02:54:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 -- 172.21.15.31:0/1146800878 wait complete. 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.446+0000 7fd48bb0a740 1 -- start start 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 Processor -- start 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 0x55d67a6c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a731770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801b2d40 con 0x55d67a2c79f0 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801b2670 con 0x55d67a2e01b0 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801af8b0 con 0x55d67a2e8390 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd48bb0a740 10 client.0 _ll_register_callbacks cb 0x7fd4740ea640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd47effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 0x55d67a6c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd47effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 0x55d67a6c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55502/0 (socket says 172.21.15.31:55502) 2024-09-21T02:54:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd47effd640 1 -- 172.21.15.31:0/2344768268 learned_addr learned my addr 172.21.15.31:0/2344768268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.447+0000 7fd47f7fe640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a731770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47e7fc640 1 -- 172.21.15.31:0/2344768268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 msgr2=0x55d67a731770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a731770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47e7fc640 1 -- 172.21.15.31:0/2344768268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 msgr2=0x55d67a6c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 0x55d67a6c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47e7fc640 1 -- 172.21.15.31:0/2344768268 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d67a731de0 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47effd640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 0x55d67a6c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd47e7fc640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6c2d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd4740e4610 tx=0x7fd4740a8ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b17a962f0bfb69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd462ff5640 1 -- 172.21.15.31:0/2344768268 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740ac600 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd462ff5640 1 -- 172.21.15.31:0/2344768268 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd4740d4130 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d67a624850 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.448+0000 7fd462ff5640 1 -- 172.21.15.31:0/2344768268 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4740aca80 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.449+0000 7fd48bb0a740 10 client.25409 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.449+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d67a731de0 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.449+0000 7fd462ff5640 1 -- 172.21.15.31:0/2344768268 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d67a624850 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.449+0000 7fd462ff5640 1 -- 172.21.15.31:0/2344768268 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd4740e9070 con 0x55d67a2e01b0 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd48bb0a740 10 client.25409 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd48bb0a740 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd48bb0a740 20 client.25409 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a309070 0x55d67a73b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd48bb0a740 4 client.25409 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d67a621c80 con 0x55d67a309070 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd47effd640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a309070 0x55d67a73b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd462ff5640 10 client.25409 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.450+0000 7fd47effd640 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a309070 0x55d67a73b3e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768bb648c2abcdf2 server_cookie=1960bb0c55add2cc in_seq=0 out_seq=0 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd462ff5640 1 -- 172.21.15.31:0/2344768268 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55d67a621c80 con 0x55d67a309070 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd462ff5640 10 client.25409 handle_command_reply: tid=0 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 client.25409 shutdown 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 msgr2=0x55d67a6c2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6c2d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd4740e4610 tx=0x7fd4740a8ab0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 shutdown_connections 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55d67a309070 0x55d67a73b3e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d67a2e8390 0x55d67a731770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d67a2e01b0 0x55d67a6c2d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd462ff5640 0 client.25409 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 --2- 172.21.15.31:0/2344768268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d67a2c79f0 0x55d67a6c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 >> 172.21.15.31:0/2344768268 conn(0x55d67a2fd460 msgr2=0x55d67a6b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 shutdown_connections 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 1 -- 172.21.15.31:0/2344768268 wait complete. 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.451+0000 7fd48bb0a740 20 client.25409 trim_cache size 0 max 16384 2024-09-21T02:54:26.463 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:54:26.464 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:54:26.465 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd4680b6680 msgr2=0x7fd4680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd4680b6680 0x7fd4680b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55d67a6af2d0 tx=0x7fd470005f90 comp rx=0 tx=0).stop 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 msgr2=0x7fd480102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 0x7fd480102c70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd47400d950 tx=0x7fd47400de10 comp rx=0 tx=0).stop 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 shutdown_connections 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd4680b6680 0x7fd4680b8b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd48010de50 0x7fd4801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd480105340 0x7fd480102c70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 --2- 172.21.15.31:0/3467595534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4801049f0 0x7fd480102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 >> 172.21.15.31:0/3467595534 conn(0x7fd4800797e0 msgr2=0x7fd480107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 shutdown_connections 2024-09-21T02:54:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:26.452+0000 7fd485596640 1 -- 172.21.15.31:0/3467595534 wait complete. 2024-09-21T02:54:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:26 smithi031 ceph-mon[31209]: pgmap v371: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 694 KiB/s wr, 14 op/s 2024-09-21T02:54:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:26 smithi165 ceph-mon[34818]: pgmap v371: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 694 KiB/s wr, 14 op/s 2024-09-21T02:54:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:26 smithi178 ceph-mon[34759]: pgmap v371: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 694 KiB/s wr, 14 op/s 2024-09-21T02:54:27.354 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:54:27.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:54:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 -- 172.21.15.31:0/3433303873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a41094b0 msgr2=0x7fc0a41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 --2- 172.21.15.31:0/3433303873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a41094b0 0x7fc0a41098c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc094009900 tx=0x7fc09402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 -- 172.21.15.31:0/3433303873 shutdown_connections 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 --2- 172.21.15.31:0/3433303873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a4109e00 0x7fc0a4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 --2- 172.21.15.31:0/3433303873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a41094b0 0x7fc0a41098c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 --2- 172.21.15.31:0/3433303873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0a4101f70 0x7fc0a4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.201+0000 7fc0abe36640 1 -- 172.21.15.31:0/3433303873 >> 172.21.15.31:0/3433303873 conn(0x7fc0a4077c60 msgr2=0x7fc0a40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:28.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.202+0000 7fc0abe36640 1 -- 172.21.15.31:0/3433303873 shutdown_connections 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.202+0000 7fc0abe36640 1 -- 172.21.15.31:0/3433303873 wait complete. 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.202+0000 7fc0abe36640 1 Processor -- start 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.202+0000 7fc0abe36640 1 -- start start 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0abe36640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0a4101f70 0x7fc0a4102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0abe36640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a41094b0 0x7fc0a4103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0abe36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 0x7fc0a4103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0abe36640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4113ac0 con 0x7fc0a4109e00 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0abe36640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4113940 con 0x7fc0a41094b0 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0abe36640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a4113c40 con 0x7fc0a4101f70 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0aa3d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 0x7fc0a4103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:28.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0a93d1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a41094b0 0x7fc0a4103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0aa3d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 0x7fc0a4103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55512/0 (socket says 172.21.15.31:55512) 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0a93d1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a41094b0 0x7fc0a4103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:56498/0 (socket says 172.21.15.31:56498) 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0aa3d3640 1 -- 172.21.15.31:0/2328638021 learned_addr learned my addr 172.21.15.31:0/2328638021 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0a9bd2640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0a4101f70 0x7fc0a4102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.203+0000 7fc0aa3d3640 1 -- 172.21.15.31:0/2328638021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0a4101f70 msgr2=0x7fc0a4102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0aa3d3640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0a4101f70 0x7fc0a4102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0aa3d3640 1 -- 172.21.15.31:0/2328638021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a41094b0 msgr2=0x7fc0a4103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0aa3d3640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a41094b0 0x7fc0a4103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0aa3d3640 1 -- 172.21.15.31:0/2328638021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a41a8750 con 0x7fc0a4109e00 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0aa3d3640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 0x7fc0a4103990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00eaa0 tx=0x7fc09c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00cdb0 con 0x7fc0a4109e00 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a41a8a40 con 0x7fc0a4109e00 2024-09-21T02:54:28.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c005020 con 0x7fc0a4109e00 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0116d0 con 0x7fc0a4109e00 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.204+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a41a8f50 con 0x7fc0a4109e00 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.206+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a406ac50 con 0x7fc0a4109e00 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.207+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc09c0051d0 con 0x7fc0a4109e00 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.208+0000 7fc09affd640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0900b6680 0x7fc0900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.208+0000 7fc0a9bd2640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0900b6680 0x7fc0900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.209+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc09c0e7c10 con 0x7fc0a4109e00 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.209+0000 7fc0a9bd2640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0900b6680 0x7fc0900b8b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4104960 tx=0x7fc0a000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.212+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc09c016030 con 0x7fc0a4109e00 2024-09-21T02:54:28.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.439+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0a402e640 con 0x7fc0a4109e00 2024-09-21T02:54:28.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.440+0000 7fc09affd640 1 -- 172.21.15.31:0/2328638021 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fc09c0b2560 con 0x7fc0a4109e00 2024-09-21T02:54:28.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:54:28.442 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:54:28.442 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0900b6680 msgr2=0x7fc0900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0900b6680 0x7fc0900b8b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4104960 tx=0x7fc0a000b390 comp rx=0 tx=0).stop 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 msgr2=0x7fc0a4103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 0x7fc0a4103990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00eaa0 tx=0x7fc09c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 shutdown_connections 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0900b6680 0x7fc0900b8b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a4109e00 0x7fc0a4103990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0a41094b0 0x7fc0a4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 --2- 172.21.15.31:0/2328638021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0a4101f70 0x7fc0a4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 >> 172.21.15.31:0/2328638021 conn(0x7fc0a4077c60 msgr2=0x7fc0a41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 shutdown_connections 2024-09-21T02:54:28.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:28.442+0000 7fc0abe36640 1 -- 172.21.15.31:0/2328638021 wait complete. 2024-09-21T02:54:28.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:28 smithi031 ceph-mon[31209]: pgmap v372: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 16 op/s 2024-09-21T02:54:28.744 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:28 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2328638021' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:28 smithi165 ceph-mon[34818]: pgmap v372: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 16 op/s 2024-09-21T02:54:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:28 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2328638021' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:28 smithi178 ceph-mon[34759]: pgmap v372: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.1 MiB/s wr, 16 op/s 2024-09-21T02:54:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:28 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2328638021' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:29.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:54:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 -- 172.21.15.31:0/1100146186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4102a80 msgr2=0x7f60e4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 --2- 172.21.15.31:0/1100146186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4102a80 0x7f60e4102e90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 -- 172.21.15.31:0/1100146186 shutdown_connections 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 --2- 172.21.15.31:0/1100146186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e41083c0 0x7f60e41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 --2- 172.21.15.31:0/1100146186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e4108d10 0x7f60e410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 --2- 172.21.15.31:0/1100146186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4102a80 0x7f60e4102e90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 -- 172.21.15.31:0/1100146186 >> 172.21.15.31:0/1100146186 conn(0x7f60e4079740 msgr2=0x7f60e4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:29.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 -- 172.21.15.31:0/1100146186 shutdown_connections 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.830+0000 7f60ea132640 1 -- 172.21.15.31:0/1100146186 wait complete. 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.831+0000 7f60ea132640 1 Processor -- start 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.831+0000 7f60ea132640 1 -- start start 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.831+0000 7f60ea132640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e4102a80 0x7f60e4104270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60ea132640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e41083c0 0x7f60e41047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60ea132640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 0x7f60e4104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60ea132640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41129e0 con 0x7f60e4108d10 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60ea132640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4112860 con 0x7f60e41083c0 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60ea132640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4112b60 con 0x7f60e4102a80 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 0x7f60e4104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e4102a80 0x7f60e4104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 0x7f60e4104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43112/0 (socket says 172.21.15.31:43112) 2024-09-21T02:54:29.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 -- 172.21.15.31:0/146353278 learned_addr learned my addr 172.21.15.31:0/146353278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e41083c0 0x7f60e41047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 -- 172.21.15.31:0/146353278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e4102a80 msgr2=0x7f60e4104270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e4102a80 0x7f60e4104270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 -- 172.21.15.31:0/146353278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e41083c0 msgr2=0x7f60e41047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e41083c0 0x7f60e41047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 -- 172.21.15.31:0/146353278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e41a87a0 con 0x7f60e4108d10 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e41083c0 0x7f60e41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.832+0000 7f60e3fff640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 0x7f60e4104cf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00eb10 tx=0x7f60dc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.833+0000 7f60e0ff9640 1 -- 172.21.15.31:0/146353278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00ce20 con 0x7f60e4108d10 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.833+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e41a8a90 con 0x7f60e4108d10 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.833+0000 7f60e0ff9640 1 -- 172.21.15.31:0/146353278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60dc005020 con 0x7f60e4108d10 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.833+0000 7f60e0ff9640 1 -- 172.21.15.31:0/146353278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc010690 con 0x7f60e4108d10 2024-09-21T02:54:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.833+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e41a9050 con 0x7f60e4108d10 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.836+0000 7f60f06a6740 1 Processor -- start 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.836+0000 7f60e0ff9640 1 -- 172.21.15.31:0/146353278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f60dc010830 con 0x7f60e4108d10 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.836+0000 7f60f06a6740 1 -- start start 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb776c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb776cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60f06a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555eb76e2ea0 con 0x555eb73a4390 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60f06a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555eb76d22d0 con 0x555eb73c5070 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60f06a6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555eb76d60e0 con 0x555eb739c1b0 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43126/0 (socket says 172.21.15.31:43126) 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e2ffd640 1 -- 172.21.15.31:0/820922054 learned_addr learned my addr 172.21.15.31:0/820922054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e37fe640 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb776cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb776c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e2ffd640 1 -- 172.21.15.31:0/820922054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 msgr2=0x555eb776cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.837+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb776cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e0ff9640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60d00b6680 0x7f60d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e2ffd640 1 -- 172.21.15.31:0/820922054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 msgr2=0x555eb776c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb776c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e2ffd640 1 -- 172.21.15.31:0/820922054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb7430650 con 0x555eb73a4390 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e3fff640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60d00b6680 0x7f60d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776d120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009f10 tx=0x7f60cc009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faba2356b9e78f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60e0ff9640 1 -- 172.21.15.31:0/146353278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f60dc0e7d70 con 0x7f60e4108d10 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60d9ffb640 1 -- 172.21.15.31:0/820922054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc010750 con 0x555eb73a4390 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60d9ffb640 1 -- 172.21.15.31:0/820922054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc01c070 con 0x555eb73a4390 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.838+0000 7f60d9ffb640 1 -- 172.21.15.31:0/820922054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc017c50 con 0x555eb73a4390 2024-09-21T02:54:29.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60e3fff640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60d00b6680 0x7f60d00b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0aadf0 tx=0x7f60dc095040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 -- 172.21.15.31:0/820922054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 msgr2=0x555eb776d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776d120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009f10 tx=0x7f60cc009970 comp rx=0 tx=0).stop 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 -- 172.21.15.31:0/820922054 shutdown_connections 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb776cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776d120 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 --2- 172.21.15.31:0/820922054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb776c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 -- 172.21.15.31:0/820922054 >> 172.21.15.31:0/820922054 conn(0x555eb73b9460 msgr2=0x555eb776a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 -- 172.21.15.31:0/820922054 shutdown_connections 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 -- 172.21.15.31:0/820922054 wait complete. 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 -- start start 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.840+0000 7f60f06a6740 1 Processor -- start 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 0x555eb7777da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb73a4390 0x555eb77782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb7784170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555eb7533700 con 0x555eb739c1b0 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555eb766d160 con 0x555eb73c5070 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555eb777ae80 con 0x555eb73a4390 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60f06a6740 10 client.0 _ll_register_callbacks cb 0x555eb76869e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 0x555eb7777da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 0x555eb7777da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43132/0 (socket says 172.21.15.31:43132) 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1188901170 learned_addr learned my addr 172.21.15.31:0/1188901170 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb7784170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e37fe640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb73a4390 0x555eb77782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1188901170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb73a4390 msgr2=0x555eb77782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb73a4390 0x555eb77782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 msgr2=0x555eb7784170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb7784170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.841+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1188901170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb769ac90 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 0x555eb7777da0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f60cc017930 tx=0x7f60cc017830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3acc6d4b9d9a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60c2ffd640 1 -- 172.21.15.31:0/1188901170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc010d90 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60c2ffd640 1 -- 172.21.15.31:0/1188901170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f60cc022410 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc01c070 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60c2ffd640 1 -- 172.21.15.31:0/1188901170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60cc022610 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eb7786bf0 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60c2ffd640 1 -- 172.21.15.31:0/1188901170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc01c070 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60c2ffd640 1 -- 172.21.15.31:0/1188901170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f60cc021400 con 0x555eb739c1b0 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 10 client.15795 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 20 client.15795 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:29.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.842+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73839f0 0x555eb7789fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.843+0000 7f60f06a6740 4 client.15795 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.843+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb76e4df0 con 0x555eb73839f0 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.843+0000 7f60e37fe640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73839f0 0x555eb7789fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.843+0000 7f60e37fe640 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73839f0 0x555eb7789fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81bafcfcaf84a71a server_cookie=d241c1c16d2b6e98 in_seq=0 out_seq=0 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.843+0000 7f60c2ffd640 10 client.15795 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.844+0000 7f60c2ffd640 1 -- 172.21.15.31:0/1188901170 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555eb76e4df0 con 0x555eb73839f0 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.844+0000 7f60c2ffd640 10 client.15795 handle_command_reply: tid=0 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.844+0000 7f60f06a6740 1 client.15795 shutdown 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 msgr2=0x555eb7777da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 0x555eb7777da0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f60cc017930 tx=0x7f60cc017830 comp rx=0 tx=0).stop 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 shutdown_connections 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73839f0 0x555eb7789fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb73a4390 0x555eb77782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73c5070 0x555eb7784170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60c2ffd640 0 client.15795 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:29.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1188901170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb739c1b0 0x555eb7777da0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 >> 172.21.15.31:0/1188901170 conn(0x555eb73b9460 msgr2=0x555eb776f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:29.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 shutdown_connections 2024-09-21T02:54:29.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 1 -- 172.21.15.31:0/1188901170 wait complete. 2024-09-21T02:54:29.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.845+0000 7f60f06a6740 20 client.15795 trim_cache size 0 max 16384 2024-09-21T02:54:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.873+0000 7f60f06a6740 1 Processor -- start 2024-09-21T02:54:29.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 -- start start 2024-09-21T02:54:29.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb77799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb7769bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41b1910 con 0x555eb73a4390 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41b3450 con 0x555eb73839f0 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60f06a6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41b1fe0 con 0x555eb739c1b0 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43144/0 (socket says 172.21.15.31:43144) 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1703024947 learned_addr learned my addr 172.21.15.31:0/1703024947 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60e37fe640 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb7769bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.874+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb77799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1703024947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 msgr2=0x555eb7769bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb7769bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1703024947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 msgr2=0x555eb77799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb77799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60e2ffd640 1 -- 172.21.15.31:0/1703024947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb7779060 con 0x555eb73a4390 2024-09-21T02:54:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776a100 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f60cc021850 tx=0x7f60cc015580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22becf01af4a509 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60d97fa640 1 -- 172.21.15.31:0/1703024947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc018cf0 con 0x555eb73a4390 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60d97fa640 1 -- 172.21.15.31:0/1703024947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc01c030 con 0x555eb73a4390 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.875+0000 7f60d97fa640 1 -- 172.21.15.31:0/1703024947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc021400 con 0x555eb73a4390 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 -- 172.21.15.31:0/1703024947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 msgr2=0x555eb776a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776a100 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f60cc021850 tx=0x7f60cc015580 comp rx=0 tx=0).stop 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 -- 172.21.15.31:0/1703024947 shutdown_connections 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb7769bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb776a100 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 --2- 172.21.15.31:0/1703024947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb77799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 -- 172.21.15.31:0/1703024947 >> 172.21.15.31:0/1703024947 conn(0x555eb73b9460 msgr2=0x555eb77792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 -- 172.21.15.31:0/1703024947 shutdown_connections 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.876+0000 7f60f06a6740 1 -- 172.21.15.31:0/1703024947 wait complete. 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.877+0000 7f60f06a6740 1 -- start start 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.877+0000 7f60f06a6740 1 Processor -- start 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb778b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb778b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb77f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41b2d80 con 0x555eb73a4390 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41b26b0 con 0x555eb73839f0 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41af8f0 con 0x555eb739c1b0 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60f06a6740 10 client.0 _ll_register_callbacks cb 0x7f60d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb77f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb778b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb77f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43160/0 (socket says 172.21.15.31:43160) 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e2ffd640 1 -- 172.21.15.31:0/2064258348 learned_addr learned my addr 172.21.15.31:0/2064258348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb778b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e37fe640 1 -- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 msgr2=0x555eb778b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e37fe640 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb778b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e37fe640 1 -- 172.21.15.31:0/2064258348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 msgr2=0x555eb77f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e37fe640 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb77f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.878+0000 7f60e37fe640 1 -- 172.21.15.31:0/2064258348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb778bf70 con 0x555eb739c1b0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60e37fe640 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb778b780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60e4105cc0 tx=0x7f60d40041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293343c8a481df04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60c2ffd640 1 -- 172.21.15.31:0/2064258348 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4035040 con 0x555eb739c1b0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60c2ffd640 1 -- 172.21.15.31:0/2064258348 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f60d4005240 con 0x555eb739c1b0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc01c030 con 0x555eb739c1b0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60c2ffd640 1 -- 172.21.15.31:0/2064258348 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60d4005460 con 0x555eb739c1b0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60f06a6740 10 client.25486 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eb77f4c80 con 0x555eb739c1b0 2024-09-21T02:54:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60c2ffd640 1 -- 172.21.15.31:0/2064258348 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc01c030 con 0x555eb739c1b0 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.879+0000 7f60c2ffd640 1 -- 172.21.15.31:0/2064258348 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f60d4003860 con 0x555eb739c1b0 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60f06a6740 10 client.25486 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60f06a6740 10 client.25486 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60f06a6740 20 client.25486 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60f06a6740 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73c5070 0x555eb77f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60f06a6740 4 client.25486 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555eb76e08a0 con 0x555eb73c5070 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73c5070 0x555eb77f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60c2ffd640 10 client.25486 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.880+0000 7f60e2ffd640 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73c5070 0x555eb77f7470 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbe3ad04515c1eb8 server_cookie=7bb32684ee80f893 in_seq=0 out_seq=0 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60c2ffd640 1 -- 172.21.15.31:0/2064258348 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555eb76e08a0 con 0x555eb73c5070 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60c2ffd640 10 client.25486 handle_command_reply: tid=0 2024-09-21T02:54:29.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 client.25486 shutdown 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 msgr2=0x555eb778b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb778b780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60e4105cc0 tx=0x7f60d40041f0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 shutdown_connections 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555eb73c5070 0x555eb77f7470 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb739c1b0 0x555eb778b780 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60c2ffd640 0 client.25486 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555eb73a4390 0x555eb77f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 --2- 172.21.15.31:0/2064258348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb73839f0 0x555eb778b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 >> 172.21.15.31:0/2064258348 conn(0x555eb73b9460 msgr2=0x555eb776f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 shutdown_connections 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 1 -- 172.21.15.31:0/2064258348 wait complete. 2024-09-21T02:54:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.881+0000 7f60f06a6740 20 client.25486 trim_cache size 0 max 16384 2024-09-21T02:54:29.888 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60d00b6680 msgr2=0x7f60d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60d00b6680 0x7f60d00b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0aadf0 tx=0x7f60dc095040 comp rx=0 tx=0).stop 2024-09-21T02:54:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 msgr2=0x7f60e4104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 0x7f60e4104cf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00eb10 tx=0x7f60dc00ef10 comp rx=0 tx=0).stop 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 shutdown_connections 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60d00b6680 0x7f60d00b8b90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60e4108d10 0x7f60e4104cf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e41083c0 0x7f60e41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 --2- 172.21.15.31:0/146353278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e4102a80 0x7f60e4104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 >> 172.21.15.31:0/146353278 conn(0x7f60e4079740 msgr2=0x7f60e4106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 shutdown_connections 2024-09-21T02:54:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:29.882+0000 7f60ea132640 1 -- 172.21.15.31:0/146353278 wait complete. 2024-09-21T02:54:30.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:54:30.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:30 smithi031 ceph-mon[31209]: pgmap v373: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:30.888 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:30.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:30 smithi165 ceph-mon[34818]: pgmap v373: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:30 smithi178 ceph-mon[34759]: pgmap v373: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:31.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 -- 172.21.15.31:0/1927706848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a41049f0 msgr2=0x7f25a4104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 --2- 172.21.15.31:0/1927706848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a41049f0 0x7f25a4104e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009900 tx=0x7f25a002a410 comp rx=0 tx=0).stop 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 -- 172.21.15.31:0/1927706848 shutdown_connections 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 --2- 172.21.15.31:0/1927706848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25a410de50 0x7f25a4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 --2- 172.21.15.31:0/1927706848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25a4105340 0x7f25a410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 --2- 172.21.15.31:0/1927706848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a41049f0 0x7f25a4104e00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 -- 172.21.15.31:0/1927706848 >> 172.21.15.31:0/1927706848 conn(0x7f25a40797e0 msgr2=0x7f25a4079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 -- 172.21.15.31:0/1927706848 shutdown_connections 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.428+0000 7f25aba7d640 1 -- 172.21.15.31:0/1927706848 wait complete. 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.429+0000 7f25aba7d640 1 Processor -- start 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.429+0000 7f25aba7d640 1 -- start start 2024-09-21T02:54:31.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.429+0000 7f25aba7d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25a41049f0 0x7f25a4102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25aba7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 0x7f25a4102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25aba7d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25a410de50 0x7f25a41031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25aba7d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4113ab0 con 0x7f25a4105340 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25aba7d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4113930 con 0x7f25a41049f0 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25aba7d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4113c30 con 0x7f25a410de50 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 0x7f25a4102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9819640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25a41049f0 0x7f25a4102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 0x7f25a4102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43180/0 (socket says 172.21.15.31:43180) 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25aa01a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25a410de50 0x7f25a41031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 -- 172.21.15.31:0/3059216565 learned_addr learned my addr 172.21.15.31:0/3059216565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 -- 172.21.15.31:0/3059216565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25a410de50 msgr2=0x7f25a41031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25a410de50 0x7f25a41031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 -- 172.21.15.31:0/3059216565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25a41049f0 msgr2=0x7f25a4102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25a41049f0 0x7f25a4102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 -- 172.21.15.31:0/3059216565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a41a87e0 con 0x7f25a4105340 2024-09-21T02:54:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.430+0000 7f25a9018640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 0x7f25a4102c70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f259400c970 tx=0x7f259400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.431+0000 7f259affd640 1 -- 172.21.15.31:0/3059216565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594007bb0 con 0x7f25a4105340 2024-09-21T02:54:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.431+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a41a8ad0 con 0x7f25a4105340 2024-09-21T02:54:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.431+0000 7f259affd640 1 -- 172.21.15.31:0/3059216565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2594007d50 con 0x7f25a4105340 2024-09-21T02:54:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.431+0000 7f259affd640 1 -- 172.21.15.31:0/3059216565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594004e10 con 0x7f25a4105340 2024-09-21T02:54:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.431+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a41a9060 con 0x7f25a4105340 2024-09-21T02:54:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.433+0000 7f25b1ff1740 1 Processor -- start 2024-09-21T02:54:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 -- start start 2024-09-21T02:54:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f290c070 0x5557f2cb36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cb3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cb4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5557f2c29ea0 con 0x5557f28eb390 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5557f2c192d0 con 0x5557f290c070 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5557f2c1d0e0 con 0x5557f28e31b0 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25a9819640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cb3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25aa01a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f290c070 0x5557f2cb36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25a9819640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cb3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:50722/0 (socket says 172.21.15.31:50722) 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f25a9819640 1 -- 172.21.15.31:0/1988509531 learned_addr learned my addr 172.21.15.31:0/1988509531 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.434+0000 7f259affd640 1 -- 172.21.15.31:0/3059216565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2594005000 con 0x7f25a4105340 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25a9819640 1 -- 172.21.15.31:0/1988509531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f290c070 msgr2=0x5557f2cb36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25a9819640 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f290c070 0x5557f2cb36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25a9819640 1 -- 172.21.15.31:0/1988509531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 msgr2=0x5557f2cb4120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:54:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25a9819640 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cb4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25a9819640 1 -- 172.21.15.31:0/1988509531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f2977650 con 0x5557f28e31b0 2024-09-21T02:54:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25aa01a640 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f290c070 0x5557f2cb36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f25a9819640 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cb3be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25a406c620 tx=0x7f25a0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71d4d90cd2c15e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f258ffff640 1 -- 172.21.15.31:0/1988509531 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0040070 con 0x5557f28e31b0 2024-09-21T02:54:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f258ffff640 1 -- 172.21.15.31:0/1988509531 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a0005db0 con 0x5557f28e31b0 2024-09-21T02:54:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.435+0000 7f258ffff640 1 -- 172.21.15.31:0/1988509531 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0035040 con 0x5557f28e31b0 2024-09-21T02:54:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.436+0000 7f259affd640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25900b6680 0x7f25900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.436+0000 7f25aa01a640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25900b6680 0x7f25900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.436+0000 7f259affd640 1 -- 172.21.15.31:0/3059216565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25940e73f0 con 0x7f25a4105340 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25aa01a640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25900b6680 0x7f25900b8b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f259c005fd0 tx=0x7f259c009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1988509531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 msgr2=0x5557f2cb3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cb3be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25a406c620 tx=0x7f25a0000bc0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1988509531 shutdown_connections 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cb3be0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cb4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1988509531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f290c070 0x5557f2cb36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1988509531 >> 172.21.15.31:0/1988509531 conn(0x5557f2900460 msgr2=0x5557f2cb1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1988509531 shutdown_connections 2024-09-21T02:54:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1988509531 wait complete. 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 -- start start 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.437+0000 7f25b1ff1740 1 Processor -- start 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28e31b0 0x5557f2cbed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f290c070 0x5557f2ccb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5557f2a7a700 con 0x5557f28eb390 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5557f2bb4160 con 0x5557f28e31b0 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5557f2cc1e20 con 0x5557f290c070 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25b1ff1740 10 client.0 _ll_register_callbacks cb 0x5557f2bcd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25a9018640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25a9819640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28e31b0 0x5557f2cbed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25aa01a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f290c070 0x5557f2ccb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25a9018640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43194/0 (socket says 172.21.15.31:43194) 2024-09-21T02:54:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.438+0000 7f25a9018640 1 -- 172.21.15.31:0/3275062000 learned_addr learned my addr 172.21.15.31:0/3275062000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9018640 1 -- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f290c070 msgr2=0x5557f2ccb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9018640 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f290c070 0x5557f2ccb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9018640 1 -- 172.21.15.31:0/3275062000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28e31b0 msgr2=0x5557f2cbed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9018640 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28e31b0 0x5557f2cbed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9018640 1 -- 172.21.15.31:0/3275062000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f2be1c90 con 0x5557f28eb390 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9819640 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28e31b0 0x5557f2cbed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25a9018640 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbf280 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f25940aa580 tx=0x7f25940b6bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67af4207a7749f44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f258d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f258d7fa640 1 -- 172.21.15.31:0/3275062000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259401d030 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f258d7fa640 1 -- 172.21.15.31:0/3275062000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25940abfe0 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a0005db0 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.439+0000 7f258d7fa640 1 -- 172.21.15.31:0/3275062000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25940ad0e0 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 10 client.15810 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557f2ccdb70 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f258d7fa640 1 -- 172.21.15.31:0/3275062000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a0005db0 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f258d7fa640 1 -- 172.21.15.31:0/3275062000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25940d0be0 con 0x5557f28eb390 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 10 client.15810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 20 client.15810 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f28ca9f0 0x5557f2cd1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 4 client.15810 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f2c2bdf0 con 0x5557f28ca9f0 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.440+0000 7f25a9819640 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f28ca9f0 0x5557f2cd1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.441+0000 7f25a9819640 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f28ca9f0 0x5557f2cd1040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1116696f686edbe9 server_cookie=36a2f010abf2d342 in_seq=0 out_seq=0 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.441+0000 7f258d7fa640 10 client.15810 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f258d7fa640 1 -- 172.21.15.31:0/3275062000 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5557f2c2bdf0 con 0x5557f28ca9f0 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f258d7fa640 10 client.15810 handle_command_reply: tid=0 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 client.15810 shutdown 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 msgr2=0x5557f2cbf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbf280 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f25940aa580 tx=0x7f25940b6bb0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 shutdown_connections 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f28ca9f0 0x5557f2cd1040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f290c070 0x5557f2ccb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbf280 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f258d7fa640 0 client.15810 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/3275062000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28e31b0 0x5557f2cbed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 >> 172.21.15.31:0/3275062000 conn(0x5557f2900460 msgr2=0x5557f2cb6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 shutdown_connections 2024-09-21T02:54:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 1 -- 172.21.15.31:0/3275062000 wait complete. 2024-09-21T02:54:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.442+0000 7f25b1ff1740 20 client.15810 trim_cache size 0 max 16384 2024-09-21T02:54:31.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 Processor -- start 2024-09-21T02:54:31.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 -- start start 2024-09-21T02:54:31.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28ca9f0 0x5557f2cbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2ccb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41b18e0 con 0x5557f28eb390 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41b3420 con 0x5557f28ca9f0 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41b1fb0 con 0x5557f28e31b0 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25aa01a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25aa01a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43208/0 (socket says 172.21.15.31:43208) 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.456+0000 7f25aa01a640 1 -- 172.21.15.31:0/1350820387 learned_addr learned my addr 172.21.15.31:0/1350820387 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25a9819640 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28ca9f0 0x5557f2cbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25a9018640 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2ccb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25aa01a640 1 -- 172.21.15.31:0/1350820387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 msgr2=0x5557f2ccb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25aa01a640 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2ccb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25aa01a640 1 -- 172.21.15.31:0/1350820387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28ca9f0 msgr2=0x5557f2cbf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25aa01a640 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28ca9f0 0x5557f2cbf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25aa01a640 1 -- 172.21.15.31:0/1350820387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f2cc1480 con 0x5557f28eb390 2024-09-21T02:54:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f25aa01a640 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbbf30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f259c005f40 tx=0x7f259c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7e33d0fd00b34e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f258f7fe640 1 -- 172.21.15.31:0/1350820387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c01d430 con 0x5557f28eb390 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f258f7fe640 1 -- 172.21.15.31:0/1350820387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f259c01da80 con 0x5557f28eb390 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.457+0000 7f258f7fe640 1 -- 172.21.15.31:0/1350820387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c020c70 con 0x5557f28eb390 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1350820387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 msgr2=0x5557f2cbbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbbf30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f259c005f40 tx=0x7f259c016000 comp rx=0 tx=0).stop 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1350820387 shutdown_connections 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2ccb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28eb390 0x5557f2cbbf30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/1350820387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28ca9f0 0x5557f2cbf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.458+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1350820387 >> 172.21.15.31:0/1350820387 conn(0x5557f2900460 msgr2=0x5557f2cbeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.459+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1350820387 shutdown_connections 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.459+0000 7f25b1ff1740 1 -- 172.21.15.31:0/1350820387 wait complete. 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.459+0000 7f25b1ff1740 1 -- start start 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.459+0000 7f25b1ff1740 1 Processor -- start 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 0x5557f2cc31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cc3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28eb390 0x5557f2cc3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41b2d50 con 0x5557f28ca9f0 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41b2680 con 0x5557f28eb390 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41af8c0 con 0x5557f28e31b0 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25b1ff1740 10 client.0 _ll_register_callbacks cb 0x7f259400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 0x5557f2cc31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 0x5557f2cc31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43212/0 (socket says 172.21.15.31:43212) 2024-09-21T02:54:31.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 -- 172.21.15.31:0/2557011482 learned_addr learned my addr 172.21.15.31:0/2557011482 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9018640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cc3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25aa01a640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28eb390 0x5557f2cc3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 -- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 msgr2=0x5557f2cc3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cc3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 -- 172.21.15.31:0/2557011482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28eb390 msgr2=0x5557f2cc3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28eb390 0x5557f2cc3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.460+0000 7f25a9819640 1 -- 172.21.15.31:0/2557011482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f2d3aad0 con 0x5557f28ca9f0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f25a9819640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 0x5557f2cc31e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009900 tx=0x7f25a00051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa16b30e6834005b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f258d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f258d7fa640 1 -- 172.21.15.31:0/2557011482 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0035040 con 0x5557f28ca9f0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f258d7fa640 1 -- 172.21.15.31:0/2557011482 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25a00314a0 con 0x5557f28ca9f0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f259c01da80 con 0x5557f28ca9f0 2024-09-21T02:54:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f258d7fa640 1 -- 172.21.15.31:0/2557011482 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25a0040040 con 0x5557f28ca9f0 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f25b1ff1740 10 client.15816 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557f2d3bd30 con 0x5557f28ca9f0 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.461+0000 7f258d7fa640 1 -- 172.21.15.31:0/2557011482 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f259c01da80 con 0x5557f28ca9f0 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f258d7fa640 1 -- 172.21.15.31:0/2557011482 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25a0067020 con 0x5557f28ca9f0 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25b1ff1740 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25b1ff1740 10 client.15816 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25b1ff1740 20 client.15816 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f290c070 0x5557f2d3e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25b1ff1740 4 client.15816 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557f2c278a0 con 0x5557f290c070 2024-09-21T02:54:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25a9018640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f290c070 0x5557f2d3e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f258d7fa640 10 client.15816 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.462+0000 7f25a9018640 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f290c070 0x5557f2d3e0d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1de48af1972a6045 server_cookie=cf903203c949e68e in_seq=0 out_seq=0 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f258d7fa640 1 -- 172.21.15.31:0/2557011482 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557f2c278a0 con 0x5557f290c070 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f258d7fa640 10 client.15816 handle_command_reply: tid=0 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 client.15816 shutdown 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 msgr2=0x5557f2cc31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 0x5557f2cc31e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009900 tx=0x7f25a00051c0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 shutdown_connections 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5557f290c070 0x5557f2d3e0d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5557f28e31b0 0x5557f2cc3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557f28eb390 0x5557f2cc3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f258d7fa640 0 client.15816 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 --2- 172.21.15.31:0/2557011482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f28ca9f0 0x5557f2cc31e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 >> 172.21.15.31:0/2557011482 conn(0x5557f2900460 msgr2=0x5557f2cc01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 shutdown_connections 2024-09-21T02:54:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 1 -- 172.21.15.31:0/2557011482 wait complete. 2024-09-21T02:54:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.463+0000 7f25b1ff1740 20 client.15816 trim_cache size 0 max 16384 2024-09-21T02:54:31.470 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25900b6680 msgr2=0x7f25900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25900b6680 0x7f25900b8b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f259c005fd0 tx=0x7f259c009600 comp rx=0 tx=0).stop 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 msgr2=0x7f25a4102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 0x7f25a4102c70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f259400c970 tx=0x7f259400ce30 comp rx=0 tx=0).stop 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 shutdown_connections 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25900b6680 0x7f25900b8b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25a410de50 0x7f25a41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25a4105340 0x7f25a4102c70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 --2- 172.21.15.31:0/3059216565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25a41049f0 0x7f25a4102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.464+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 >> 172.21.15.31:0/3059216565 conn(0x7f25a40797e0 msgr2=0x7f25a4107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.465+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 shutdown_connections 2024-09-21T02:54:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:31.465+0000 7f25aba7d640 1 -- 172.21.15.31:0/3059216565 wait complete. 2024-09-21T02:54:32.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:54:32.394 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:32.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:32 smithi031 ceph-mon[31209]: pgmap v374: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-21T02:54:32.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:32 smithi165 ceph-mon[34818]: pgmap v374: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-21T02:54:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 -- 172.21.15.31:0/829741337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 msgr2=0x7fc2f41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 --2- 172.21.15.31:0/829741337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f41098d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009900 tx=0x7fc2e802a410 comp rx=0 tx=0).stop 2024-09-21T02:54:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 -- 172.21.15.31:0/829741337 shutdown_connections 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 --2- 172.21.15.31:0/829741337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 0x7fc2f4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 --2- 172.21.15.31:0/829741337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f41098d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 --2- 172.21.15.31:0/829741337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2f4103b80 0x7fc2f4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 -- 172.21.15.31:0/829741337 >> 172.21.15.31:0/829741337 conn(0x7fc2f40ff610 msgr2=0x7fc2f4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 -- 172.21.15.31:0/829741337 shutdown_connections 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.937+0000 7fc2f915d640 1 -- 172.21.15.31:0/829741337 wait complete. 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.938+0000 7fc2f915d640 1 Processor -- start 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 -- start start 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2f4103b80 0x7fc2f419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 0x7fc2f419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f4113ad0 con 0x7fc2f41094c0 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f4113950 con 0x7fc2f4109e10 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f915d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f4113c50 con 0x7fc2f4103b80 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43232/0 (socket says 172.21.15.31:43232) 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f259c640 1 -- 172.21.15.31:0/2049119506 learned_addr learned my addr 172.21.15.31:0/2049119506 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 0x7fc2f419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.939+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2f4103b80 0x7fc2f419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f259c640 1 -- 172.21.15.31:0/2049119506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2f4103b80 msgr2=0x7fc2f419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f259c640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2f4103b80 0x7fc2f419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f259c640 1 -- 172.21.15.31:0/2049119506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 msgr2=0x7fc2f419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f259c640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 0x7fc2f419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f259c640 1 -- 172.21.15.31:0/2049119506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f41a03f0 con 0x7fc2f41094c0 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f359e640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 0x7fc2f419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f259c640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f419aa20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc2e802a930 tx=0x7fc2e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2dbfff640 1 -- 172.21.15.31:0/2049119506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e803f030 con 0x7fc2f41094c0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f41a0680 con 0x7fc2f41094c0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2dbfff640 1 -- 172.21.15.31:0/2049119506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e8003aa0 con 0x7fc2f41094c0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2dbfff640 1 -- 172.21.15.31:0/2049119506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8035070 con 0x7fc2f41094c0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.940+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f41a6e40 con 0x7fc2f41094c0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 Processor -- start 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 -- start start 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2dbfff640 1 -- 172.21.15.31:0/2049119506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc2e8042030 con 0x7fc2f41094c0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed944eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944ebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed94461ea0 con 0x55ed94144070 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.943+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed944512d0 con 0x55ed9411b1b0 2024-09-21T02:54:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed944550e0 con 0x55ed94123390 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed944eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed944eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43240/0 (socket says 172.21.15.31:43240) 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 -- 172.21.15.31:0/3167056947 learned_addr learned my addr 172.21.15.31:0/3167056947 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944ebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f259c640 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 -- 172.21.15.31:0/3167056947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 msgr2=0x55ed944ec120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944ec120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 -- 172.21.15.31:0/3167056947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 msgr2=0x55ed944ebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944ebbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 -- 172.21.15.31:0/3167056947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed941af650 con 0x55ed94144070 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944ebbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.944+0000 7fc2f359e640 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed944eb6a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec010750 tx=0x7fc2ec010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d2bff3e928d1af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:32.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.945+0000 7fc2d97fa640 1 -- 172.21.15.31:0/3167056947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00ead0 con 0x55ed94144070 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.945+0000 7fc2dbfff640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc2dc0b6680 0x7fc2dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.945+0000 7fc2d97fa640 1 -- 172.21.15.31:0/3167056947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec010e60 con 0x55ed94144070 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.945+0000 7fc2d97fa640 1 -- 172.21.15.31:0/3167056947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0113e0 con 0x55ed94144070 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.945+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc2dc0b6680 0x7fc2dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.945+0000 7fc2dbfff640 1 -- 172.21.15.31:0/2049119506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc2e8107800 con 0x7fc2f41094c0 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/3167056947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 msgr2=0x55ed944eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc2dc0b6680 0x7fc2dc0b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ed944ea320 tx=0x7fc2e400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed944eb6a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec010750 tx=0x7fc2ec010c10 comp rx=0 tx=0).stop 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/3167056947 shutdown_connections 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944ec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944ebbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/3167056947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed944eb6a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/3167056947 >> 172.21.15.31:0/3167056947 conn(0x55ed94138460 msgr2=0x55ed944e9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/3167056947 shutdown_connections 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/3167056947 wait complete. 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.946+0000 7fc2ff6d1740 1 -- start start 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 Processor -- start 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed945031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed942b2700 con 0x55ed94144070 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed943ec160 con 0x55ed9411b1b0 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed944f9e80 con 0x55ed94123390 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2ff6d1740 10 client.0 _ll_register_callbacks cb 0x55ed944059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed945031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed945031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43254/0 (socket says 172.21.15.31:43254) 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 -- 172.21.15.31:0/1724667087 learned_addr learned my addr 172.21.15.31:0/1724667087 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f259c640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 -- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 msgr2=0x55ed944f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 -- 172.21.15.31:0/1724667087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 msgr2=0x55ed944f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f359e640 1 -- 172.21.15.31:0/1724667087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed94419c90 con 0x55ed94144070 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.947+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2f359e640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed945031a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec000f80 tx=0x7fc2ec004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a044126b2c9491 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1724667087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec02c060 con 0x55ed94144070 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1724667087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc2ec00eae0 con 0x55ed94144070 2024-09-21T02:54:32.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ec010e60 con 0x55ed94144070 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1724667087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec00ece0 con 0x55ed94144070 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2ff6d1740 10 client.15828 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed94505bc0 con 0x55ed94144070 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.948+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1724667087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec010e60 con 0x55ed94144070 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1724667087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc2ec00f040 con 0x55ed94144070 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2ff6d1740 10 client.15828 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2ff6d1740 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2ff6d1740 20 client.15828 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed941029f0 0x55ed94508fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2ff6d1740 4 client.15828 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed94463df0 con 0x55ed941029f0 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed941029f0 0x55ed94508fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed941029f0 0x55ed94508fd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=707e2df696002863 server_cookie=6f350eaccbda7f53 in_seq=0 out_seq=0 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.949+0000 7fc2d6ff5640 10 client.15828 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1724667087 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed94463df0 con 0x55ed941029f0 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2d6ff5640 10 client.15828 handle_command_reply: tid=0 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 client.15828 shutdown 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 msgr2=0x55ed945031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed945031a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec000f80 tx=0x7fc2ec004f00 comp rx=0 tx=0).stop 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 shutdown_connections 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed941029f0 0x55ed94508fd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed94123390 0x55ed944f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed945031a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1724667087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed9411b1b0 0x55ed944f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.950+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 >> 172.21.15.31:0/1724667087 conn(0x55ed94138460 msgr2=0x55ed944ee880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:32.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.951+0000 7fc2d6ff5640 0 client.15828 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:32.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.951+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 shutdown_connections 2024-09-21T02:54:32.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.951+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1724667087 wait complete. 2024-09-21T02:54:32.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.951+0000 7fc2ff6d1740 20 client.15828 trim_cache size 0 max 16384 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 Processor -- start 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 -- start start 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed944ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed944f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed94503110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41af770 con 0x55ed94144070 2024-09-21T02:54:32.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41b12b0 con 0x55ed94123390 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41afe40 con 0x55ed9411b1b0 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed94503110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed94503110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43270/0 (socket says 172.21.15.31:43270) 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f359e640 1 -- 172.21.15.31:0/429668287 learned_addr learned my addr 172.21.15.31:0/429668287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f259c640 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed944f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed944ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f359e640 1 -- 172.21.15.31:0/429668287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 msgr2=0x55ed944ed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.965+0000 7fc2f359e640 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed944ed1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2f359e640 1 -- 172.21.15.31:0/429668287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 msgr2=0x55ed944f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2f359e640 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed944f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2f359e640 1 -- 172.21.15.31:0/429668287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed94102c20 con 0x55ed94144070 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2f259c640 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed944f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:54:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2f359e640 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed94503110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec010750 tx=0x7fc2ec005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5964ebbafbce3759 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2d8ff9640 1 -- 172.21.15.31:0/429668287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec02c040 con 0x55ed94144070 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2d8ff9640 1 -- 172.21.15.31:0/429668287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec0057d0 con 0x55ed94144070 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.966+0000 7fc2d8ff9640 1 -- 172.21.15.31:0/429668287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01cb30 con 0x55ed94144070 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/429668287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 msgr2=0x55ed94503110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed94503110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec010750 tx=0x7fc2ec005c70 comp rx=0 tx=0).stop 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/429668287 shutdown_connections 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed94503110 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed944f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/429668287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed944ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/429668287 >> 172.21.15.31:0/429668287 conn(0x55ed94138460 msgr2=0x55ed944f07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/429668287 shutdown_connections 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.967+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/429668287 wait complete. 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 -- start start 2024-09-21T02:54:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 Processor -- start 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed9450c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed94509c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed9450a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41b0be0 con 0x55ed94144070 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41b0510 con 0x55ed94123390 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41ad750 con 0x55ed9411b1b0 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2ff6d1740 10 client.0 _ll_register_callbacks cb 0x7fc2ec016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2f259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed94509c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed9450a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2f259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed94509c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:36356/0 (socket says 172.21.15.31:36356) 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2f259c640 1 -- 172.21.15.31:0/1364028169 learned_addr learned my addr 172.21.15.31:0/1364028169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed9450a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43280/0 (socket says 172.21.15.31:43280) 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.968+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed9450c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f259c640 1 -- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 msgr2=0x55ed9450c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f259c640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed9450c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f259c640 1 -- 172.21.15.31:0/1364028169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 msgr2=0x55ed9450a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f259c640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed9450a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f259c640 1 -- 172.21.15.31:0/1364028169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9450a800 con 0x55ed94123390 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f359e640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed9450a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed9450c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2f259c640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed94509c20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80ca990 tx=0x7fc2e8031a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e535ef1cdc97ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1364028169 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8049040 con 0x55ed94123390 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1364028169 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc2e80d6c00 con 0x55ed94123390 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ec0057d0 con 0x55ed94123390 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1364028169 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2e80b4e60 con 0x55ed94123390 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2ff6d1740 10 client.25463 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.969+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed94577920 con 0x55ed94123390 2024-09-21T02:54:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1364028169 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ec0057d0 con 0x55ed94123390 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1364028169 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc2e804c060 con 0x55ed94123390 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2ff6d1740 10 client.25463 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2ff6d1740 10 client.25463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2ff6d1740 20 client.25463 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed940d7da0 0x55ed9457a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2ff6d1740 4 client.25463 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed9445f8a0 con 0x55ed940d7da0 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.970+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed940d7da0 0x55ed9457a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2d6ff5640 10 client.25463 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2f2d9d640 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed940d7da0 0x55ed9457a110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deea6f7166cec074 server_cookie=d533bcdf1aa02de in_seq=0 out_seq=0 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2d6ff5640 1 -- 172.21.15.31:0/1364028169 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed9445f8a0 con 0x55ed940d7da0 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2d6ff5640 10 client.25463 handle_command_reply: tid=0 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 client.25463 shutdown 2024-09-21T02:54:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 msgr2=0x55ed94509c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed94509c20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80ca990 tx=0x7fc2e8031a10 comp rx=0 tx=0).stop 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 shutdown_connections 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55ed940d7da0 0x55ed9457a110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed94144070 0x55ed9450a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed94123390 0x55ed94509c20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2d6ff5640 0 client.25463 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 --2- 172.21.15.31:0/1364028169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed9411b1b0 0x55ed9450c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 >> 172.21.15.31:0/1364028169 conn(0x55ed94138460 msgr2=0x55ed944f07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 shutdown_connections 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.971+0000 7fc2ff6d1740 1 -- 172.21.15.31:0/1364028169 wait complete. 2024-09-21T02:54:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2ff6d1740 20 client.25463 trim_cache size 0 max 16384 2024-09-21T02:54:32.987 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:32.987 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:32 smithi178 ceph-mon[34759]: pgmap v374: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-21T02:54:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc2dc0b6680 msgr2=0x7fc2dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc2dc0b6680 0x7fc2dc0b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ed944ea320 tx=0x7fc2e400b390 comp rx=0 tx=0).stop 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 msgr2=0x7fc2f419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f419aa20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc2e802a930 tx=0x7fc2e8004270 comp rx=0 tx=0).stop 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 shutdown_connections 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc2dc0b6680 0x7fc2dc0b8b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2f4109e10 0x7fc2f419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2f41094c0 0x7fc2f419aa20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 --2- 172.21.15.31:0/2049119506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2f4103b80 0x7fc2f419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.972+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 >> 172.21.15.31:0/2049119506 conn(0x7fc2f40ff610 msgr2=0x7fc2f4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.973+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 shutdown_connections 2024-09-21T02:54:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:32.973+0000 7fc2f915d640 1 -- 172.21.15.31:0/2049119506 wait complete. 2024-09-21T02:54:33.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:54:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/4009188690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 msgr2=0x7fbf74108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/4009188690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf74108610 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009900 tx=0x7fbf6402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/4009188690 shutdown_connections 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/4009188690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf74108b50 0x7fbf7410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/4009188690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf74108610 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/4009188690 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf741028c0 0x7fbf74102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.492+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/4009188690 >> 172.21.15.31:0/4009188690 conn(0x7fbf740fe350 msgr2=0x7fbf741007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.493+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/4009188690 shutdown_connections 2024-09-21T02:54:34.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.493+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/4009188690 wait complete. 2024-09-21T02:54:34.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.493+0000 7fbf7ab9f640 1 Processor -- start 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 -- start start 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf741028c0 0x7fbf7419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf7419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf74108b50 0x7fbf7419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741127b0 con 0x7fbf74108200 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112630 con 0x7fbf74108b50 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7ab9f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112930 con 0x7fbf741028c0 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7939c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf7419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7939c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf7419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43308/0 (socket says 172.21.15.31:43308) 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7939c640 1 -- 172.21.15.31:0/1152070535 learned_addr learned my addr 172.21.15.31:0/1152070535 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf741028c0 0x7fbf7419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf74108b50 0x7fbf7419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7939c640 1 -- 172.21.15.31:0/1152070535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf741028c0 msgr2=0x7fbf7419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7939c640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf741028c0 0x7fbf7419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.494+0000 7fbf7939c640 1 -- 172.21.15.31:0/1152070535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf74108b50 msgr2=0x7fbf7419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf7939c640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf74108b50 0x7fbf7419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf7939c640 1 -- 172.21.15.31:0/1152070535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741a0160 con 0x7fbf74108200 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf7939c640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf7419a850 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402a930 tx=0x7fbf64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf6affd640 1 -- 172.21.15.31:0/1152070535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6403f030 con 0x7fbf74108200 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf741a03f0 con 0x7fbf74108200 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf6affd640 1 -- 172.21.15.31:0/1152070535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf64003aa0 con 0x7fbf74108200 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf6affd640 1 -- 172.21.15.31:0/1152070535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64035070 con 0x7fbf74108200 2024-09-21T02:54:34.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.495+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a0920 con 0x7fbf74108200 2024-09-21T02:54:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 Processor -- start 2024-09-21T02:54:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 -- start start 2024-09-21T02:54:34.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789cb8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf6affd640 1 -- 172.21.15.31:0/1152070535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbf64042030 con 0x7fbf74108200 2024-09-21T02:54:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898f0390 0x557789cb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557789c2eea0 con 0x5577898e81b0 2024-09-21T02:54:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.498+0000 7fbf82375740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557789c1e2d0 con 0x557789911070 2024-09-21T02:54:34.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf82375740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557789c220e0 con 0x5577898f0390 2024-09-21T02:54:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf7a39e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789cb8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf7a39e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789cb8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:36378/0 (socket says 172.21.15.31:36378) 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cb8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43322/0 (socket says 172.21.15.31:43322) 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf7a39e640 1 -- 172.21.15.31:0/1905182957 learned_addr learned my addr 172.21.15.31:0/1905182957 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf7939c640 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898f0390 0x557789cb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 -- 172.21.15.31:0/1905182957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898f0390 msgr2=0x557789cb92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898f0390 0x557789cb92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 -- 172.21.15.31:0/1905182957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 msgr2=0x557789cb8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789cb8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf79b9d640 1 -- 172.21.15.31:0/1905182957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778997c650 con 0x5577898e81b0 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.499+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789cb8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cb8da0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000abf0 tx=0x7fbf7000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4128fd024f527203 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf5ffff640 1 -- 172.21.15.31:0/1905182957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70005220 con 0x5577898e81b0 2024-09-21T02:54:34.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf6affd640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbf600b6730 0x7fbf600b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf5ffff640 1 -- 172.21.15.31:0/1905182957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf70004c50 con 0x5577898e81b0 2024-09-21T02:54:34.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf5ffff640 1 -- 172.21.15.31:0/1905182957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70018730 con 0x5577898e81b0 2024-09-21T02:54:34.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbf600b6730 0x7fbf600b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.500+0000 7fbf6affd640 1 -- 172.21.15.31:0/1152070535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbf64107700 con 0x7fbf74108200 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 -- 172.21.15.31:0/1905182957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 msgr2=0x557789cb8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cb8da0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000abf0 tx=0x7fbf7000f790 comp rx=0 tx=0).stop 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbf600b6730 0x7fbf600b8c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c005f40 tx=0x7fbf6c009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 -- 172.21.15.31:0/1905182957 shutdown_connections 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898f0390 0x557789cb92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cb8da0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 --2- 172.21.15.31:0/1905182957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789cb8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 -- 172.21.15.31:0/1905182957 >> 172.21.15.31:0/1905182957 conn(0x557789905460 msgr2=0x557789cb6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 -- 172.21.15.31:0/1905182957 shutdown_connections 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 -- 172.21.15.31:0/1905182957 wait complete. 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.501+0000 7fbf82375740 1 -- start start 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 Processor -- start 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557789911070 0x557789cd0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557789a7f700 con 0x5577898e81b0 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557789bb9160 con 0x5577898f0390 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557789cc7040 con 0x557789911070 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf82375740 10 client.0 _ll_register_callbacks cb 0x557789bd29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:34.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf79b9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf79b9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cc3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43326/0 (socket says 172.21.15.31:43326) 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf79b9d640 1 -- 172.21.15.31:0/3239302802 learned_addr learned my addr 172.21.15.31:0/3239302802 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.502+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557789911070 0x557789cd0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf7939c640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf79b9d640 1 -- 172.21.15.31:0/3239302802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557789911070 msgr2=0x557789cd0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557789911070 0x557789cd0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf79b9d640 1 -- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 msgr2=0x557789cc4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cc4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf79b9d640 1 -- 172.21.15.31:0/3239302802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557789be6c90 con 0x5577898e81b0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cc3f40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbf7001a040 tx=0x7fbf70004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd787c296c6c80d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/3239302802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70027070 con 0x5577898e81b0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf70004c50 con 0x5577898e81b0 2024-09-21T02:54:34.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/3239302802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbf70005010 con 0x5577898e81b0 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.503+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/3239302802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf70033050 con 0x5577898e81b0 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 10 client.15846 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557789cd7180 con 0x5577898e81b0 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/3239302802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf70004c50 con 0x5577898e81b0 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/3239302802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf70023430 con 0x5577898e81b0 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 20 client.15846 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898cf9f0 0x557789cda5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 4 client.15846 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557789c30df0 con 0x5577898cf9f0 2024-09-21T02:54:34.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.504+0000 7fbf7939c640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898cf9f0 0x557789cda5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.505+0000 7fbf7939c640 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898cf9f0 0x557789cda5a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3b49e249f25cdb2 server_cookie=7da5d73faf17c1a1 in_seq=0 out_seq=0 2024-09-21T02:54:34.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.505+0000 7fbf5d7fa640 10 client.15846 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/3239302802 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557789c30df0 con 0x5577898cf9f0 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf5d7fa640 10 client.15846 handle_command_reply: tid=0 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 client.15846 shutdown 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 msgr2=0x557789cc3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cc3f40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbf7001a040 tx=0x7fbf70004b10 comp rx=0 tx=0).stop 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 shutdown_connections 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898cf9f0 0x557789cda5a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557789911070 0x557789cd0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cc4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 --2- 172.21.15.31:0/3239302802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898e81b0 0x557789cc3f40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 >> 172.21.15.31:0/3239302802 conn(0x557789905460 msgr2=0x557789cbb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:34.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf5d7fa640 0 client.15846 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:34.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 shutdown_connections 2024-09-21T02:54:34.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 1 -- 172.21.15.31:0/3239302802 wait complete. 2024-09-21T02:54:34.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.506+0000 7fbf82375740 20 client.15846 trim_cache size 0 max 16384 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 Processor -- start 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 -- start start 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cc5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cb9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 0x557789cb5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ad300 con 0x557789911070 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741aee40 con 0x5577898f0390 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf82375740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ad9d0 con 0x5577898e81b0 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf7a39e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 0x557789cb5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf7a39e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 0x557789cb5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43330/0 (socket says 172.21.15.31:43330) 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.520+0000 7fbf7a39e640 1 -- 172.21.15.31:0/1892519958 learned_addr learned my addr 172.21.15.31:0/1892519958 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7939c640 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cb9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cc5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7a39e640 1 -- 172.21.15.31:0/1892519958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 msgr2=0x557789cc5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cc5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7a39e640 1 -- 172.21.15.31:0/1892519958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 msgr2=0x557789cb9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cb9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7a39e640 1 -- 172.21.15.31:0/1892519958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557789cb9680 con 0x557789911070 2024-09-21T02:54:34.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 0x557789cb5d20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c011db0 tx=0x7fbf6c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f57b170cf287c3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:34.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf5f7fe640 1 -- 172.21.15.31:0/1892519958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c004b20 con 0x557789911070 2024-09-21T02:54:34.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf5f7fe640 1 -- 172.21.15.31:0/1892519958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6c003e20 con 0x557789911070 2024-09-21T02:54:34.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.521+0000 7fbf5f7fe640 1 -- 172.21.15.31:0/1892519958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c004740 con 0x557789911070 2024-09-21T02:54:34.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 -- 172.21.15.31:0/1892519958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 msgr2=0x557789cb5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 0x557789cb5d20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c011db0 tx=0x7fbf6c00b000 comp rx=0 tx=0).stop 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 -- 172.21.15.31:0/1892519958 shutdown_connections 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557789911070 0x557789cb5d20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5577898f0390 0x557789cb9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 --2- 172.21.15.31:0/1892519958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cc5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 -- 172.21.15.31:0/1892519958 >> 172.21.15.31:0/1892519958 conn(0x557789905460 msgr2=0x557789cb9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 -- 172.21.15.31:0/1892519958 shutdown_connections 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.522+0000 7fbf82375740 1 -- 172.21.15.31:0/1892519958 wait complete. 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.523+0000 7fbf82375740 1 -- start start 2024-09-21T02:54:34.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.523+0000 7fbf82375740 1 Processor -- start 2024-09-21T02:54:34.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.523+0000 7fbf82375740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cdafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.523+0000 7fbf82375740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 0x557789cdb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf82375740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789d3cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf82375740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ae770 con 0x5577898f0390 2024-09-21T02:54:34.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf82375740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ae0a0 con 0x557789911070 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf82375740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ab2e0 con 0x5577898e81b0 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf82375740 10 client.0 _ll_register_callbacks cb 0x7fbf6c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 0x557789cdb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 0x557789cdb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43344/0 (socket says 172.21.15.31:43344) 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 -- 172.21.15.31:0/769453675 learned_addr learned my addr 172.21.15.31:0/769453675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7a39e640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789d3cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cdafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 -- 172.21.15.31:0/769453675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 msgr2=0x557789cdafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cdafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 -- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 msgr2=0x557789d3cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789d3cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 -- 172.21.15.31:0/769453675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557789d3d3c0 con 0x5577898f0390 2024-09-21T02:54:34.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf7939c640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 0x557789cdb5c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf640cd500 tx=0x7fbf6410d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38314ab777a8166 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.524+0000 7fbf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/769453675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64049040 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/769453675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbf640d6b00 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6c003e20 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/769453675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf640ccec0 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 10 client.15852 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557789d44810 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/769453675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6c003e20 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/769453675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf6404c060 con 0x5577898f0390 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 10 client.15852 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 10 client.15852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 20 client.15852 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898a4da0 0x557789d46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:34.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 4 client.15852 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:34.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.525+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557789c2c8a0 con 0x5577898a4da0 2024-09-21T02:54:34.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.526+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898a4da0 0x557789d46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:34.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.526+0000 7fbf5d7fa640 10 client.15852 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:34.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.526+0000 7fbf79b9d640 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898a4da0 0x557789d46f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db0804c16ac3d598 server_cookie=1dae5395c92a4b75 in_seq=0 out_seq=0 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.526+0000 7fbf5d7fa640 1 -- 172.21.15.31:0/769453675 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557789c2c8a0 con 0x5577898a4da0 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.526+0000 7fbf5d7fa640 10 client.15852 handle_command_reply: tid=0 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.526+0000 7fbf82375740 1 client.15852 shutdown 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 msgr2=0x557789cdb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 0x557789cdb5c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf640cd500 tx=0x7fbf6410d550 comp rx=0 tx=0).stop 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 shutdown_connections 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5577898a4da0 0x557789d46f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557789911070 0x557789d3cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf5d7fa640 0 client.15852 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5577898f0390 0x557789cdb5c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 --2- 172.21.15.31:0/769453675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5577898e81b0 0x557789cdafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 >> 172.21.15.31:0/769453675 conn(0x557789905460 msgr2=0x557789cba020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:34.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 shutdown_connections 2024-09-21T02:54:34.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 1 -- 172.21.15.31:0/769453675 wait complete. 2024-09-21T02:54:34.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.527+0000 7fbf82375740 20 client.15852 trim_cache size 0 max 16384 2024-09-21T02:54:34.533 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbf600b6730 msgr2=0x7fbf600b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbf600b6730 0x7fbf600b8c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c005f40 tx=0x7fbf6c009600 comp rx=0 tx=0).stop 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 msgr2=0x7fbf7419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf7419a850 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402a930 tx=0x7fbf64004270 comp rx=0 tx=0).stop 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 shutdown_connections 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbf600b6730 0x7fbf600b8c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf74108b50 0x7fbf7419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf74108200 0x7fbf7419a850 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 --2- 172.21.15.31:0/1152070535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf741028c0 0x7fbf7419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 >> 172.21.15.31:0/1152070535 conn(0x7fbf740fe350 msgr2=0x7fbf7406bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:34.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 shutdown_connections 2024-09-21T02:54:34.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:34.528+0000 7fbf7ab9f640 1 -- 172.21.15.31:0/1152070535 wait complete. 2024-09-21T02:54:34.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:34 smithi031 ceph-mon[31209]: pgmap v375: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 866 KiB/s wr, 14 op/s 2024-09-21T02:54:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:34 smithi165 ceph-mon[34818]: pgmap v375: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 866 KiB/s wr, 14 op/s 2024-09-21T02:54:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:34 smithi178 ceph-mon[34759]: pgmap v375: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 866 KiB/s wr, 14 op/s 2024-09-21T02:54:35.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:54:35.517 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 -- 172.21.15.31:0/1804607589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9318102ae0 msgr2=0x7f9318102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 --2- 172.21.15.31:0/1804607589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9318102ae0 0x7f9318102ef0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9308009900 tx=0x7f930802a410 comp rx=0 tx=0).stop 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 -- 172.21.15.31:0/1804607589 shutdown_connections 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 --2- 172.21.15.31:0/1804607589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93181083d0 0x7f93181087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 --2- 172.21.15.31:0/1804607589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 0x7f931810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 --2- 172.21.15.31:0/1804607589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9318102ae0 0x7f9318102ef0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 -- 172.21.15.31:0/1804607589 >> 172.21.15.31:0/1804607589 conn(0x7f93180797b0 msgr2=0x7f9318079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 -- 172.21.15.31:0/1804607589 shutdown_connections 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.049+0000 7f931db46640 1 -- 172.21.15.31:0/1804607589 wait complete. 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.050+0000 7f931db46640 1 Processor -- start 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.050+0000 7f931db46640 1 -- start start 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f931db46640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9318102ae0 0x7f931819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f931db46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 0x7f931819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f931db46640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 0x7f931819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f931db46640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9318113b00 con 0x7f93181083d0 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f931db46640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9318113980 con 0x7f9318108d20 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f931db46640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9318113c80 con 0x7f9318102ae0 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f930effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 0x7f931819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f930effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 0x7f931819aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43358/0 (socket says 172.21.15.31:43358) 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f930effd640 1 -- 172.21.15.31:0/977386668 learned_addr learned my addr 172.21.15.31:0/977386668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f93177fe640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9318102ae0 0x7f931819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.051+0000 7f9317fff640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 0x7f931819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f930effd640 1 -- 172.21.15.31:0/977386668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9318102ae0 msgr2=0x7f931819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f930effd640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9318102ae0 0x7f931819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f930effd640 1 -- 172.21.15.31:0/977386668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 msgr2=0x7f931819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f930effd640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 0x7f931819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f930effd640 1 -- 172.21.15.31:0/977386668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93181a0400 con 0x7f93181083d0 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f9317fff640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 0x7f931819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f930effd640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 0x7f931819aa30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f930000d750 tx=0x7f930000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f93157fa640 1 -- 172.21.15.31:0/977386668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300017070 con 0x7f93181083d0 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f93157fa640 1 -- 172.21.15.31:0/977386668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93000054d0 con 0x7f93181083d0 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93181a06f0 con 0x7f93181083d0 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f93157fa640 1 -- 172.21.15.31:0/977386668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300004c30 con 0x7f93181083d0 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.052+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93181a6e50 con 0x7f93181083d0 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.055+0000 7f93240ba740 1 Processor -- start 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.055+0000 7f93240ba740 1 -- start start 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.055+0000 7f93157fa640 1 -- 172.21.15.31:0/977386668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9300010460 con 0x7f93181083d0 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93240ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x5632088fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93240ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56320852e1b0 0x5632088fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93240ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x5632088fed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93240ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563208874ea0 con 0x563208557070 2024-09-21T02:54:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93240ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632088642d0 con 0x56320852e1b0 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93240ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5632088680e0 con 0x563208536390 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x5632088fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x5632088fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43366/0 (socket says 172.21.15.31:43366) 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 -- 172.21.15.31:0/597006166 learned_addr learned my addr 172.21.15.31:0/597006166 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f93177fe640 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56320852e1b0 0x5632088fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f930effd640 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x5632088fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 -- 172.21.15.31:0/597006166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 msgr2=0x5632088fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x5632088fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 -- 172.21.15.31:0/597006166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56320852e1b0 msgr2=0x5632088fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56320852e1b0 0x5632088fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 -- 172.21.15.31:0/597006166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632085c2650 con 0x563208557070 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.056+0000 7f9317fff640 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x5632088fe2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9310009fb0 tx=0x7f9310010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274046fb3ce7a11d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:36.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.057+0000 7f930dffb640 1 -- 172.21.15.31:0/597006166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310016070 con 0x563208557070 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.057+0000 7f93157fa640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93040b6680 0x7f93040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.057+0000 7f930dffb640 1 -- 172.21.15.31:0/597006166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9310012bc0 con 0x563208557070 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.057+0000 7f930dffb640 1 -- 172.21.15.31:0/597006166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310011ba0 con 0x563208557070 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.057+0000 7f93177fe640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93040b6680 0x7f93040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.057+0000 7f93157fa640 1 -- 172.21.15.31:0/977386668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f93000e70c0 con 0x7f93181083d0 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 -- 172.21.15.31:0/597006166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 msgr2=0x5632088fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93177fe640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93040b6680 0x7f93040b8b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9308031440 tx=0x7f9308009610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x5632088fe2c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9310009fb0 tx=0x7f9310010800 comp rx=0 tx=0).stop 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 -- 172.21.15.31:0/597006166 shutdown_connections 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x5632088fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56320852e1b0 0x5632088fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 --2- 172.21.15.31:0/597006166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x5632088fe2c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 -- 172.21.15.31:0/597006166 >> 172.21.15.31:0/597006166 conn(0x56320854b460 msgr2=0x5632088fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 -- 172.21.15.31:0/597006166 shutdown_connections 2024-09-21T02:54:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 -- 172.21.15.31:0/597006166 wait complete. 2024-09-21T02:54:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.058+0000 7f93240ba740 1 -- start start 2024-09-21T02:54:36.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 Processor -- start 2024-09-21T02:54:36.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320852e1b0 0x5632089099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563208536390 0x563208909f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x563208915d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5632086c5700 con 0x563208557070 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632087ff160 con 0x563208536390 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56320890caa0 con 0x56320852e1b0 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93240ba740 10 client.0 _ll_register_callbacks cb 0x5632088189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f9317fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x563208915d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f9317fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x563208915d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43368/0 (socket says 172.21.15.31:43368) 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f9317fff640 1 -- 172.21.15.31:0/1888109629 learned_addr learned my addr 172.21.15.31:0/1888109629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.059+0000 7f93177fe640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320852e1b0 0x5632089099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f930effd640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563208536390 0x563208909f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93177fe640 1 -- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563208536390 msgr2=0x563208909f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93177fe640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563208536390 0x563208909f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93177fe640 1 -- 172.21.15.31:0/1888109629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 msgr2=0x563208915d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93177fe640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x563208915d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93177fe640 1 -- 172.21.15.31:0/1888109629 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320882cc90 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f9317fff640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x563208915d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93177fe640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320852e1b0 0x5632089099c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9308009230 tx=0x7f9308009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f4c37b44c96ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f92f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1888109629 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930804d070 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9310012bc0 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.060+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1888109629 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9308005030 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1888109629 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9308059050 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 10 client.25546 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56320891cba0 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1888109629 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9310012bc0 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1888109629 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93080035f0 con 0x56320852e1b0 2024-09-21T02:54:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 10 client.25546 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 20 client.25546 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5632085159f0 0x563208920070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 4 client.25546 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.061+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563208876df0 con 0x5632085159f0 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.062+0000 7f930effd640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5632085159f0 0x563208920070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.062+0000 7f930effd640 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5632085159f0 0x563208920070 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac7bdecd3f04c5a server_cookie=d7d7865bae11587c in_seq=0 out_seq=0 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.062+0000 7f92f6ffd640 10 client.25546 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1888109629 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563208876df0 con 0x5632085159f0 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f92f6ffd640 10 client.25546 handle_command_reply: tid=0 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 client.25546 shutdown 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320852e1b0 msgr2=0x5632089099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320852e1b0 0x5632089099c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9308009230 tx=0x7f9308009260 comp rx=0 tx=0).stop 2024-09-21T02:54:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 shutdown_connections 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5632085159f0 0x563208920070 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563208557070 0x563208915d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f92f6ffd640 0 client.25546 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563208536390 0x563208909f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 --2- 172.21.15.31:0/1888109629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56320852e1b0 0x5632089099c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 >> 172.21.15.31:0/1888109629 conn(0x56320854b460 msgr2=0x5632089014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 shutdown_connections 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 1 -- 172.21.15.31:0/1888109629 wait complete. 2024-09-21T02:54:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.063+0000 7f93240ba740 20 client.25546 trim_cache size 0 max 16384 2024-09-21T02:54:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.092+0000 7f93240ba740 1 Processor -- start 2024-09-21T02:54:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.092+0000 7f93240ba740 1 -- start start 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93240ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x563208915c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93240ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x5632088ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93240ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93240ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93181ad450 con 0x56320852e1b0 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93240ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93181aef90 con 0x5632085159f0 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93240ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93181adb20 con 0x563208536390 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x5632088ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x5632088ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43370/0 (socket says 172.21.15.31:43370) 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 -- 172.21.15.31:0/56817521 learned_addr learned my addr 172.21.15.31:0/56817521 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:36.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f9317fff640 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f93177fe640 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x563208915c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 -- 172.21.15.31:0/56817521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 msgr2=0x56320890b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 -- 172.21.15.31:0/56817521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 msgr2=0x563208915c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x563208915c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.093+0000 7f930effd640 1 -- 172.21.15.31:0/56817521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632089007e0 con 0x56320852e1b0 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.094+0000 7f930effd640 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x5632088ffcb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f93000aa240 tx=0x7f9300094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddb992860dd62db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:36.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.094+0000 7f930d7fa640 1 -- 172.21.15.31:0/56817521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300020040 con 0x56320852e1b0 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.094+0000 7f930d7fa640 1 -- 172.21.15.31:0/56817521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93000acae0 con 0x56320852e1b0 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.094+0000 7f930d7fa640 1 -- 172.21.15.31:0/56817521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93000e4b50 con 0x56320852e1b0 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 -- 172.21.15.31:0/56817521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 msgr2=0x5632088ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x5632088ffcb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f93000aa240 tx=0x7f9300094040 comp rx=0 tx=0).stop 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 -- 172.21.15.31:0/56817521 shutdown_connections 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x5632088ffcb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 --2- 172.21.15.31:0/56817521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x563208915c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 -- 172.21.15.31:0/56817521 >> 172.21.15.31:0/56817521 conn(0x56320854b460 msgr2=0x56320890aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 -- 172.21.15.31:0/56817521 shutdown_connections 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 -- 172.21.15.31:0/56817521 wait complete. 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.095+0000 7f93240ba740 1 -- start start 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 Processor -- start 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x5632089121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x56320890e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93181ae8c0 con 0x56320852e1b0 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93181ae1f0 con 0x5632085159f0 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93181ab430 con 0x563208536390 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93240ba740 10 client.0 _ll_register_callbacks cb 0x7f9308005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:36.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f930effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x56320890e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f930effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x56320890e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43382/0 (socket says 172.21.15.31:43382) 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f930effd640 1 -- 172.21.15.31:0/1611294400 learned_addr learned my addr 172.21.15.31:0/1611294400 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f9317fff640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.096+0000 7f93177fe640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x5632089121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f930effd640 1 -- 172.21.15.31:0/1611294400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 msgr2=0x56320890ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f930effd640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f930effd640 1 -- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 msgr2=0x5632089121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f930effd640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x5632089121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f930effd640 1 -- 172.21.15.31:0/1611294400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320890f0f0 con 0x56320852e1b0 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f93177fe640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x5632089121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f930effd640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x56320890e670 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f93000a95a0 tx=0x7f93000ac7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52247a32691484f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f92f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:36.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1611294400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93000d4060 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1611294400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f93000a9240 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93000acae0 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1611294400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9300020050 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f93240ba740 10 client.15870 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.097+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56320898b660 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1611294400 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93000acae0 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1611294400 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93000d4200 con 0x56320852e1b0 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93240ba740 10 client.15870 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93240ba740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93240ba740 20 client.15870 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93240ba740 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563208557070 0x56320898dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93240ba740 4 client.15870 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:36.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56320890a2b0 con 0x563208557070 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f93177fe640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563208557070 0x56320898dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.098+0000 7f92f6ffd640 10 client.15870 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93177fe640 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563208557070 0x56320898dd30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5312fb3bba0daf28 server_cookie=1668729becd74dbb in_seq=0 out_seq=0 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f92f6ffd640 1 -- 172.21.15.31:0/1611294400 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56320890a2b0 con 0x563208557070 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f92f6ffd640 10 client.15870 handle_command_reply: tid=0 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 client.15870 shutdown 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 msgr2=0x56320890e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x56320890e670 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f93000a95a0 tx=0x7f93000ac7b0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 shutdown_connections 2024-09-21T02:54:36.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563208557070 0x56320898dd30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563208536390 0x56320890ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56320852e1b0 0x56320890e670 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 --2- 172.21.15.31:0/1611294400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632085159f0 0x5632089121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f92f6ffd640 0 client.15870 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 >> 172.21.15.31:0/1611294400 conn(0x56320854b460 msgr2=0x56320890b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 shutdown_connections 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 1 -- 172.21.15.31:0/1611294400 wait complete. 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.099+0000 7f93240ba740 20 client.15870 trim_cache size 0 max 16384 2024-09-21T02:54:36.106 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93040b6680 msgr2=0x7f93040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93040b6680 0x7f93040b8b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9308031440 tx=0x7f9308009610 comp rx=0 tx=0).stop 2024-09-21T02:54:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 msgr2=0x7f931819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 0x7f931819aa30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f930000d750 tx=0x7f930000dc10 comp rx=0 tx=0).stop 2024-09-21T02:54:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 shutdown_connections 2024-09-21T02:54:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f93040b6680 0x7f93040b8b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9318108d20 0x7f931819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93181083d0 0x7f931819aa30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 --2- 172.21.15.31:0/977386668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9318102ae0 0x7f931819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.100+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 >> 172.21.15.31:0/977386668 conn(0x7f93180797b0 msgr2=0x7f9318106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.101+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 shutdown_connections 2024-09-21T02:54:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:36.101+0000 7f931db46640 1 -- 172.21.15.31:0/977386668 wait complete. 2024-09-21T02:54:36.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:36 smithi165 ceph-mon[34818]: pgmap v376: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:36 smithi178 ceph-mon[34759]: pgmap v376: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:36 smithi031 ceph-mon[31209]: pgmap v376: 97 pgs: 97 active+clean; 2.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:37.825 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:54:37.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:54:38.100 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.599+0000 7ff947fdf640 1 -- 172.21.15.31:0/176852126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940102a60 msgr2=0x7ff940102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.599+0000 7ff947fdf640 1 --2- 172.21.15.31:0/176852126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940102a60 0x7ff940102e70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009900 tx=0x7ff93c02a410 comp rx=0 tx=0).stop 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 -- 172.21.15.31:0/176852126 shutdown_connections 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 --2- 172.21.15.31:0/176852126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9401083a0 0x7ff9401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 --2- 172.21.15.31:0/176852126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940108cf0 0x7ff94010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 --2- 172.21.15.31:0/176852126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940102a60 0x7ff940102e70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 -- 172.21.15.31:0/176852126 >> 172.21.15.31:0/176852126 conn(0x7ff940079730 msgr2=0x7ff940079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 -- 172.21.15.31:0/176852126 shutdown_connections 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.600+0000 7ff947fdf640 1 -- 172.21.15.31:0/176852126 wait complete. 2024-09-21T02:54:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 Processor -- start 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 -- start start 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940102a60 0x7ff9401990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9401083a0 0x7ff940199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 0x7ff940199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401129b0 con 0x7ff940108cf0 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff940112830 con 0x7ff940102a60 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff947fdf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff940112b30 con 0x7ff9401083a0 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff94657c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 0x7ff940199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.601+0000 7ff94657c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 0x7ff940199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43402/0 (socket says 172.21.15.31:43402) 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 -- 172.21.15.31:0/1146658330 learned_addr learned my addr 172.21.15.31:0/1146658330 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94557a640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9401083a0 0x7ff940199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940102a60 0x7ff9401990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 -- 172.21.15.31:0/1146658330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9401083a0 msgr2=0x7ff940199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9401083a0 0x7ff940199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 -- 172.21.15.31:0/1146658330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940102a60 msgr2=0x7ff9401990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940102a60 0x7ff9401990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 -- 172.21.15.31:0/1146658330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9401a87a0 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff94657c640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 0x7ff940199b70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff93800efd0 tx=0x7ff93800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff936ffd640 1 -- 172.21.15.31:0/1146658330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938003b70 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9401a8a90 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff936ffd640 1 -- 172.21.15.31:0/1146658330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff938003d10 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.602+0000 7ff936ffd640 1 -- 172.21.15.31:0/1146658330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380108a0 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.603+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9401a8fa0 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.606+0000 7ff936ffd640 1 -- 172.21.15.31:0/1146658330 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff938010b40 con 0x7ff940108cf0 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 Processor -- start 2024-09-21T02:54:38.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 -- start start 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff936ffd640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff92c0b6680 0x7ff92c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 0x55e0b211d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94557a640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff92c0b6680 0x7ff92c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b211d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff936ffd640 1 -- 172.21.15.31:0/1146658330 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff9380e77d0 con 0x7ff940108cf0 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b211deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0b2093e60 con 0x55e0b1d76350 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0b2083290 con 0x55e0b1d4d490 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.607+0000 7ff94e553740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0b20870a0 con 0x55e0b1d55670 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 0x55e0b211d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 0x55e0b211d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43404/0 (socket says 172.21.15.31:43404) 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 -- 172.21.15.31:0/1987976829 learned_addr learned my addr 172.21.15.31:0/1987976829 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff94557a640 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b211d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 -- 172.21.15.31:0/1987976829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 msgr2=0x55e0b211deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b211deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 -- 172.21.15.31:0/1987976829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 msgr2=0x55e0b211d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b211d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 -- 172.21.15.31:0/1987976829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b1de1930 con 0x55e0b1d76350 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff94557a640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff92c0b6680 0x7ff92c0b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff940100c50 tx=0x7ff930011810 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.608+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 0x55e0b211d430 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000c00 tx=0x7ff93c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8815197922249f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.609+0000 7ff92bfff640 1 -- 172.21.15.31:0/1987976829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c035040 con 0x55e0b1d76350 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.609+0000 7ff92bfff640 1 -- 172.21.15.31:0/1987976829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c0054f0 con 0x55e0b1d76350 2024-09-21T02:54:38.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.609+0000 7ff92bfff640 1 -- 172.21.15.31:0/1987976829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0035f0 con 0x55e0b1d76350 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 -- 172.21.15.31:0/1987976829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 msgr2=0x55e0b211d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 0x55e0b211d430 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000c00 tx=0x7ff93c0040c0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 -- 172.21.15.31:0/1987976829 shutdown_connections 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b211deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b211d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 --2- 172.21.15.31:0/1987976829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d76350 0x55e0b211d430 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 -- 172.21.15.31:0/1987976829 >> 172.21.15.31:0/1987976829 conn(0x55e0b1d6a740 msgr2=0x55e0b211b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:38.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 -- 172.21.15.31:0/1987976829 shutdown_connections 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.610+0000 7ff94e553740 1 -- 172.21.15.31:0/1987976829 wait complete. 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.611+0000 7ff94e553740 1 -- start start 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.611+0000 7ff94e553740 1 Processor -- start 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 0x55e0b21290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b2129620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d76350 0x55e0b2134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 0x55e0b21290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 0x55e0b21290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43416/0 (socket says 172.21.15.31:43416) 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff945d7b640 1 -- 172.21.15.31:0/4040694130 learned_addr learned my addr 172.21.15.31:0/4040694130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94557a640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b2129620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1ee4890 con 0x55e0b1d4d490 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0b201dd40 con 0x55e0b1d76350 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0b212bbb0 con 0x55e0b1d55670 2024-09-21T02:54:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94657c640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d76350 0x55e0b2134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.612+0000 7ff94e553740 10 client.0 _ll_register_callbacks cb 0x55e0b2081c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff94657c640 1 -- 172.21.15.31:0/4040694130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 msgr2=0x55e0b2129620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff94657c640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b2129620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff94657c640 1 -- 172.21.15.31:0/4040694130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 msgr2=0x55e0b21290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff94657c640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 0x55e0b21290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff94657c640 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b21354b0 con 0x55e0b1d76350 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff94557a640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b2129620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.613+0000 7ff945d7b640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 0x55e0b21290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff94657c640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d76350 0x55e0b2134ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9380e4ee0 tx=0x7ff9380ad470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5886857e6f9f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff9297fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff9297fa640 1 -- 172.21.15.31:0/4040694130 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380aca40 con 0x55e0b1d76350 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff9297fa640 1 -- 172.21.15.31:0/4040694130 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff9380a8f30 con 0x55e0b1d76350 2024-09-21T02:54:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c0054f0 con 0x55e0b1d76350 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff9297fa640 1 -- 172.21.15.31:0/4040694130 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9380adce0 con 0x55e0b1d76350 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff94e553740 10 client.25511 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0b2137dc0 con 0x55e0b1d76350 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.614+0000 7ff9297fa640 1 -- 172.21.15.31:0/4040694130 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9380b20c0 con 0x55e0b1d76350 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff9297fa640 1 -- 172.21.15.31:0/4040694130 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff9380a7ce0 con 0x55e0b1d76350 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff94e553740 10 client.25511 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff94e553740 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff94e553740 20 client.25511 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff94e553740 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d34cd0 0x55e0b213b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff94e553740 4 client.25511 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff93c0054f0 con 0x55e0b1d34cd0 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.615+0000 7ff945d7b640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d34cd0 0x55e0b213b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.616+0000 7ff9297fa640 10 client.25511 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:38.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.616+0000 7ff945d7b640 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d34cd0 0x55e0b213b170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce6d56679f09635 server_cookie=9a584a2f4b7e2a18 in_seq=0 out_seq=0 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.616+0000 7ff9297fa640 1 -- 172.21.15.31:0/4040694130 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff93c0054f0 con 0x55e0b1d34cd0 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff9297fa640 10 client.25511 handle_command_reply: tid=0 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 client.25511 shutdown 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d76350 msgr2=0x55e0b2134ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d76350 0x55e0b2134ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9380e4ee0 tx=0x7ff9380ad470 comp rx=0 tx=0).stop 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 shutdown_connections 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d34cd0 0x55e0b213b170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b2129620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff9297fa640 0 client.25511 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d76350 0x55e0b2134ea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 --2- 172.21.15.31:0/4040694130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d4d490 0x55e0b21290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 >> 172.21.15.31:0/4040694130 conn(0x55e0b1d6a740 msgr2=0x55e0b2120610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 shutdown_connections 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 1 -- 172.21.15.31:0/4040694130 wait complete. 2024-09-21T02:54:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.617+0000 7ff94e553740 20 client.25511 trim_cache size 0 max 16384 2024-09-21T02:54:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.655+0000 7ff94e553740 1 Processor -- start 2024-09-21T02:54:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.655+0000 7ff94e553740 1 -- start start 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.655+0000 7ff94e553740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b2129500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.655+0000 7ff94e553740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d4d490 0x55e0b212aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94e553740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d55670 0x55e0b2122510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94e553740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b17f0 con 0x55e0b1d34cd0 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94e553740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b3330 con 0x55e0b1d55670 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94e553740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b1ec0 con 0x55e0b1d4d490 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b2129500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b2129500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43424/0 (socket says 172.21.15.31:43424) 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 -- 172.21.15.31:0/3176168770 learned_addr learned my addr 172.21.15.31:0/3176168770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94557a640 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d4d490 0x55e0b212aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94657c640 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d55670 0x55e0b2122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 -- 172.21.15.31:0/3176168770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d4d490 msgr2=0x55e0b212aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d4d490 0x55e0b212aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 -- 172.21.15.31:0/3176168770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d55670 msgr2=0x55e0b2122510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d55670 0x55e0b2122510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff945d7b640 1 -- 172.21.15.31:0/3176168770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b211fc80 con 0x55e0b1d34cd0 2024-09-21T02:54:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.656+0000 7ff94557a640 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d4d490 0x55e0b212aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.657+0000 7ff945d7b640 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b2129500 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02aa00 tx=0x7ff93c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b58744deb1648b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.657+0000 7ff92b7fe640 1 -- 172.21.15.31:0/3176168770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c035040 con 0x55e0b1d34cd0 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.657+0000 7ff92b7fe640 1 -- 172.21.15.31:0/3176168770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c004a30 con 0x55e0b1d34cd0 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.657+0000 7ff92b7fe640 1 -- 172.21.15.31:0/3176168770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003a50 con 0x55e0b1d34cd0 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 -- 172.21.15.31:0/3176168770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 msgr2=0x55e0b2129500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b2129500 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02aa00 tx=0x7ff93c005a20 comp rx=0 tx=0).stop 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 -- 172.21.15.31:0/3176168770 shutdown_connections 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d4d490 0x55e0b212aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d55670 0x55e0b2122510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 --2- 172.21.15.31:0/3176168770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b2129500 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 -- 172.21.15.31:0/3176168770 >> 172.21.15.31:0/3176168770 conn(0x55e0b1d6a740 msgr2=0x55e0b212a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 -- 172.21.15.31:0/3176168770 shutdown_connections 2024-09-21T02:54:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.658+0000 7ff94e553740 1 -- 172.21.15.31:0/3176168770 wait complete. 2024-09-21T02:54:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.659+0000 7ff94e553740 1 -- start start 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.659+0000 7ff94e553740 1 Processor -- start 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b212cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b212d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b212d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b2c60 con 0x55e0b1d34cd0 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401b2590 con 0x55e0b1d4d490 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401af7d0 con 0x55e0b1d55670 2024-09-21T02:54:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b212cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff945d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b212cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43426/0 (socket says 172.21.15.31:43426) 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94657c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b212d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94557a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b212d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff945d7b640 1 -- 172.21.15.31:0/1004104423 learned_addr learned my addr 172.21.15.31:0/1004104423 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.660+0000 7ff94e553740 10 client.0 _ll_register_callbacks cb 0x7ff938004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff945d7b640 1 -- 172.21.15.31:0/1004104423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 msgr2=0x55e0b212d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b212d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff945d7b640 1 -- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 msgr2=0x55e0b212d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b212d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff945d7b640 1 -- 172.21.15.31:0/1004104423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b21a4af0 con 0x55e0b1d34cd0 2024-09-21T02:54:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff94557a640 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b212d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:38.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff945d7b640 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b212cda0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff93c003710 tx=0x7ff93c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acecc67d2610daf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff9297fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff9297fa640 1 -- 172.21.15.31:0/1004104423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c035040 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff9297fa640 1 -- 172.21.15.31:0/1004104423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff93c003be0 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.661+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c004a30 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff9297fa640 1 -- 172.21.15.31:0/1004104423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff93c003de0 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 10 client.15888 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0b21a5d40 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff9297fa640 1 -- 172.21.15.31:0/1004104423 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c064070 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff9297fa640 1 -- 172.21.15.31:0/1004104423 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff93c031700 con 0x55e0b1d34cd0 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 10 client.15888 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 20 client.15888 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:38.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d76350 0x55e0b21a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 4 client.15888 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.662+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff93c004a30 con 0x55e0b1d76350 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.663+0000 7ff94557a640 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d76350 0x55e0b21a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.663+0000 7ff9297fa640 10 client.15888 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.663+0000 7ff94557a640 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d76350 0x55e0b21a8080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f88c3d139be4810 server_cookie=45c5db85ea8854e7 in_seq=0 out_seq=0 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.665+0000 7ff9297fa640 1 -- 172.21.15.31:0/1004104423 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff93c004a30 con 0x55e0b1d76350 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.665+0000 7ff9297fa640 10 client.15888 handle_command_reply: tid=0 2024-09-21T02:54:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.665+0000 7ff94e553740 1 client.15888 shutdown 2024-09-21T02:54:38.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 msgr2=0x55e0b212cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b212cda0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff93c003710 tx=0x7ff93c005fd0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 shutdown_connections 2024-09-21T02:54:38.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e0b1d76350 0x55e0b21a8080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0b1d55670 0x55e0b212d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0b1d4d490 0x55e0b212d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff9297fa640 0 client.15888 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 --2- 172.21.15.31:0/1004104423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0b1d34cd0 0x55e0b212cda0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 >> 172.21.15.31:0/1004104423 conn(0x55e0b1d6a740 msgr2=0x55e0b212a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 shutdown_connections 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 1 -- 172.21.15.31:0/1004104423 wait complete. 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.666+0000 7ff94e553740 20 client.15888 trim_cache size 0 max 16384 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:54:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:54:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "682624d1-600d-40b9-ad5a-e348fe3e999b", 2024-09-21T02:54:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:54:38.670 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:54:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff92c0b6680 msgr2=0x7ff92c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff92c0b6680 0x7ff92c0b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff940100c50 tx=0x7ff930011810 comp rx=0 tx=0).stop 2024-09-21T02:54:38.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 msgr2=0x7ff940199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 0x7ff940199b70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff93800efd0 tx=0x7ff93800c6a0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 shutdown_connections 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff92c0b6680 0x7ff92c0b8b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9401083a0 0x7ff940199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff940108cf0 0x7ff940199b70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 --2- 172.21.15.31:0/1146658330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff940102a60 0x7ff9401990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 >> 172.21.15.31:0/1146658330 conn(0x7ff940079730 msgr2=0x7ff940101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 shutdown_connections 2024-09-21T02:54:38.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:38.667+0000 7ff947fdf640 1 -- 172.21.15.31:0/1146658330 wait complete. 2024-09-21T02:54:38.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:38 smithi031 ceph-mon[31209]: pgmap v377: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-09-21T02:54:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:38 smithi165 ceph-mon[34818]: pgmap v377: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-09-21T02:54:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:38 smithi178 ceph-mon[34759]: pgmap v377: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-09-21T02:54:39.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:54:39.636 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:39.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:39 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:54:39.913 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:39 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:39.913 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:39 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:54:39.913 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:39 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:39.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:39 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:54:39.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:39 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:39.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:39 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:54:39.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:39 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:39.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:39 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:54:39.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:39 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:39.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:39 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:54:39.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:39 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 -- 172.21.15.31:0/4024428427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c102a80 msgr2=0x7f346c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 --2- 172.21.15.31:0/4024428427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c102a80 0x7f346c102e90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3454009900 tx=0x7f345402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 -- 172.21.15.31:0/4024428427 shutdown_connections 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 --2- 172.21.15.31:0/4024428427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c108d10 0x7f346c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 --2- 172.21.15.31:0/4024428427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c1083c0 0x7f346c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 --2- 172.21.15.31:0/4024428427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c102a80 0x7f346c102e90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 -- 172.21.15.31:0/4024428427 >> 172.21.15.31:0/4024428427 conn(0x7f346c079740 msgr2=0x7f346c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 -- 172.21.15.31:0/4024428427 shutdown_connections 2024-09-21T02:54:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.176+0000 7f3472b9c640 1 -- 172.21.15.31:0/4024428427 wait complete. 2024-09-21T02:54:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.177+0000 7f3472b9c640 1 Processor -- start 2024-09-21T02:54:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 -- start start 2024-09-21T02:54:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c102a80 0x7f346c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c1083c0 0x7f346c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 0x7f346c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f346c112980 con 0x7f346c108d10 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f346c112800 con 0x7f346c102a80 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3472b9c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f346c112b00 con 0x7f346c1083c0 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3471139640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 0x7f346c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3471139640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 0x7f346c1a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34536/0 (socket says 172.21.15.31:34536) 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3471139640 1 -- 172.21.15.31:0/1981112107 learned_addr learned my addr 172.21.15.31:0/1981112107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.178+0000 7f3470938640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c102a80 0x7f346c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f346bfff640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c1083c0 0x7f346c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3471139640 1 -- 172.21.15.31:0/1981112107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c1083c0 msgr2=0x7f346c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3471139640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c1083c0 0x7f346c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3471139640 1 -- 172.21.15.31:0/1981112107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c102a80 msgr2=0x7f346c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3471139640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c102a80 0x7f346c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3471139640 1 -- 172.21.15.31:0/1981112107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c1a2660 con 0x7f346c108d10 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3470938640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c102a80 0x7f346c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3471139640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 0x7f346c1a1f30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f346400d950 tx=0x7f346400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3469ffb640 1 -- 172.21.15.31:0/1981112107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464017030 con 0x7f346c108d10 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a2950 con 0x7f346c108d10 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.179+0000 7f3469ffb640 1 -- 172.21.15.31:0/1981112107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3464004020 con 0x7f346c108d10 2024-09-21T02:54:40.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.180+0000 7f3469ffb640 1 -- 172.21.15.31:0/1981112107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464003b50 con 0x7f346c108d10 2024-09-21T02:54:40.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.180+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a90b0 con 0x7f346c108d10 2024-09-21T02:54:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.182+0000 7f3469ffb640 1 -- 172.21.15.31:0/1981112107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f346401a070 con 0x7f346c108d10 2024-09-21T02:54:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 Processor -- start 2024-09-21T02:54:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 -- start start 2024-09-21T02:54:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 0x55cc9d921650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d921b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d559390 0x55cc9d9220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cc9d897e50 con 0x55cc9d57a070 2024-09-21T02:54:40.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cc9d887280 con 0x55cc9d559390 2024-09-21T02:54:40.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3479110740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc9d88b090 con 0x55cc9d5511b0 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f3469ffb640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34580b6680 0x7f34580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f346bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 0x55cc9d921650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f346bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 0x55cc9d921650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34542/0 (socket says 172.21.15.31:34542) 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.183+0000 7f346bfff640 1 -- 172.21.15.31:0/2492074741 learned_addr learned my addr 172.21.15.31:0/2492074741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3470938640 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d921b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3471139640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34580b6680 0x7f34580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d559390 0x55cc9d9220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3469ffb640 1 -- 172.21.15.31:0/1981112107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f34640e6de0 con 0x7f346c108d10 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 -- 172.21.15.31:0/2492074741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 msgr2=0x55cc9d921b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d921b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 -- 172.21.15.31:0/2492074741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d559390 msgr2=0x55cc9d9220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d559390 0x55cc9d9220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 -- 172.21.15.31:0/2492074741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9d5e5650 con 0x55cc9d57a070 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f346bfff640 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 0x55cc9d921650 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f345c003ef0 tx=0x7f345c009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc7394a4de9784c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3471139640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34580b6680 0x7f34580b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f34640e4600 tx=0x7f3464079900 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3462ffd640 1 -- 172.21.15.31:0/2492074741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0124b0 con 0x55cc9d57a070 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3462ffd640 1 -- 172.21.15.31:0/2492074741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c012b00 con 0x55cc9d57a070 2024-09-21T02:54:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.184+0000 7f3462ffd640 1 -- 172.21.15.31:0/2492074741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c010a70 con 0x55cc9d57a070 2024-09-21T02:54:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 -- 172.21.15.31:0/2492074741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 msgr2=0x55cc9d921650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 0x55cc9d921650 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f345c003ef0 tx=0x7f345c009db0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 -- 172.21.15.31:0/2492074741 shutdown_connections 2024-09-21T02:54:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d921b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d559390 0x55cc9d9220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 --2- 172.21.15.31:0/2492074741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d57a070 0x55cc9d921650 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 -- 172.21.15.31:0/2492074741 >> 172.21.15.31:0/2492074741 conn(0x55cc9d56e460 msgr2=0x55cc9d91f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.185+0000 7f3479110740 1 -- 172.21.15.31:0/2492074741 shutdown_connections 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.186+0000 7f3479110740 1 -- 172.21.15.31:0/2492074741 wait complete. 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.186+0000 7f3479110740 1 -- start start 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.186+0000 7f3479110740 1 Processor -- start 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d92ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d92d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d57a070 0x55cc9d9390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cc9d6e8700 con 0x55cc9d559390 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cc9d822110 con 0x55cc9d57a070 2024-09-21T02:54:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc9d92fdd0 con 0x55cc9d5511b0 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3479110740 10 client.0 _ll_register_callbacks cb 0x55cc9d83b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d92d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d92d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34558/0 (socket says 172.21.15.31:34558) 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 -- 172.21.15.31:0/586969890 learned_addr learned my addr 172.21.15.31:0/586969890 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f346bfff640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d92ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f346bfff640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d57a070 0x55cc9d9390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 -- 172.21.15.31:0/586969890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 msgr2=0x55cc9d92ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d92ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 -- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d57a070 msgr2=0x55cc9d9390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d57a070 0x55cc9d9390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f3470938640 1 -- 172.21.15.31:0/586969890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9d84fc40 con 0x55cc9d559390 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.187+0000 7f346bfff640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d92ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f3470938640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d92d200 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3454009900 tx=0x7f3454031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572bf0b92ee3e223 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f344bfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f344bfff640 1 -- 172.21.15.31:0/586969890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454035040 con 0x55cc9d559390 2024-09-21T02:54:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c012b00 con 0x55cc9d559390 2024-09-21T02:54:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f344bfff640 1 -- 172.21.15.31:0/586969890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3454004240 con 0x55cc9d559390 2024-09-21T02:54:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f344bfff640 1 -- 172.21.15.31:0/586969890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3454004440 con 0x55cc9d559390 2024-09-21T02:54:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f3479110740 10 client.15900 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc9d93baa0 con 0x55cc9d559390 2024-09-21T02:54:40.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.188+0000 7f344bfff640 1 -- 172.21.15.31:0/586969890 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c012b00 con 0x55cc9d559390 2024-09-21T02:54:40.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f344bfff640 1 -- 172.21.15.31:0/586969890 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f345404d020 con 0x55cc9d559390 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f3479110740 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f3479110740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f3479110740 20 client.15900 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f3479110740 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d5389f0 0x55cc9d93efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f3479110740 4 client.15900 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc9d899da0 con 0x55cc9d5389f0 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f346bfff640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d5389f0 0x55cc9d93efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f346bfff640 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d5389f0 0x55cc9d93efb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22a35d0d9c0e97b server_cookie=7726fa02e3359055 in_seq=0 out_seq=0 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.189+0000 7f344bfff640 10 client.15900 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f344bfff640 1 -- 172.21.15.31:0/586969890 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc9d899da0 con 0x55cc9d5389f0 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f344bfff640 10 client.15900 handle_command_reply: tid=0 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f3479110740 1 client.15900 shutdown 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 msgr2=0x55cc9d92d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f3479110740 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d92d200 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3454009900 tx=0x7f3454031950 comp rx=0 tx=0).stop 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 shutdown_connections 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.190+0000 7f3479110740 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d5389f0 0x55cc9d93efb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d57a070 0x55cc9d9390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d92d200 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f344bfff640 0 client.15900 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 1 --2- 172.21.15.31:0/586969890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5511b0 0x55cc9d92ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 >> 172.21.15.31:0/586969890 conn(0x55cc9d56e460 msgr2=0x55cc9d924830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 shutdown_connections 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 1 -- 172.21.15.31:0/586969890 wait complete. 2024-09-21T02:54:40.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.191+0000 7f3479110740 20 client.15900 trim_cache size 0 max 16384 2024-09-21T02:54:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 Processor -- start 2024-09-21T02:54:40.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 -- start start 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5389f0 0x55cc9d922d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5511b0 0x55cc9d938fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d929eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1b18d0 con 0x55cc9d559390 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1b3410 con 0x55cc9d5511b0 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3479110740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1b1fa0 con 0x55cc9d5389f0 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f346bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d929eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f346bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d929eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34568/0 (socket says 172.21.15.31:34568) 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f346bfff640 1 -- 172.21.15.31:0/587892348 learned_addr learned my addr 172.21.15.31:0/587892348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f346bfff640 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5389f0 0x55cc9d922d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.209+0000 7f3470938640 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5511b0 0x55cc9d938fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f346bfff640 1 -- 172.21.15.31:0/587892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5389f0 msgr2=0x55cc9d922d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f346bfff640 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5389f0 0x55cc9d922d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f346bfff640 1 -- 172.21.15.31:0/587892348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5511b0 msgr2=0x55cc9d938fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f346bfff640 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5511b0 0x55cc9d938fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f346bfff640 1 -- 172.21.15.31:0/587892348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9d922460 con 0x55cc9d559390 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f346bfff640 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d929eb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f345c003c50 tx=0x7f345c022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8004497532d669ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f34627fc640 1 -- 172.21.15.31:0/587892348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0049d0 con 0x55cc9d559390 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f34627fc640 1 -- 172.21.15.31:0/587892348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c012960 con 0x55cc9d559390 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.210+0000 7f34627fc640 1 -- 172.21.15.31:0/587892348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c010870 con 0x55cc9d559390 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 -- 172.21.15.31:0/587892348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 msgr2=0x55cc9d929eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d929eb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f345c003c50 tx=0x7f345c022040 comp rx=0 tx=0).stop 2024-09-21T02:54:40.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 -- 172.21.15.31:0/587892348 shutdown_connections 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d559390 0x55cc9d929eb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5511b0 0x55cc9d938fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 --2- 172.21.15.31:0/587892348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d5389f0 0x55cc9d922d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 -- 172.21.15.31:0/587892348 >> 172.21.15.31:0/587892348 conn(0x55cc9d56e460 msgr2=0x55cc9d922680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 -- 172.21.15.31:0/587892348 shutdown_connections 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.211+0000 7f3479110740 1 -- 172.21.15.31:0/587892348 wait complete. 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 -- start start 2024-09-21T02:54:40.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 Processor -- start 2024-09-21T02:54:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5389f0 0x55cc9d935a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 0x55cc9d930f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d559390 0x55cc9d931660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1b2d40 con 0x55cc9d5511b0 2024-09-21T02:54:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1b2670 con 0x55cc9d5389f0 2024-09-21T02:54:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.212+0000 7f3479110740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1af8b0 con 0x55cc9d559390 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3479110740 10 client.0 _ll_register_callbacks cb 0x7f3454009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 0x55cc9d930f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f346bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5389f0 0x55cc9d935a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f346bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5389f0 0x55cc9d935a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:34586/0 (socket says 172.21.15.31:34586) 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 0x55cc9d930f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34580/0 (socket says 172.21.15.31:34580) 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f346bfff640 1 -- 172.21.15.31:0/1462376847 learned_addr learned my addr 172.21.15.31:0/1462376847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f346bfff640 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d559390 0x55cc9d931660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 -- 172.21.15.31:0/1462376847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d559390 msgr2=0x55cc9d931660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d559390 0x55cc9d931660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 -- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5389f0 msgr2=0x55cc9d935a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5389f0 0x55cc9d935a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 -- 172.21.15.31:0/1462376847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9d9a8970 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f3470938640 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 0x55cc9d930f50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3454004e50 tx=0x7f3454000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1178f081f2644a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f344bfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.213+0000 7f344bfff640 1 -- 172.21.15.31:0/1462376847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454035070 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c012960 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f344bfff640 1 -- 172.21.15.31:0/1462376847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f34540050d0 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f344bfff640 1 -- 172.21.15.31:0/1462376847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34540052d0 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 10 client.15906 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc9d9a9b00 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f344bfff640 1 -- 172.21.15.31:0/1462376847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c012960 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f344bfff640 1 -- 172.21.15.31:0/1462376847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f345404d040 con 0x55cc9d5511b0 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 10 client.15906 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 20 client.15906 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:40.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d57a070 0x55cc9d9ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 4 client.15906 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:54:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.214+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc9d895850 con 0x55cc9d57a070 2024-09-21T02:54:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.215+0000 7f346bfff640 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d57a070 0x55cc9d9ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.215+0000 7f346bfff640 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d57a070 0x55cc9d9ac200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01e690158888eb8 server_cookie=d3fc56c7675c961a in_seq=0 out_seq=0 2024-09-21T02:54:40.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.215+0000 7f344bfff640 10 client.15906 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.215+0000 7f344bfff640 1 -- 172.21.15.31:0/1462376847 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55cc9d895850 con 0x55cc9d57a070 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.215+0000 7f344bfff640 10 client.15906 handle_command_reply: tid=0 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.215+0000 7f3479110740 1 client.15906 shutdown 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 msgr2=0x55cc9d930f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 0x55cc9d930f50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3454004e50 tx=0x7f3454000bc0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 shutdown_connections 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc9d57a070 0x55cc9d9ac200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc9d559390 0x55cc9d931660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f344bfff640 0 client.15906 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:40.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc9d5511b0 0x55cc9d930f50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 --2- 172.21.15.31:0/1462376847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc9d5389f0 0x55cc9d935a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 >> 172.21.15.31:0/1462376847 conn(0x55cc9d56e460 msgr2=0x55cc9d92e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 shutdown_connections 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 1 -- 172.21.15.31:0/1462376847 wait complete. 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.216+0000 7f3479110740 20 client.15906 trim_cache size 0 max 16384 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:54:40.222 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:40.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:54:40.224 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34580b6680 msgr2=0x7f34580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34580b6680 0x7f34580b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f34640e4600 tx=0x7f3464079900 comp rx=0 tx=0).stop 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 msgr2=0x7f346c1a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 0x7f346c1a1f30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f346400d950 tx=0x7f346400de10 comp rx=0 tx=0).stop 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 shutdown_connections 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34580b6680 0x7f34580b8b90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c1083c0 0x7f346c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f346c108d10 0x7f346c1a1f30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 --2- 172.21.15.31:0/1981112107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f346c102a80 0x7f346c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:40.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 >> 172.21.15.31:0/1981112107 conn(0x7f346c079740 msgr2=0x7f346c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:40.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 shutdown_connections 2024-09-21T02:54:40.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:40.217+0000 7f3472b9c640 1 -- 172.21.15.31:0/1981112107 wait complete. 2024-09-21T02:54:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:40 smithi165 ceph-mon[34818]: pgmap v378: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:40 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:54:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:40 smithi178 ceph-mon[34759]: pgmap v378: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:40 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:54:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:40 smithi031 ceph-mon[31209]: pgmap v378: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 862 KiB/s wr, 12 op/s 2024-09-21T02:54:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:40 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:54:42.934 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:42 smithi031 ceph-mon[31209]: pgmap v379: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-21T02:54:42.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:42 smithi165 ceph-mon[34818]: pgmap v379: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-21T02:54:42.967 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:54:42.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:54:42.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:42 smithi178 ceph-mon[34759]: pgmap v379: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-09-21T02:54:43.247 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 -- 172.21.15.31:0/2939027077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c1083a0 msgr2=0x7f404c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 --2- 172.21.15.31:0/2939027077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c1083a0 0x7f404c1087b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4034009900 tx=0x7f403402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 -- 172.21.15.31:0/2939027077 shutdown_connections 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 --2- 172.21.15.31:0/2939027077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c108cf0 0x7f404c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 --2- 172.21.15.31:0/2939027077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c1083a0 0x7f404c1087b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 --2- 172.21.15.31:0/2939027077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f404c102a60 0x7f404c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 -- 172.21.15.31:0/2939027077 >> 172.21.15.31:0/2939027077 conn(0x7f404c079730 msgr2=0x7f404c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 -- 172.21.15.31:0/2939027077 shutdown_connections 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.020+0000 7f405180a640 1 -- 172.21.15.31:0/2939027077 wait complete. 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.021+0000 7f405180a640 1 Processor -- start 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.021+0000 7f405180a640 1 -- start start 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f405180a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 0x7f404c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f405180a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c1083a0 0x7f404c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f405180a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f404c108cf0 0x7f404c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f405180a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f404c112960 con 0x7f404c102a60 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f405180a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1127e0 con 0x7f404c1083a0 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f405180a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f404c112ae0 con 0x7f404c108cf0 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 0x7f404c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 0x7f404c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34598/0 (socket says 172.21.15.31:34598) 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 -- 172.21.15.31:0/2983387083 learned_addr learned my addr 172.21.15.31:0/2983387083 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404a7fc640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c1083a0 0x7f404c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404b7fe640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f404c108cf0 0x7f404c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 -- 172.21.15.31:0/2983387083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f404c108cf0 msgr2=0x7f404c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f404c108cf0 0x7f404c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 -- 172.21.15.31:0/2983387083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c1083a0 msgr2=0x7f404c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c1083a0 0x7f404c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404affd640 1 -- 172.21.15.31:0/2983387083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c1a0510 con 0x7f404c102a60 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.022+0000 7f404a7fc640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c1083a0 0x7f404c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.023+0000 7f404affd640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 0x7f404c19a5a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f404400eb90 tx=0x7f404400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.023+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044004d20 con 0x7f404c102a60 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.023+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1a0800 con 0x7f404c102a60 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.023+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f404400ee90 con 0x7f404c102a60 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.023+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044010670 con 0x7f404c102a60 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.023+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a6f60 con 0x7f404c102a60 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.024+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c103ce0 con 0x7f404c102a60 2024-09-21T02:54:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.026+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4044010910 con 0x7f404c102a60 2024-09-21T02:54:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.027+0000 7f4050808640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f40380b6680 0x7f40380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.027+0000 7f404a7fc640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f40380b6680 0x7f40380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.027+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f40440e8040 con 0x7f404c102a60 2024-09-21T02:54:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.027+0000 7f404a7fc640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f40380b6680 0x7f40380b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4034000c00 tx=0x7f4034031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.035+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4044056710 con 0x7f404c102a60 2024-09-21T02:54:44.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.344+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f404c102e70 con 0x7f404c102a60 2024-09-21T02:54:44.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.345+0000 7f4050808640 1 -- 172.21.15.31:0/2983387083 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f40440ae550 con 0x7f404c102a60 2024-09-21T02:54:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:54:44.347 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:54:44.347 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:54:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f40380b6680 msgr2=0x7f40380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f40380b6680 0x7f40380b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4034000c00 tx=0x7f4034031400 comp rx=0 tx=0).stop 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 msgr2=0x7f404c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 0x7f404c19a5a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f404400eb90 tx=0x7f404400c6a0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 shutdown_connections 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f40380b6680 0x7f40380b8b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f404c108cf0 0x7f404c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f404c1083a0 0x7f404c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 --2- 172.21.15.31:0/2983387083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f404c102a60 0x7f404c19a5a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 >> 172.21.15.31:0/2983387083 conn(0x7f404c079730 msgr2=0x7f404c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 shutdown_connections 2024-09-21T02:54:44.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:44.349+0000 7f405180a640 1 -- 172.21.15.31:0/2983387083 wait complete. 2024-09-21T02:54:44.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:44 smithi165 ceph-mon[34818]: pgmap v380: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 868 KiB/s wr, 14 op/s 2024-09-21T02:54:44.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:44 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2983387083' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:44.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:44 smithi178 ceph-mon[34759]: pgmap v380: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 868 KiB/s wr, 14 op/s 2024-09-21T02:54:44.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:44 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2983387083' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:44.994 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:44 smithi031 ceph-mon[31209]: pgmap v380: 97 pgs: 97 active+clean; 2.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 868 KiB/s wr, 14 op/s 2024-09-21T02:54:44.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:44 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2983387083' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:45.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:54:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:45.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.848+0000 7ff576b30640 1 -- 172.21.15.31:0/2709738794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 msgr2=0x7ff570104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.848+0000 7ff576b30640 1 --2- 172.21.15.31:0/2709738794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff570104c50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff560009900 tx=0x7ff56002a410 comp rx=0 tx=0).stop 2024-09-21T02:54:45.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 -- 172.21.15.31:0/2709738794 shutdown_connections 2024-09-21T02:54:45.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 --2- 172.21.15.31:0/2709738794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff57010ddf0 0x7ff570110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 --2- 172.21.15.31:0/2709738794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff570105270 0x7ff57010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 --2- 172.21.15.31:0/2709738794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff570104c50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 -- 172.21.15.31:0/2709738794 >> 172.21.15.31:0/2709738794 conn(0x7ff570079730 msgr2=0x7ff570079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:45.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 -- 172.21.15.31:0/2709738794 shutdown_connections 2024-09-21T02:54:45.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.849+0000 7ff576b30640 1 -- 172.21.15.31:0/2709738794 wait complete. 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 Processor -- start 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 -- start start 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff57019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570105270 0x7ff57019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff57010ddf0 0x7ff5701a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff570113a90 con 0x7ff570104840 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff570113910 con 0x7ff57010ddf0 2024-09-21T02:54:45.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.850+0000 7ff576b30640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff570113c10 con 0x7ff570105270 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff57019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff57019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34628/0 (socket says 172.21.15.31:34628) 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 -- 172.21.15.31:0/2562146058 learned_addr learned my addr 172.21.15.31:0/2562146058 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff56ffff640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570105270 0x7ff57019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5750cd640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff57010ddf0 0x7ff5701a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 -- 172.21.15.31:0/2562146058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570105270 msgr2=0x7ff57019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570105270 0x7ff57019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 -- 172.21.15.31:0/2562146058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff57010ddf0 msgr2=0x7ff5701a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff57010ddf0 0x7ff5701a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 -- 172.21.15.31:0/2562146058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5701a2610 con 0x7ff570104840 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff5748cc640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff57019c6a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff560032fd0 tx=0x7ff560004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.851+0000 7ff56dffb640 1 -- 172.21.15.31:0/2562146058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56003f030 con 0x7ff570104840 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.852+0000 7ff56dffb640 1 -- 172.21.15.31:0/2562146058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff560004490 con 0x7ff570104840 2024-09-21T02:54:45.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.852+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5701a28a0 con 0x7ff570104840 2024-09-21T02:54:45.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.852+0000 7ff56dffb640 1 -- 172.21.15.31:0/2562146058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560035070 con 0x7ff570104840 2024-09-21T02:54:45.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.852+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5701a9060 con 0x7ff570104840 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.854+0000 7ff57d0a4740 1 Processor -- start 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.854+0000 7ff57d0a4740 1 -- start start 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff56dffb640 1 -- 172.21.15.31:0/2562146058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff560042030 con 0x7ff570104840 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 0x55b709545be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b709546120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b7094bbea0 con 0x55b7091751b0 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b7094ab2d0 con 0x55b70919e070 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7094af0e0 con 0x55b70917d390 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 0x55b709545be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 0x55b709545be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34644/0 (socket says 172.21.15.31:34644) 2024-09-21T02:54:45.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 -- 172.21.15.31:0/495689305 learned_addr learned my addr 172.21.15.31:0/495689305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5750cd640 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff56ffff640 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b709546120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 -- 172.21.15.31:0/495689305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 msgr2=0x55b709546120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b709546120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 -- 172.21.15.31:0/495689305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 msgr2=0x55b7095456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.855+0000 7ff5748cc640 1 -- 172.21.15.31:0/495689305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b709209650 con 0x55b7091751b0 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff5748cc640 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 0x55b709545be0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff560105300 tx=0x7ff56010bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754e3d2af1ec7000 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff556ffd640 1 -- 172.21.15.31:0/495689305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560110b00 con 0x55b7091751b0 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff556ffd640 1 -- 172.21.15.31:0/495689305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff560115070 con 0x55b7091751b0 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff556ffd640 1 -- 172.21.15.31:0/495689305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56010e610 con 0x55b7091751b0 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff56dffb640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5580b6680 0x7ff5580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff5750cd640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5580b6680 0x7ff5580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.856+0000 7ff56dffb640 1 -- 172.21.15.31:0/2562146058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff560107c20 con 0x7ff570104840 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 -- 172.21.15.31:0/495689305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 msgr2=0x55b709545be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 0x55b709545be0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff560105300 tx=0x7ff56010bef0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff5750cd640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5580b6680 0x7ff5580b8b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff568005fd0 tx=0x7ff568009740 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 -- 172.21.15.31:0/495689305 shutdown_connections 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b709546120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7091751b0 0x55b709545be0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/495689305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 -- 172.21.15.31:0/495689305 >> 172.21.15.31:0/495689305 conn(0x55b709192460 msgr2=0x55b709543590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 -- 172.21.15.31:0/495689305 shutdown_connections 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.857+0000 7ff57d0a4740 1 -- 172.21.15.31:0/495689305 wait complete. 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 -- start start 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 Processor -- start 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7091751b0 0x55b709550db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b7095512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b70955cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b70930c700 con 0x55b70919e070 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b709446160 con 0x55b7091751b0 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b709553c40 con 0x55b70917d390 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.858+0000 7ff57d0a4740 10 client.0 _ll_register_callbacks cb 0x55b70945f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5750cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b70955cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b7095512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5750cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b70955cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34648/0 (socket says 172.21.15.31:34648) 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7091751b0 0x55b709550db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5750cd640 1 -- 172.21.15.31:0/933473747 learned_addr learned my addr 172.21.15.31:0/933473747 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 -- 172.21.15.31:0/933473747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 msgr2=0x55b7095512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b7095512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 -- 172.21.15.31:0/933473747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 msgr2=0x55b70955cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b70955cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 -- 172.21.15.31:0/933473747 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b709473c90 con 0x55b7091751b0 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff5748cc640 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7091751b0 0x55b709550db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff560003fd0 tx=0x7ff56002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61d44da9eb5853f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff54ffff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.859+0000 7ff54ffff640 1 -- 172.21.15.31:0/933473747 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560049070 con 0x55b7091751b0 2024-09-21T02:54:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff54ffff640 1 -- 172.21.15.31:0/933473747 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff56003b460 con 0x55b7091751b0 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff560115070 con 0x55b7091751b0 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff54ffff640 1 -- 172.21.15.31:0/933473747 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5600ca0e0 con 0x55b7091751b0 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 10 client.25553 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b70955f910 con 0x55b7091751b0 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff54ffff640 1 -- 172.21.15.31:0/933473747 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff560115070 con 0x55b7091751b0 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff54ffff640 1 -- 172.21.15.31:0/933473747 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff56010d050 con 0x55b7091751b0 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 10 client.25553 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 10 client.25553 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 20 client.25553 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b70915c9f0 0x55b709562c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 4 client.25553 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.860+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7094bddf0 con 0x55b70915c9f0 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.861+0000 7ff56ffff640 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b70915c9f0 0x55b709562c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.861+0000 7ff56ffff640 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b70915c9f0 0x55b709562c70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6629007403172e91 server_cookie=311310ebf12c5ddb in_seq=0 out_seq=0 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.861+0000 7ff54ffff640 10 client.25553 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff54ffff640 1 -- 172.21.15.31:0/933473747 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7094bddf0 con 0x55b70915c9f0 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff54ffff640 10 client.25553 handle_command_reply: tid=0 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 client.25553 shutdown 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7091751b0 msgr2=0x55b709550db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7091751b0 0x55b709550db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff560003fd0 tx=0x7ff56002a410 comp rx=0 tx=0).stop 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 shutdown_connections 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b70915c9f0 0x55b709562c70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b70917d390 0x55b7095512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b70955cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff54ffff640 0 client.25553 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/933473747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7091751b0 0x55b709550db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 >> 172.21.15.31:0/933473747 conn(0x55b709192460 msgr2=0x55b709548830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 shutdown_connections 2024-09-21T02:54:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.862+0000 7ff57d0a4740 1 -- 172.21.15.31:0/933473747 wait complete. 2024-09-21T02:54:45.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.863+0000 7ff57d0a4740 20 client.25553 trim_cache size 0 max 16384 2024-09-21T02:54:45.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.876+0000 7ff57d0a4740 1 Processor -- start 2024-09-21T02:54:45.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.876+0000 7ff57d0a4740 1 -- start start 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709552c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70917d390 0x55b70955ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b709546d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5701b1990 con 0x55b70919e070 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5701b34d0 con 0x55b70917d390 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5701b2060 con 0x55b7091751b0 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b709546d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b709546d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34664/0 (socket says 172.21.15.31:34664) 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 -- 172.21.15.31:0/1780066386 learned_addr learned my addr 172.21.15.31:0/1780066386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff56ffff640 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70917d390 0x55b70955ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5748cc640 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709552c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 -- 172.21.15.31:0/1780066386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 msgr2=0x55b709552c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709552c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 -- 172.21.15.31:0/1780066386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70917d390 msgr2=0x55b70955ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70917d390 0x55b70955ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff5750cd640 1 -- 172.21.15.31:0/1780066386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b709552070 con 0x55b70919e070 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.877+0000 7ff56ffff640 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70917d390 0x55b70955ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.878+0000 7ff5750cd640 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b709546d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff56801e840 tx=0x7ff568015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e356f09135f84a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.878+0000 7ff5567fc640 1 -- 172.21.15.31:0/1780066386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56801c430 con 0x55b70919e070 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.878+0000 7ff5567fc640 1 -- 172.21.15.31:0/1780066386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56801ca80 con 0x55b70919e070 2024-09-21T02:54:45.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.878+0000 7ff5567fc640 1 -- 172.21.15.31:0/1780066386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56801f9e0 con 0x55b70919e070 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 -- 172.21.15.31:0/1780066386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 msgr2=0x55b709546d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b709546d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff56801e840 tx=0x7ff568015000 comp rx=0 tx=0).stop 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 -- 172.21.15.31:0/1780066386 shutdown_connections 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70919e070 0x55b709546d00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70917d390 0x55b70955ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/1780066386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709552c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 -- 172.21.15.31:0/1780066386 >> 172.21.15.31:0/1780066386 conn(0x55b709192460 msgr2=0x55b7095465f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 -- 172.21.15.31:0/1780066386 shutdown_connections 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 -- 172.21.15.31:0/1780066386 wait complete. 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 -- start start 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.879+0000 7ff57d0a4740 1 Processor -- start 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709555620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 0x55b709555b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5701b2e00 con 0x55b70917d390 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5701b2730 con 0x55b70919e070 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5701af970 con 0x55b7091751b0 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff57d0a4740 10 client.0 _ll_register_callbacks cb 0x7ff5680095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 0x55b709555b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff56ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 0x55b709555b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34672/0 (socket says 172.21.15.31:34672) 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff56ffff640 1 -- 172.21.15.31:0/3839697664 learned_addr learned my addr 172.21.15.31:0/3839697664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff5750cd640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.880+0000 7ff5748cc640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709555620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff56ffff640 1 -- 172.21.15.31:0/3839697664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 msgr2=0x55b709555620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff56ffff640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709555620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff56ffff640 1 -- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 msgr2=0x55b7095cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff56ffff640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff56ffff640 1 -- 172.21.15.31:0/3839697664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7095cca00 con 0x55b70917d390 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff5748cc640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709555620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff56ffff640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 0x55b709555b60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55b7095631e0 tx=0x7ff56400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116552c7e0a6994f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff54ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff54ffff640 1 -- 172.21.15.31:0/3839697664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564005220 con 0x55b70917d390 2024-09-21T02:54:45.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff54ffff640 1 -- 172.21.15.31:0/3839697664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff5640053c0 con 0x55b70917d390 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56801ca80 con 0x55b70917d390 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff54ffff640 1 -- 172.21.15.31:0/3839697664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5640055c0 con 0x55b70917d390 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff57d0a4740 10 client.15930 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.881+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7095cdc80 con 0x55b70917d390 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff54ffff640 1 -- 172.21.15.31:0/3839697664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56801ca80 con 0x55b70917d390 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff54ffff640 1 -- 172.21.15.31:0/3839697664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff56402b030 con 0x55b70917d390 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff57d0a4740 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff57d0a4740 10 client.15930 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff57d0a4740 20 client.15930 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b709131da0 0x55b7095d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff57d0a4740 4 client.15930 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:45.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7094b98a0 con 0x55b709131da0 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.882+0000 7ff5748cc640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b709131da0 0x55b7095d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff54ffff640 10 client.15930 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff5748cc640 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b709131da0 0x55b7095d0490 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46a82418771447a8 server_cookie=7084d1dc7dcabe4 in_seq=0 out_seq=0 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff54ffff640 1 -- 172.21.15.31:0/3839697664 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7094b98a0 con 0x55b709131da0 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff54ffff640 10 client.15930 handle_command_reply: tid=0 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 client.15930 shutdown 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 msgr2=0x55b709555b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 0x55b709555b60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55b7095631e0 tx=0x7ff56400bf90 comp rx=0 tx=0).stop 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 shutdown_connections 2024-09-21T02:54:45.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b70919e070 0x55b7095cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b70917d390 0x55b709555b60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55b709131da0 0x55b7095d0490 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 --2- 172.21.15.31:0/3839697664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7091751b0 0x55b709555620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff54ffff640 0 client.15930 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.883+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 >> 172.21.15.31:0/3839697664 conn(0x55b709192460 msgr2=0x55b709546fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.884+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 shutdown_connections 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.884+0000 7ff57d0a4740 1 -- 172.21.15.31:0/3839697664 wait complete. 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.884+0000 7ff57d0a4740 20 client.15930 trim_cache size 0 max 16384 2024-09-21T02:54:45.889 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5580b6680 msgr2=0x7ff5580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5580b6680 0x7ff5580b8b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff568005fd0 tx=0x7ff568009740 comp rx=0 tx=0).stop 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 msgr2=0x7ff57019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff57019c6a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff560032fd0 tx=0x7ff560004270 comp rx=0 tx=0).stop 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 shutdown_connections 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5580b6680 0x7ff5580b8b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff570105270 0x7ff57019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff57010ddf0 0x7ff5701a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 --2- 172.21.15.31:0/2562146058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff570104840 0x7ff57019c6a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:45.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 >> 172.21.15.31:0/2562146058 conn(0x7ff570079730 msgr2=0x7ff57010cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:45.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 shutdown_connections 2024-09-21T02:54:45.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:45.885+0000 7ff576b30640 1 -- 172.21.15.31:0/2562146058 wait complete. 2024-09-21T02:54:46.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:54:46.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:46 smithi031 ceph-mon[31209]: pgmap v381: 97 pgs: 97 active+clean; 59 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 14 op/s 2024-09-21T02:54:46.994 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:46 smithi178 ceph-mon[34759]: pgmap v381: 97 pgs: 97 active+clean; 59 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 14 op/s 2024-09-21T02:54:46.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:46 smithi165 ceph-mon[34818]: pgmap v381: 97 pgs: 97 active+clean; 59 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 14 op/s 2024-09-21T02:54:46.994 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:47.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 -- 172.21.15.31:0/450239119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 msgr2=0x7f260810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 --2- 172.21.15.31:0/450239119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260810f550 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2600009ca0 tx=0x7f260002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 -- 172.21.15.31:0/450239119 shutdown_connections 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 --2- 172.21.15.31:0/450239119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26081083d0 0x7f26081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 --2- 172.21.15.31:0/450239119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260810f550 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 --2- 172.21.15.31:0/450239119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2608102ae0 0x7f2608102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 -- 172.21.15.31:0/450239119 >> 172.21.15.31:0/450239119 conn(0x7f26080797b0 msgr2=0x7f2608079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 -- 172.21.15.31:0/450239119 shutdown_connections 2024-09-21T02:54:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.417+0000 7f260eba6640 1 -- 172.21.15.31:0/450239119 wait complete. 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 Processor -- start 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 -- start start 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2608102ae0 0x7f260819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 0x7f260819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2608113b00 con 0x7f2608108d20 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2608113980 con 0x7f26081083d0 2024-09-21T02:54:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260eba6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2608113c80 con 0x7f2608102ae0 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f2607fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 0x7f260819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260d143640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f2607fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 0x7f260819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:34690/0 (socket says 172.21.15.31:34690) 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f2607fff640 1 -- 172.21.15.31:0/4291377558 learned_addr learned my addr 172.21.15.31:0/4291377558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.418+0000 7f260c942640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2608102ae0 0x7f260819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2607fff640 1 -- 172.21.15.31:0/4291377558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2608102ae0 msgr2=0x7f260819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2607fff640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2608102ae0 0x7f260819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2607fff640 1 -- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 msgr2=0x7f260819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2607fff640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2607fff640 1 -- 172.21.15.31:0/4291377558 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081a03c0 con 0x7f26081083d0 2024-09-21T02:54:47.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f260d143640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2607fff640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 0x7f260819a9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00d750 tx=0x7f25fc00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2605ffb640 1 -- 172.21.15.31:0/4291377558 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc017070 con 0x7f26081083d0 2024-09-21T02:54:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2605ffb640 1 -- 172.21.15.31:0/4291377558 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc0054d0 con 0x7f26081083d0 2024-09-21T02:54:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26081a06b0 con 0x7f26081083d0 2024-09-21T02:54:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f2605ffb640 1 -- 172.21.15.31:0/4291377558 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc004c30 con 0x7f26081083d0 2024-09-21T02:54:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.419+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081a6e10 con 0x7f26081083d0 2024-09-21T02:54:47.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.422+0000 7f261511a740 1 Processor -- start 2024-09-21T02:54:47.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.422+0000 7f261511a740 1 -- start start 2024-09-21T02:54:47.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.422+0000 7f2605ffb640 1 -- 172.21.15.31:0/4291377558 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f25fc004dd0 con 0x7f26081083d0 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f261511a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b7f070 0x560699f266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f261511a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f26be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f261511a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f27120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f261511a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560699e9cea0 con 0x560699b561b0 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f261511a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560699e8c2d0 con 0x560699b5e390 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f261511a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560699e900e0 con 0x560699b7f070 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f26be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f26be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34710/0 (socket says 172.21.15.31:34710) 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 -- 172.21.15.31:0/1618751167 learned_addr learned my addr 172.21.15.31:0/1618751167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f2607fff640 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f27120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260d143640 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b7f070 0x560699f266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 -- 172.21.15.31:0/1618751167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b7f070 msgr2=0x560699f266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b7f070 0x560699f266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 -- 172.21.15.31:0/1618751167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 msgr2=0x560699f27120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f27120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 -- 172.21.15.31:0/1618751167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560699bea650 con 0x560699b561b0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.423+0000 7f260c942640 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f26be0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009cf0 tx=0x7f25f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6718182b218f5bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.424+0000 7f25eeffd640 1 -- 172.21.15.31:0/1618751167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004230 con 0x560699b561b0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.424+0000 7f2605ffb640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25f00b6680 0x7f25f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.424+0000 7f25eeffd640 1 -- 172.21.15.31:0/1618751167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f80043d0 con 0x560699b561b0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.424+0000 7f25eeffd640 1 -- 172.21.15.31:0/1618751167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8019880 con 0x560699b561b0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.424+0000 7f260d143640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25f00b6680 0x7f25f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.424+0000 7f2605ffb640 1 -- 172.21.15.31:0/4291377558 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25fc0e7070 con 0x7f26081083d0 2024-09-21T02:54:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f260d143640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25f00b6680 0x7f25f00b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2600006010 tx=0x7f2600032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 -- 172.21.15.31:0/1618751167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 msgr2=0x560699f26be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f26be0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009cf0 tx=0x7f25f8011810 comp rx=0 tx=0).stop 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 -- 172.21.15.31:0/1618751167 shutdown_connections 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f27120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f26be0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 --2- 172.21.15.31:0/1618751167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b7f070 0x560699f266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 -- 172.21.15.31:0/1618751167 >> 172.21.15.31:0/1618751167 conn(0x560699b73460 msgr2=0x560699f24590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 -- 172.21.15.31:0/1618751167 shutdown_connections 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 -- 172.21.15.31:0/1618751167 wait complete. 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 -- start start 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.425+0000 7f261511a740 1 Processor -- start 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f32240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 0x560699f3e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560699ced700 con 0x560699b7f070 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560699e27160 con 0x560699b5e390 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560699f34e80 con 0x560699b561b0 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f261511a740 10 client.0 _ll_register_callbacks cb 0x560699e409e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f260d143640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 0x560699f3e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f260d143640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 0x560699f3e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34720/0 (socket says 172.21.15.31:34720) 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f260d143640 1 -- 172.21.15.31:0/1729548517 learned_addr learned my addr 172.21.15.31:0/1729548517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f260c942640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.426+0000 7f2607fff640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f32240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f260d143640 1 -- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 msgr2=0x560699f31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f260d143640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f260d143640 1 -- 172.21.15.31:0/1729548517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 msgr2=0x560699f32240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f260d143640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f32240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f260d143640 1 -- 172.21.15.31:0/1729548517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560699e54c90 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f260d143640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 0x560699f3e170 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2600009a60 tx=0x7f260002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9024a27a6b8576 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f25e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f25e7fff640 1 -- 172.21.15.31:0/1729548517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600004250 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f25e7fff640 1 -- 172.21.15.31:0/1729548517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f26000043f0 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f80043d0 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f25e7fff640 1 -- 172.21.15.31:0/1729548517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26000045f0 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f261511a740 10 client.15942 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.427+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560699f40b30 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f25e7fff640 1 -- 172.21.15.31:0/1729548517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f80043d0 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f25e7fff640 1 -- 172.21.15.31:0/1729548517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2600052070 con 0x560699b7f070 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f261511a740 10 client.15942 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f261511a740 10 client.15942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f261511a740 20 client.15942 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f261511a740 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b3d9f0 0x560699f440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f261511a740 4 client.15942 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560699e9edf0 con 0x560699b3d9f0 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.428+0000 7f260c942640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b3d9f0 0x560699f440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.429+0000 7f260c942640 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b3d9f0 0x560699f440b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4ca02dc71dbb65b8 server_cookie=541d739762ab4d2c in_seq=0 out_seq=0 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.429+0000 7f25e7fff640 10 client.15942 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f25e7fff640 1 -- 172.21.15.31:0/1729548517 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560699e9edf0 con 0x560699b3d9f0 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f25e7fff640 10 client.15942 handle_command_reply: tid=0 2024-09-21T02:54:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 client.15942 shutdown 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 msgr2=0x560699f3e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 0x560699f3e170 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2600009a60 tx=0x7f260002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 shutdown_connections 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b3d9f0 0x560699f440b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b7f070 0x560699f3e170 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f25e7fff640 0 client.15942 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f32240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 --2- 172.21.15.31:0/1729548517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 >> 172.21.15.31:0/1729548517 conn(0x560699b73460 msgr2=0x560699f29830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 shutdown_connections 2024-09-21T02:54:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 1 -- 172.21.15.31:0/1729548517 wait complete. 2024-09-21T02:54:47.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.430+0000 7f261511a740 20 client.15942 trim_cache size 0 max 16384 2024-09-21T02:54:47.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 Processor -- start 2024-09-21T02:54:47.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 -- start start 2024-09-21T02:54:47.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 0x560699f27de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26081af670 con 0x560699b3d9f0 2024-09-21T02:54:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b11b0 con 0x560699b5e390 2024-09-21T02:54:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f261511a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26081afd40 con 0x560699b561b0 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f260c942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 0x560699f27de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f260c942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 0x560699f27de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34724/0 (socket says 172.21.15.31:34724) 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.444+0000 7f260c942640 1 -- 172.21.15.31:0/1849525435 learned_addr learned my addr 172.21.15.31:0/1849525435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f2607fff640 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260d143640 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260c942640 1 -- 172.21.15.31:0/1849525435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 msgr2=0x560699f23bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260c942640 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f23bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260c942640 1 -- 172.21.15.31:0/1849525435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 msgr2=0x560699f24100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260c942640 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f24100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260c942640 1 -- 172.21.15.31:0/1849525435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560699f34440 con 0x560699b3d9f0 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260d143640 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f24100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f260c942640 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 0x560699f27de0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f25f8011d10 tx=0x7f25f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975b39b9eeeb4490 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f25ee7fc640 1 -- 172.21.15.31:0/1849525435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80054a0 con 0x560699b3d9f0 2024-09-21T02:54:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f25ee7fc640 1 -- 172.21.15.31:0/1849525435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f8005640 con 0x560699b3d9f0 2024-09-21T02:54:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.445+0000 7f25ee7fc640 1 -- 172.21.15.31:0/1849525435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8019990 con 0x560699b3d9f0 2024-09-21T02:54:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 -- 172.21.15.31:0/1849525435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 msgr2=0x560699f27de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 0x560699f27de0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f25f8011d10 tx=0x7f25f8005f90 comp rx=0 tx=0).stop 2024-09-21T02:54:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 -- 172.21.15.31:0/1849525435 shutdown_connections 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b561b0 0x560699f23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b5e390 0x560699f24100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 --2- 172.21.15.31:0/1849525435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b3d9f0 0x560699f27de0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 -- 172.21.15.31:0/1849525435 >> 172.21.15.31:0/1849525435 conn(0x560699b73460 msgr2=0x560699f276d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 -- 172.21.15.31:0/1849525435 shutdown_connections 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.446+0000 7f261511a740 1 -- 172.21.15.31:0/1849525435 wait complete. 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.447+0000 7f261511a740 1 -- start start 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.447+0000 7f261511a740 1 Processor -- start 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.447+0000 7f261511a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b3d9f0 0x560699f36840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.447+0000 7f261511a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.447+0000 7f261511a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b5e390 0x560699fadab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f261511a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b0ae0 con 0x560699b561b0 2024-09-21T02:54:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f261511a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b0410 con 0x560699b3d9f0 2024-09-21T02:54:47.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f261511a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26081ad650 con 0x560699b5e390 2024-09-21T02:54:47.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f261511a740 10 client.0 _ll_register_callbacks cb 0x7f2600009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:47.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f36d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34728/0 (socket says 172.21.15.31:34728) 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 -- 172.21.15.31:0/4148609274 learned_addr learned my addr 172.21.15.31:0/4148609274 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f260c942640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b3d9f0 0x560699f36840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f260d143640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b5e390 0x560699fadab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 -- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b5e390 msgr2=0x560699fadab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b5e390 0x560699fadab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 -- 172.21.15.31:0/4148609274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b3d9f0 msgr2=0x560699f36840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b3d9f0 0x560699f36840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 -- 172.21.15.31:0/4148609274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560699fae040 con 0x560699b561b0 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f2607fff640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f36d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0aa200 tx=0x7f25fc0acd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439efb67e69bee60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.448+0000 7f25e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f25e7fff640 1 -- 172.21.15.31:0/4148609274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc020040 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f25e7fff640 1 -- 172.21.15.31:0/4148609274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f25fc0e90e0 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f8005640 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f25e7fff640 1 -- 172.21.15.31:0/4148609274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25fc0acaa0 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 10 client.15948 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560699faf320 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f25e7fff640 1 -- 172.21.15.31:0/4148609274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc0b19c0 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f25e7fff640 1 -- 172.21.15.31:0/4148609274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25fc0940b0 con 0x560699b561b0 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 10 client.15948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 20 client.15948 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b7f070 0x560699fb1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 4 client.15948 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.449+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f25f8005640 con 0x560699b7f070 2024-09-21T02:54:47.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.450+0000 7f260c942640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b7f070 0x560699fb1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:47.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.450+0000 7f25e7fff640 10 client.15948 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:47.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.450+0000 7f260c942640 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b7f070 0x560699fb1b30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24058aa73f8e3185 server_cookie=88eb697e0b0a5069 in_seq=0 out_seq=0 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.450+0000 7f25e7fff640 1 -- 172.21.15.31:0/4148609274 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f25f8005640 con 0x560699b7f070 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.450+0000 7f25e7fff640 10 client.15948 handle_command_reply: tid=0 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.450+0000 7f261511a740 1 client.15948 shutdown 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 msgr2=0x560699f36d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f36d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0aa200 tx=0x7f25fc0acd60 comp rx=0 tx=0).stop 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 shutdown_connections 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x560699b7f070 0x560699fb1b30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f25e7fff640 0 client.15948 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:54:47.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560699b5e390 0x560699fadab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560699b561b0 0x560699f36d80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 --2- 172.21.15.31:0/4148609274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560699b3d9f0 0x560699f36840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 >> 172.21.15.31:0/4148609274 conn(0x560699b73460 msgr2=0x560699f331b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 shutdown_connections 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 1 -- 172.21.15.31:0/4148609274 wait complete. 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.451+0000 7f261511a740 20 client.15948 trim_cache size 0 max 16384 2024-09-21T02:54:47.457 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25f00b6680 msgr2=0x7f25f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25f00b6680 0x7f25f00b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2600006010 tx=0x7f2600032ed0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 msgr2=0x7f260819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 0x7f260819a9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00d750 tx=0x7f25fc00dc10 comp rx=0 tx=0).stop 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 shutdown_connections 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25f00b6680 0x7f25f00b8b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2608108d20 0x7f260819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26081083d0 0x7f260819a9f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 --2- 172.21.15.31:0/4291377558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2608102ae0 0x7f260819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 >> 172.21.15.31:0/4291377558 conn(0x7f26080797b0 msgr2=0x7f2608106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 shutdown_connections 2024-09-21T02:54:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:47.452+0000 7f260eba6640 1 -- 172.21.15.31:0/4291377558 wait complete. 2024-09-21T02:54:48.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:54:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:48.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:48 smithi031 ceph-mon[31209]: pgmap v382: 97 pgs: 97 active+clean; 123 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-21T02:54:48.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:48 smithi165 ceph-mon[34818]: pgmap v382: 97 pgs: 97 active+clean; 123 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-21T02:54:48.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:48 smithi178 ceph-mon[34759]: pgmap v382: 97 pgs: 97 active+clean; 123 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-21T02:54:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 -- 172.21.15.31:0/4040194494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 msgr2=0x7f8b841098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 --2- 172.21.15.31:0/4040194494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b841098d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009900 tx=0x7f8b6c02a410 comp rx=0 tx=0).stop 2024-09-21T02:54:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 -- 172.21.15.31:0/4040194494 shutdown_connections 2024-09-21T02:54:48.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 --2- 172.21.15.31:0/4040194494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b84109e10 0x7f8b84110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 --2- 172.21.15.31:0/4040194494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b841098d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 --2- 172.21.15.31:0/4040194494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b84103b80 0x7f8b84103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 -- 172.21.15.31:0/4040194494 >> 172.21.15.31:0/4040194494 conn(0x7f8b840ff610 msgr2=0x7f8b84101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 -- 172.21.15.31:0/4040194494 shutdown_connections 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.992+0000 7f8b8a231640 1 -- 172.21.15.31:0/4040194494 wait complete. 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 Processor -- start 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 -- start start 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b84103b80 0x7f8b84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b84075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b84109e10 0x7f8b84076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84113ad0 con 0x7f8b841094c0 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84113950 con 0x7f8b84103b80 2024-09-21T02:54:48.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b8a231640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84113c50 con 0x7f8b84109e10 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.993+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b84075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b84075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34750/0 (socket says 172.21.15.31:34750) 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 -- 172.21.15.31:0/2796464744 learned_addr learned my addr 172.21.15.31:0/2796464744 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b837fe640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b84103b80 0x7f8b84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b83fff640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b84109e10 0x7f8b84076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 -- 172.21.15.31:0/2796464744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b84109e10 msgr2=0x7f8b84076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b84109e10 0x7f8b84076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 -- 172.21.15.31:0/2796464744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b84103b80 msgr2=0x7f8b84075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b84103b80 0x7f8b84075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 -- 172.21.15.31:0/2796464744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b841a87a0 con 0x7f8b841094c0 2024-09-21T02:54:48.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b84075c50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c02a930 tx=0x7f8b6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:48.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b817fa640 1 -- 172.21.15.31:0/2796464744 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c035070 con 0x7f8b841094c0 2024-09-21T02:54:48.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8419a1f0 con 0x7f8b841094c0 2024-09-21T02:54:48.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b817fa640 1 -- 172.21.15.31:0/2796464744 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b6c004490 con 0x7f8b841094c0 2024-09-21T02:54:48.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.994+0000 7f8b817fa640 1 -- 172.21.15.31:0/2796464744 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0496a0 con 0x7f8b841094c0 2024-09-21T02:54:48.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.995+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8419a720 con 0x7f8b841094c0 2024-09-21T02:54:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.997+0000 7f8b907a5740 1 Processor -- start 2024-09-21T02:54:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.997+0000 7f8b817fa640 1 -- 172.21.15.31:0/2796464744 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8b6c049840 con 0x7f8b841094c0 2024-09-21T02:54:49.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.997+0000 7f8b907a5740 1 -- start start 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.997+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fd32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49fd3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b907a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555f49f49ea0 con 0x555f49c031b0 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b907a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f49f392d0 con 0x555f49c2c070 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b907a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555f49f3d0e0 con 0x555f49c0b390 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34760/0 (socket says 172.21.15.31:34760) 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 -- 172.21.15.31:0/1131146975 learned_addr learned my addr 172.21.15.31:0/1131146975 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b83fff640 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fd32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49fd3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 -- 172.21.15.31:0/1131146975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 msgr2=0x555f49fd3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49fd3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 -- 172.21.15.31:0/1131146975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 msgr2=0x555f49fd32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fd32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 -- 172.21.15.31:0/1131146975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f49c97650 con 0x555f49c031b0 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b83fff640 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fd32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b837fe640 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd3800 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400bcb0 tx=0x7f8b740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb1e3cba6fe0094 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.998+0000 7f8b817fa640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b700b6700 0x7f8b700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.999+0000 7f8b79ffb640 1 -- 172.21.15.31:0/1131146975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7401c480 con 0x555f49c031b0 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.999+0000 7f8b83fff640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b700b6700 0x7f8b700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.999+0000 7f8b817fa640 1 -- 172.21.15.31:0/2796464744 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8b6c1104c0 con 0x7f8b841094c0 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.999+0000 7f8b79ffb640 1 -- 172.21.15.31:0/1131146975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b7401cad0 con 0x555f49c031b0 2024-09-21T02:54:49.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:48.999+0000 7f8b79ffb640 1 -- 172.21.15.31:0/1131146975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74011c30 con 0x555f49c031b0 2024-09-21T02:54:49.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 -- 172.21.15.31:0/1131146975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 msgr2=0x555f49fd3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:49.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b83fff640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b700b6700 0x7f8b700b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005fd0 tx=0x7f8b7c009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:49.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd3800 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400bcb0 tx=0x7f8b740107a0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 -- 172.21.15.31:0/1131146975 shutdown_connections 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49fd3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd3800 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1131146975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fd32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 -- 172.21.15.31:0/1131146975 >> 172.21.15.31:0/1131146975 conn(0x555f49c20460 msgr2=0x555f49fd11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 -- 172.21.15.31:0/1131146975 shutdown_connections 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 -- 172.21.15.31:0/1131146975 wait complete. 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.000+0000 7f8b907a5740 1 -- start start 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 Processor -- start 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c031b0 0x555f49fde970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fdeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c2c070 0x555f49fead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555f49d9a700 con 0x555f49c0b390 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f49ed4160 con 0x555f49c031b0 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555f49fe1aa0 con 0x555f49c2c070 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b907a5740 10 client.0 _ll_register_callbacks cb 0x555f49eed9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fdeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fdeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34768/0 (socket says 172.21.15.31:34768) 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b7bfff640 1 -- 172.21.15.31:0/3590470268 learned_addr learned my addr 172.21.15.31:0/3590470268 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b837fe640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c031b0 0x555f49fde970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.001+0000 7f8b83fff640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c2c070 0x555f49fead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b7bfff640 1 -- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c2c070 msgr2=0x555f49fead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c2c070 0x555f49fead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b7bfff640 1 -- 172.21.15.31:0/3590470268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c031b0 msgr2=0x555f49fde970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c031b0 0x555f49fde970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b7bfff640 1 -- 172.21.15.31:0/3590470268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f49f01c90 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fdeeb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c031440 tx=0x7f8b6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba372ed790ee9cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b62ffd640 1 -- 172.21.15.31:0/3590470268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0d6100 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b62ffd640 1 -- 172.21.15.31:0/3590470268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8b6c0dfac0 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b7401cad0 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b62ffd640 1 -- 172.21.15.31:0/3590470268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b6c0d5aa0 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b907a5740 10 client.15960 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f49fed750 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.002+0000 7f8b62ffd640 1 -- 172.21.15.31:0/3590470268 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b7401cad0 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b62ffd640 1 -- 172.21.15.31:0/3590470268 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8b6c110030 con 0x555f49c0b390 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b907a5740 10 client.15960 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b907a5740 10 client.15960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b907a5740 20 client.15960 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b907a5740 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bea9f0 0x555f49ff0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b907a5740 4 client.15960 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f49f4bdf0 con 0x555f49bea9f0 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b837fe640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bea9f0 0x555f49ff0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b837fe640 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bea9f0 0x555f49ff0bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c77ecf760ed4b73b server_cookie=726b7be013772c5c in_seq=0 out_seq=0 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.003+0000 7f8b62ffd640 10 client.15960 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.004+0000 7f8b62ffd640 1 -- 172.21.15.31:0/3590470268 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f49f4bdf0 con 0x555f49bea9f0 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.004+0000 7f8b62ffd640 10 client.15960 handle_command_reply: tid=0 2024-09-21T02:54:49.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.004+0000 7f8b907a5740 1 client.15960 shutdown 2024-09-21T02:54:49.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.004+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 msgr2=0x555f49fdeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:49.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.004+0000 7f8b907a5740 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fdeeb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c031440 tx=0x7f8b6c000bc0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 shutdown_connections 2024-09-21T02:54:49.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bea9f0 0x555f49ff0bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c2c070 0x555f49fead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fdeeb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b62ffd640 0 client.15960 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 --2- 172.21.15.31:0/3590470268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c031b0 0x555f49fde970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 >> 172.21.15.31:0/3590470268 conn(0x555f49c20460 msgr2=0x555f49fd64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 shutdown_connections 2024-09-21T02:54:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 1 -- 172.21.15.31:0/3590470268 wait complete. 2024-09-21T02:54:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.005+0000 7f8b907a5740 20 client.15960 trim_cache size 0 max 16384 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.018+0000 7f8b907a5740 1 Processor -- start 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.018+0000 7f8b907a5740 1 -- start start 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49feac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fdbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b907a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841ad570 con 0x555f49c031b0 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b907a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841af0b0 con 0x555f49c2c070 2024-09-21T02:54:49.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b907a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841adc40 con 0x555f49c0b390 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34770/0 (socket says 172.21.15.31:34770) 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 -- 172.21.15.31:0/895525027 learned_addr learned my addr 172.21.15.31:0/895525027 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49feac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b83fff640 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fdbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 -- 172.21.15.31:0/895525027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 msgr2=0x555f49feac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49feac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 -- 172.21.15.31:0/895525027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 msgr2=0x555f49fdbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fdbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b837fe640 1 -- 172.21.15.31:0/895525027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f49fdfc30 con 0x555f49c031b0 2024-09-21T02:54:49.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.019+0000 7f8b83fff640 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fdbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:54:49.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.020+0000 7f8b837fe640 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd4a00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8b74011760 tx=0x7f8b740116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14514267f478c08e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:49.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.020+0000 7f8b797fa640 1 -- 172.21.15.31:0/895525027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74005040 con 0x555f49c031b0 2024-09-21T02:54:49.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.020+0000 7f8b797fa640 1 -- 172.21.15.31:0/895525027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b74011800 con 0x555f49c031b0 2024-09-21T02:54:49.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.020+0000 7f8b797fa640 1 -- 172.21.15.31:0/895525027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74003870 con 0x555f49c031b0 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 -- 172.21.15.31:0/895525027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 msgr2=0x555f49fd4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd4a00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8b74011760 tx=0x7f8b740116f0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 -- 172.21.15.31:0/895525027 shutdown_connections 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c0b390 0x555f49feac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f49fdbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 --2- 172.21.15.31:0/895525027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c031b0 0x555f49fd4a00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 -- 172.21.15.31:0/895525027 >> 172.21.15.31:0/895525027 conn(0x555f49c20460 msgr2=0x555f49fd42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 -- 172.21.15.31:0/895525027 shutdown_connections 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.021+0000 7f8b907a5740 1 -- 172.21.15.31:0/895525027 wait complete. 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 -- start start 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 Processor -- start 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c031b0 0x555f49fe33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fe3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f4a05a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841ae9e0 con 0x555f49c0b390 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841ae310 con 0x555f49c2c070 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841ab550 con 0x555f49c031b0 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b907a5740 10 client.0 _ll_register_callbacks cb 0x7f8b6c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fe3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b837fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c031b0 0x555f49fe33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fe3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34774/0 (socket says 172.21.15.31:34774) 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b83fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f4a05a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.022+0000 7f8b7bfff640 1 -- 172.21.15.31:0/1631234873 learned_addr learned my addr 172.21.15.31:0/1631234873 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b837fe640 1 -- 172.21.15.31:0/1631234873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 msgr2=0x555f4a05a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b837fe640 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f4a05a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b837fe640 1 -- 172.21.15.31:0/1631234873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 msgr2=0x555f49fe3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b837fe640 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fe3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b837fe640 1 -- 172.21.15.31:0/1631234873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4a05aa80 con 0x555f49c031b0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fe3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b837fe640 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c031b0 0x555f49fe33c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b74011010 tx=0x7f8b740035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2eebbada30f7181 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b62ffd640 1 -- 172.21.15.31:0/1631234873 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b740119b0 con 0x555f49c031b0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b62ffd640 1 -- 172.21.15.31:0/1631234873 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8b74011b50 con 0x555f49c031b0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.023+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b74011800 con 0x555f49c031b0 2024-09-21T02:54:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b62ffd640 1 -- 172.21.15.31:0/1631234873 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b74011d90 con 0x555f49c031b0 2024-09-21T02:54:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 10 client.25645 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f4a05bbd0 con 0x555f49c031b0 2024-09-21T02:54:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b62ffd640 1 -- 172.21.15.31:0/1631234873 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b74011800 con 0x555f49c031b0 2024-09-21T02:54:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b62ffd640 1 -- 172.21.15.31:0/1631234873 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8b7403e020 con 0x555f49c031b0 2024-09-21T02:54:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 10 client.25645 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 10 client.25645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 20 client.25645 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bbfda0 0x555f4a05e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 4 client.25645 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f49f478a0 con 0x555f49bbfda0 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.024+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bbfda0 0x555f4a05e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b62ffd640 10 client.25645 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b7bfff640 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bbfda0 0x555f4a05e100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53e4ecf0e56d2d51 server_cookie=49b67e143145fffd in_seq=0 out_seq=0 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b62ffd640 1 -- 172.21.15.31:0/1631234873 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f49f478a0 con 0x555f49bbfda0 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b62ffd640 10 client.25645 handle_command_reply: tid=0 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 client.25645 shutdown 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c031b0 msgr2=0x555f49fe33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c031b0 0x555f49fe33c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b74011010 tx=0x7f8b740035f0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 shutdown_connections 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555f49bbfda0 0x555f4a05e100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f49c2c070 0x555f4a05a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f49c0b390 0x555f49fe3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 --2- 172.21.15.31:0/1631234873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555f49c031b0 0x555f49fe33c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b62ffd640 0 client.25645 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.025+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 >> 172.21.15.31:0/1631234873 conn(0x555f49c20460 msgr2=0x555f49fdfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 shutdown_connections 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b907a5740 1 -- 172.21.15.31:0/1631234873 wait complete. 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b907a5740 20 client.25645 trim_cache size 0 max 16384 2024-09-21T02:54:49.031 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b700b6700 msgr2=0x7f8b700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b700b6700 0x7f8b700b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c005fd0 tx=0x7f8b7c009600 comp rx=0 tx=0).stop 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 msgr2=0x7f8b84075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b84075c50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c02a930 tx=0x7f8b6c004270 comp rx=0 tx=0).stop 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 shutdown_connections 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8b700b6700 0x7f8b700b8c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b84109e10 0x7f8b84076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b841094c0 0x7f8b84075c50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 --2- 172.21.15.31:0/2796464744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b84103b80 0x7f8b84075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.026+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 >> 172.21.15.31:0/2796464744 conn(0x7f8b840ff610 msgr2=0x7f8b84101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.027+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 shutdown_connections 2024-09-21T02:54:49.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:49.027+0000 7f8b8a231640 1 -- 172.21.15.31:0/2796464744 wait complete. 2024-09-21T02:54:49.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:54:50.000 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.544+0000 7fc819376640 1 -- 172.21.15.31:0/1436526119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 msgr2=0x7fc81410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.544+0000 7fc819376640 1 --2- 172.21.15.31:0/1436526119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc81410f520 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009ca0 tx=0x7fc80c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 -- 172.21.15.31:0/1436526119 shutdown_connections 2024-09-21T02:54:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 --2- 172.21.15.31:0/1436526119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc81410f520 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 --2- 172.21.15.31:0/1436526119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8141083a0 0x7fc8141087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 --2- 172.21.15.31:0/1436526119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc814102a60 0x7fc814102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 -- 172.21.15.31:0/1436526119 >> 172.21.15.31:0/1436526119 conn(0x7fc814079730 msgr2=0x7fc814079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 -- 172.21.15.31:0/1436526119 shutdown_connections 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 -- 172.21.15.31:0/1436526119 wait complete. 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.545+0000 7fc819376640 1 Processor -- start 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 -- start start 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc814102a60 0x7fc8140ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141083a0 0x7fc8140ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc8141000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc814112960 con 0x7fc814108cf0 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141127e0 con 0x7fc814102a60 2024-09-21T02:54:50.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc819376640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc814112ae0 con 0x7fc8141083a0 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc8137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc8141000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc8137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc8141000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44924/0 (socket says 172.21.15.31:44924) 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc8137fe640 1 -- 172.21.15.31:0/3028169125 learned_addr learned my addr 172.21.15.31:0/3028169125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc812ffd640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc814102a60 0x7fc8140ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.546+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141083a0 0x7fc8140ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8137fe640 1 -- 172.21.15.31:0/3028169125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141083a0 msgr2=0x7fc8140ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8137fe640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141083a0 0x7fc8140ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8137fe640 1 -- 172.21.15.31:0/3028169125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc814102a60 msgr2=0x7fc8140ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8137fe640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc814102a60 0x7fc8140ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8137fe640 1 -- 172.21.15.31:0/3028169125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8141a8780 con 0x7fc814108cf0 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141083a0 0x7fc8140ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc812ffd640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc814102a60 0x7fc8140ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc8137fe640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc8141000f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005cc0 tx=0x7fc80c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:50.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc80bfff640 1 -- 172.21.15.31:0/3028169125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003b70 con 0x7fc814108cf0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8141a8a10 con 0x7fc814108cf0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8141a8fa0 con 0x7fc814108cf0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc80bfff640 1 -- 172.21.15.31:0/3028169125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80c0318f0 con 0x7fc814108cf0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.547+0000 7fc80bfff640 1 -- 172.21.15.31:0/3028169125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c02f950 con 0x7fc814108cf0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.550+0000 7fc81f8ea740 1 Processor -- start 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.551+0000 7fc80bfff640 1 -- 172.21.15.31:0/3028169125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc80c02faf0 con 0x7fc814108cf0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.551+0000 7fc81f8ea740 1 -- start start 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x5560638202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x556063820800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063820d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556063796ea0 con 0x556063458390 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5560637862d0 con 0x556063479070 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55606378a0e0 con 0x5560634501b0 2024-09-21T02:54:50.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063820d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063820d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44934/0 (socket says 172.21.15.31:44934) 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 -- 172.21.15.31:0/2736653784 learned_addr learned my addr 172.21.15.31:0/2736653784 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x5560638202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc812ffd640 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x556063820800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc80bfff640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc8000b6680 0x7fc8000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 -- 172.21.15.31:0/2736653784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 msgr2=0x556063820800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x556063820800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 -- 172.21.15.31:0/2736653784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 msgr2=0x5560638202c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.552+0000 7fc8127fc640 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x5560638202c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc8127fc640 1 -- 172.21.15.31:0/2736653784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560634e4650 con 0x556063458390 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc8137fe640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc8000b6680 0x7fc8000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc8127fc640 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063820d40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc011b20 tx=0x7fc7fc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a57d6d6907bb81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc80bfff640 1 -- 172.21.15.31:0/3028169125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc80c1088c0 con 0x7fc814108cf0 2024-09-21T02:54:50.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc8097fa640 1 -- 172.21.15.31:0/2736653784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc011050 con 0x556063458390 2024-09-21T02:54:50.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc8097fa640 1 -- 172.21.15.31:0/2736653784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0111f0 con 0x556063458390 2024-09-21T02:54:50.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.553+0000 7fc8097fa640 1 -- 172.21.15.31:0/2736653784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0035f0 con 0x556063458390 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 -- 172.21.15.31:0/2736653784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 msgr2=0x556063820d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc8137fe640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc8000b6680 0x7fc8000b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc80c08d810 tx=0x7fc80c098230 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063820d40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc011b20 tx=0x7fc7fc005850 comp rx=0 tx=0).stop 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 -- 172.21.15.31:0/2736653784 shutdown_connections 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x556063820800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063820d40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/2736653784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x5560638202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 -- 172.21.15.31:0/2736653784 >> 172.21.15.31:0/2736653784 conn(0x55606346d460 msgr2=0x55606381e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 -- 172.21.15.31:0/2736653784 shutdown_connections 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.554+0000 7fc81f8ea740 1 -- 172.21.15.31:0/2736653784 wait complete. 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.555+0000 7fc81f8ea740 1 -- start start 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.555+0000 7fc81f8ea740 1 Processor -- start 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x55606382b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x55606382be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x556063837dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5560635e7700 con 0x556063458390 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556063721160 con 0x556063479070 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55606382eaa0 con 0x5560634501b0 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x55606382b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc812ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x55606382be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x55606382b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:46756/0 (socket says 172.21.15.31:46756) 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc8127fc640 1 -- 172.21.15.31:0/3997534444 learned_addr learned my addr 172.21.15.31:0/3997534444 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc81f8ea740 10 client.0 _ll_register_callbacks cb 0x55606373a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x556063837dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc812ffd640 1 -- 172.21.15.31:0/3997534444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 msgr2=0x55606382b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc812ffd640 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x55606382b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc812ffd640 1 -- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 msgr2=0x556063837dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc812ffd640 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x556063837dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.556+0000 7fc812ffd640 1 -- 172.21.15.31:0/3997534444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606374ec90 con 0x556063458390 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc812ffd640 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x55606382be80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc8141010c0 tx=0x7fc80400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d81dec83c8cc871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc7fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:50.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/3997534444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804012110 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc0111f0 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/3997534444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc804004050 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/3997534444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8040122b0 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc81f8ea740 10 client.15978 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55606383a7f0 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.557+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/3997534444 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0111f0 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/3997534444 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc804023020 con 0x556063458390 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc81f8ea740 10 client.15978 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc81f8ea740 10 client.15978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc81f8ea740 20 client.15978 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5560634379f0 0x55606383dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc81f8ea740 4 client.15978 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:50.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556063798df0 con 0x5560634379f0 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.558+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5560634379f0 0x55606383dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.559+0000 7fc7fa7fc640 10 client.15978 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.559+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5560634379f0 0x55606383dcf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e371763cce50de6c server_cookie=39e5ab9cc335b112 in_seq=0 out_seq=0 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.559+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/3997534444 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556063798df0 con 0x5560634379f0 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.559+0000 7fc7fa7fc640 10 client.15978 handle_command_reply: tid=0 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.559+0000 7fc81f8ea740 1 client.15978 shutdown 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 msgr2=0x55606382be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x55606382be80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc8141010c0 tx=0x7fc80400f790 comp rx=0 tx=0).stop 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 shutdown_connections 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5560634379f0 0x55606383dcf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc7fa7fc640 0 client.15978 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556063479070 0x556063837dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x55606382be80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3997534444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634501b0 0x55606382b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 >> 172.21.15.31:0/3997534444 conn(0x55606346d460 msgr2=0x556063823450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:50.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 shutdown_connections 2024-09-21T02:54:50.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.560+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3997534444 wait complete. 2024-09-21T02:54:50.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.561+0000 7fc81f8ea740 20 client.15978 trim_cache size 0 max 16384 2024-09-21T02:54:50.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.589+0000 7fc81f8ea740 1 Processor -- start 2024-09-21T02:54:50.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.589+0000 7fc81f8ea740 1 -- start start 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x55606382d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063837cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063825ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141b18a0 con 0x556063458390 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141b33e0 con 0x5560634501b0 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141b1f70 con 0x5560634379f0 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x55606382d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x55606382d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:46766/0 (socket says 172.21.15.31:46766) 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc8127fc640 1 -- 172.21.15.31:0/3268344969 learned_addr learned my addr 172.21.15.31:0/3268344969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:50.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.590+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063825ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc812ffd640 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063837cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc8127fc640 1 -- 172.21.15.31:0/3268344969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 msgr2=0x556063837cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063837cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc8127fc640 1 -- 172.21.15.31:0/3268344969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 msgr2=0x556063825ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063825ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc8127fc640 1 -- 172.21.15.31:0/3268344969 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606381eaa0 con 0x5560634379f0 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc8127fc640 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x55606382d820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009730 tx=0x7fc7fc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a356a7678eb90f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc808ff9640 1 -- 172.21.15.31:0/3268344969 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0050d0 con 0x5560634379f0 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc808ff9640 1 -- 172.21.15.31:0/3268344969 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0035f0 con 0x5560634379f0 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.591+0000 7fc808ff9640 1 -- 172.21.15.31:0/3268344969 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc01f6a0 con 0x5560634379f0 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3268344969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 msgr2=0x55606382d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x55606382d820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009730 tx=0x7fc7fc005850 comp rx=0 tx=0).stop 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3268344969 shutdown_connections 2024-09-21T02:54:50.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063825ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063837cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/3268344969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x55606382d820 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3268344969 >> 172.21.15.31:0/3268344969 conn(0x55606346d460 msgr2=0x5560638253d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3268344969 shutdown_connections 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.592+0000 7fc81f8ea740 1 -- 172.21.15.31:0/3268344969 wait complete. 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 -- start start 2024-09-21T02:54:50.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 Processor -- start 2024-09-21T02:54:50.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x5560638346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063830b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063831230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141b2d10 con 0x556063458390 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141b2640 con 0x5560634501b0 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141af880 con 0x5560634379f0 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.593+0000 7fc81f8ea740 10 client.0 _ll_register_callbacks cb 0x7fc804009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x5560638346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x5560638346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:46768/0 (socket says 172.21.15.31:46768) 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 -- 172.21.15.31:0/646217153 learned_addr learned my addr 172.21.15.31:0/646217153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063831230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc812ffd640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063830b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 -- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 msgr2=0x556063830b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063830b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 -- 172.21.15.31:0/646217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 msgr2=0x556063831230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063831230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 -- 172.21.15.31:0/646217153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560638a8700 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc8127fc640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x5560638346a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0115e0 tx=0x7fc7fc0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6998e99dc9674c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc7fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/646217153 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc03a020 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/646217153 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc7fc005390 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.594+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc0035f0 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/646217153 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc005590 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 10 client.25663 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5560638a97d0 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/646217153 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0035f0 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/646217153 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7fc01f9e0 con 0x5560634379f0 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 10 client.25663 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:50.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 10 client.25663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 20 client.25663 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556063479070 0x5560638abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 4 client.25663 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560637948a0 con 0x556063479070 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.595+0000 7fc812ffd640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556063479070 0x5560638abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.596+0000 7fc7fa7fc640 10 client.25663 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:50.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.596+0000 7fc812ffd640 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556063479070 0x5560638abca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad6c0deaec5e0602 server_cookie=fcadf6270f43333e in_seq=0 out_seq=0 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc7fa7fc640 1 -- 172.21.15.31:0/646217153 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560637948a0 con 0x556063479070 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc7fa7fc640 10 client.25663 handle_command_reply: tid=0 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 client.25663 shutdown 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 msgr2=0x5560638346a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x5560638346a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0115e0 tx=0x7fc7fc0096c0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 shutdown_connections 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x556063479070 0x5560638abca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc7fa7fc640 0 client.25663 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:54:50.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556063458390 0x556063831230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5560634501b0 0x556063830b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 --2- 172.21.15.31:0/646217153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5560634379f0 0x5560638346a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 >> 172.21.15.31:0/646217153 conn(0x55606346d460 msgr2=0x55606382d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 shutdown_connections 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 1 -- 172.21.15.31:0/646217153 wait complete. 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.597+0000 7fc81f8ea740 20 client.25663 trim_cache size 0 max 16384 2024-09-21T02:54:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc8000b6680 msgr2=0x7fc8000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc8000b6680 0x7fc8000b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc80c08d810 tx=0x7fc80c098230 comp rx=0 tx=0).stop 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 msgr2=0x7fc8141000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc8141000f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005cc0 tx=0x7fc80c031700 comp rx=0 tx=0).stop 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 shutdown_connections 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc8000b6680 0x7fc8000b8b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8141083a0 0x7fc8140ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc814108cf0 0x7fc8141000f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 --2- 172.21.15.31:0/3028169125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc814102a60 0x7fc8140ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:50.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 >> 172.21.15.31:0/3028169125 conn(0x7fc814079730 msgr2=0x7fc8141019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:50.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 shutdown_connections 2024-09-21T02:54:50.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:50.598+0000 7fc819376640 1 -- 172.21.15.31:0/3028169125 wait complete. 2024-09-21T02:54:50.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:50 smithi031 ceph-mon[31209]: pgmap v383: 97 pgs: 97 active+clean; 183 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-21T02:54:50.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:50 smithi165 ceph-mon[34818]: pgmap v383: 97 pgs: 97 active+clean; 183 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-21T02:54:50.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:50 smithi178 ceph-mon[34759]: pgmap v383: 97 pgs: 97 active+clean; 183 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 16 op/s 2024-09-21T02:54:51.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:54:51.669 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 -- 172.21.15.31:0/387020341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a41094c0 msgr2=0x7f97a41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 --2- 172.21.15.31:0/387020341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a41094c0 0x7f97a41098d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f978c009900 tx=0x7f978c02a410 comp rx=0 tx=0).stop 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 -- 172.21.15.31:0/387020341 shutdown_connections 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 --2- 172.21.15.31:0/387020341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a4109e10 0x7f97a4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 --2- 172.21.15.31:0/387020341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a41094c0 0x7f97a41098d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 --2- 172.21.15.31:0/387020341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4103b80 0x7f97a4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.203+0000 7f97a97f5640 1 -- 172.21.15.31:0/387020341 >> 172.21.15.31:0/387020341 conn(0x7f97a40ff610 msgr2=0x7f97a4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.204+0000 7f97a97f5640 1 -- 172.21.15.31:0/387020341 shutdown_connections 2024-09-21T02:54:52.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.204+0000 7f97a97f5640 1 -- 172.21.15.31:0/387020341 wait complete. 2024-09-21T02:54:52.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.204+0000 7f97a97f5640 1 Processor -- start 2024-09-21T02:54:52.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 -- start start 2024-09-21T02:54:52.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4103b80 0x7f97a419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a41094c0 0x7f97a419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 0x7f97a419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4113ad0 con 0x7f97a4109e10 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4113950 con 0x7f97a41094c0 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a97f5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4113c50 con 0x7f97a4103b80 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 0x7f97a419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 0x7f97a419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44962/0 (socket says 172.21.15.31:44962) 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 -- 172.21.15.31:0/2460324952 learned_addr learned my addr 172.21.15.31:0/2460324952 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4103b80 0x7f97a419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f979a7fc640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a41094c0 0x7f97a419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 -- 172.21.15.31:0/2460324952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4103b80 msgr2=0x7f97a419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4103b80 0x7f97a419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 -- 172.21.15.31:0/2460324952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a41094c0 msgr2=0x7f97a419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a41094c0 0x7f97a419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.205+0000 7f97a37fe640 1 -- 172.21.15.31:0/2460324952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a41a03f0 con 0x7f97a4109e10 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f979a7fc640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a41094c0 0x7f97a419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f97a37fe640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 0x7f97a419fcc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f979c00eaa0 tx=0x7f979c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f97a0ff9640 1 -- 172.21.15.31:0/2460324952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c00cdb0 con 0x7f97a4109e10 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f97a0ff9640 1 -- 172.21.15.31:0/2460324952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979c005020 con 0x7f97a4109e10 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a41a06e0 con 0x7f97a4109e10 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f97a0ff9640 1 -- 172.21.15.31:0/2460324952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0116d0 con 0x7f97a4109e10 2024-09-21T02:54:52.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.206+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a41a6e40 con 0x7f97a4109e10 2024-09-21T02:54:52.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.209+0000 7f97afd69740 1 Processor -- start 2024-09-21T02:54:52.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.209+0000 7f97a0ff9640 1 -- 172.21.15.31:0/2460324952 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f979c011870 con 0x7f97a4109e10 2024-09-21T02:54:52.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.209+0000 7f97afd69740 1 -- start start 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.209+0000 7f97afd69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.209+0000 7f97afd69740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97afd69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97afd69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5644aa647ea0 con 0x5644aa309390 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97afd69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644aa6372d0 con 0x5644aa32a070 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97afd69740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644aa63b0e0 con 0x5644aa3011b0 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f979a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f979a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44966/0 (socket says 172.21.15.31:44966) 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f979a7fc640 1 -- 172.21.15.31:0/3670631738 learned_addr learned my addr 172.21.15.31:0/3670631738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:52.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3670631738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 msgr2=0x5644aa6d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3670631738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 msgr2=0x5644aa6d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.210+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3670631738 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644aa395650 con 0x5644aa3011b0 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6d1be0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f979400bbf0 tx=0x7f97940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff7da6432690e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f97997fa640 1 -- 172.21.15.31:0/3670631738 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794004030 con 0x5644aa3011b0 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f97997fa640 1 -- 172.21.15.31:0/3670631738 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97940035f0 con 0x5644aa3011b0 2024-09-21T02:54:52.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f97a0ff9640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97900b6680 0x7f97900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f97997fa640 1 -- 172.21.15.31:0/3670631738 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97940116a0 con 0x5644aa3011b0 2024-09-21T02:54:52.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f979a7fc640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97900b6680 0x7f97900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.211+0000 7f97a0ff9640 1 -- 172.21.15.31:0/2460324952 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f979c0e7970 con 0x7f97a4109e10 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 -- 172.21.15.31:0/3670631738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 msgr2=0x5644aa6d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f979a7fc640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97900b6680 0x7f97900b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f978c032fd0 tx=0x7f978c040040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6d1be0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f979400bbf0 tx=0x7f97940107a0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 -- 172.21.15.31:0/3670631738 shutdown_connections 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6d1be0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 --2- 172.21.15.31:0/3670631738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 -- 172.21.15.31:0/3670631738 >> 172.21.15.31:0/3670631738 conn(0x5644aa31e460 msgr2=0x5644aa6cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 -- 172.21.15.31:0/3670631738 shutdown_connections 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.212+0000 7f97afd69740 1 -- 172.21.15.31:0/3670631738 wait complete. 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.213+0000 7f97afd69740 1 -- start start 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.213+0000 7f97afd69740 1 Processor -- start 2024-09-21T02:54:52.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.213+0000 7f97afd69740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.213+0000 7f97afd69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6e1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.213+0000 7f97afd69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f97afd69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5644aa498700 con 0x5644aa309390 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f97afd69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644aa5d2160 con 0x5644aa32a070 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f97afd69740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644aa6e4230 con 0x5644aa3011b0 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f97afd69740 10 client.0 _ll_register_callbacks cb 0x5644aa5eb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6e1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6e1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44968/0 (socket says 172.21.15.31:44968) 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 -- 172.21.15.31:0/3686638774 learned_addr learned my addr 172.21.15.31:0/3686638774 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f97a37fe640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 -- 172.21.15.31:0/3686638774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 msgr2=0x5644aa6e10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6e10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 -- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 msgr2=0x5644aa6e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 -- 172.21.15.31:0/3686638774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644aa5ffc90 con 0x5644aa309390 2024-09-21T02:54:52.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6e1620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f978c009230 tx=0x7f978c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d72affa93366a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:52.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.214+0000 7f978a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f978a7fc640 1 -- 172.21.15.31:0/3686638774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c035040 con 0x5644aa309390 2024-09-21T02:54:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f978a7fc640 1 -- 172.21.15.31:0/3686638774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f978c0038b0 con 0x5644aa309390 2024-09-21T02:54:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97940035f0 con 0x5644aa309390 2024-09-21T02:54:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f978a7fc640 1 -- 172.21.15.31:0/3686638774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f978c003ab0 con 0x5644aa309390 2024-09-21T02:54:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 10 client.15996 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:52.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644aa6efec0 con 0x5644aa309390 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f978a7fc640 1 -- 172.21.15.31:0/3686638774 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97940035f0 con 0x5644aa309390 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f978a7fc640 1 -- 172.21.15.31:0/3686638774 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f978c05a070 con 0x5644aa309390 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 10 client.15996 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 20 client.15996 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa2e89f0 0x5644aa6f3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 4 client.15996 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.215+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644aa649df0 con 0x5644aa2e89f0 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.216+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa2e89f0 0x5644aa6f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.216+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa2e89f0 0x5644aa6f3470 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d515c1113a192e9f server_cookie=d793eb73024155dc in_seq=0 out_seq=0 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.216+0000 7f978a7fc640 10 client.15996 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f978a7fc640 1 -- 172.21.15.31:0/3686638774 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644aa649df0 con 0x5644aa2e89f0 2024-09-21T02:54:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f978a7fc640 10 client.15996 handle_command_reply: tid=0 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 client.15996 shutdown 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 msgr2=0x5644aa6e1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6e1620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f978c009230 tx=0x7f978c02a410 comp rx=0 tx=0).stop 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 shutdown_connections 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa2e89f0 0x5644aa6f3470 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa32a070 0x5644aa6e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa309390 0x5644aa6e1620 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 --2- 172.21.15.31:0/3686638774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa3011b0 0x5644aa6e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 >> 172.21.15.31:0/3686638774 conn(0x5644aa31e460 msgr2=0x5644aa6d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f978a7fc640 0 client.15996 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 shutdown_connections 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.217+0000 7f97afd69740 1 -- 172.21.15.31:0/3686638774 wait complete. 2024-09-21T02:54:52.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.218+0000 7f97afd69740 20 client.15996 trim_cache size 0 max 16384 2024-09-21T02:54:52.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.234+0000 7f97afd69740 1 Processor -- start 2024-09-21T02:54:52.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.234+0000 7f97afd69740 1 -- start start 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97afd69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97afd69740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6de2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97afd69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6de810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97afd69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41ad540 con 0x5644aa2bdda0 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97afd69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41af080 con 0x5644aa3011b0 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97afd69740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41adc10 con 0x5644aa32a070 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6de810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6de810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44970/0 (socket says 172.21.15.31:44970) 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 -- 172.21.15.31:0/812084513 learned_addr learned my addr 172.21.15.31:0/812084513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:52.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f979a7fc640 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6de2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 -- 172.21.15.31:0/812084513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 msgr2=0x5644aa6de2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6de2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 -- 172.21.15.31:0/812084513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 msgr2=0x5644aa6cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a37fe640 1 -- 172.21.15.31:0/812084513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644aa6d44d0 con 0x5644aa2bdda0 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.235+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.236+0000 7f97a37fe640 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6de810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f979c0aaa50 tx=0x7f979c0ad250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00226e85a8baef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.236+0000 7f9798ff9640 1 -- 172.21.15.31:0/812084513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c01f040 con 0x5644aa2bdda0 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.236+0000 7f9798ff9640 1 -- 172.21.15.31:0/812084513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979c0b22c0 con 0x5644aa2bdda0 2024-09-21T02:54:52.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.236+0000 7f9798ff9640 1 -- 172.21.15.31:0/812084513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0054f0 con 0x5644aa2bdda0 2024-09-21T02:54:52.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 -- 172.21.15.31:0/812084513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 msgr2=0x5644aa6de810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6de810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f979c0aaa50 tx=0x7f979c0ad250 comp rx=0 tx=0).stop 2024-09-21T02:54:52.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 -- 172.21.15.31:0/812084513 shutdown_connections 2024-09-21T02:54:52.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6de2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6de810 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 --2- 172.21.15.31:0/812084513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 -- 172.21.15.31:0/812084513 >> 172.21.15.31:0/812084513 conn(0x5644aa31e460 msgr2=0x5644aa6ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 -- 172.21.15.31:0/812084513 shutdown_connections 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 -- 172.21.15.31:0/812084513 wait complete. 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 -- start start 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.237+0000 7f97afd69740 1 Processor -- start 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41ae9b0 con 0x5644aa2bdda0 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41ae2e0 con 0x5644aa3011b0 2024-09-21T02:54:52.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41ab520 con 0x5644aa32a070 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97afd69740 10 client.0 _ll_register_callbacks cb 0x7f978c032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6e9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44974/0 (socket says 172.21.15.31:44974) 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3401079387 learned_addr learned my addr 172.21.15.31:0/3401079387 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a37fe640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3401079387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 msgr2=0x5644aa6e56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6e56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 msgr2=0x5644aa6e5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6e5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a2ffd640 1 -- 172.21.15.31:0/3401079387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644aa6e5e30 con 0x5644aa2bdda0 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.238+0000 7f97a37fe640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:52.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97a2ffd640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6e9cb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5644aa6d0320 tx=0x7f979400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda4b24a58db1ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f978a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f978a7fc640 1 -- 172.21.15.31:0/3401079387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794005240 con 0x5644aa2bdda0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f978a7fc640 1 -- 172.21.15.31:0/3401079387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f97940053e0 con 0x5644aa2bdda0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f979c0b22c0 con 0x5644aa2bdda0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f978a7fc640 1 -- 172.21.15.31:0/3401079387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9794035050 con 0x5644aa2bdda0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97afd69740 10 client.16002 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644aa75dd80 con 0x5644aa2bdda0 2024-09-21T02:54:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f978a7fc640 1 -- 172.21.15.31:0/3401079387 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979c0b22c0 con 0x5644aa2bdda0 2024-09-21T02:54:52.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f978a7fc640 1 -- 172.21.15.31:0/3401079387 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f97940285e0 con 0x5644aa2bdda0 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97afd69740 10 client.16002 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97afd69740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.239+0000 7f97afd69740 20 client.16002 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f97afd69740 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa75e1c0 0x5644aa7605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f97afd69740 4 client.16002 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644aa6458a0 con 0x5644aa75e1c0 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa75e1c0 0x5644aa7605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f978a7fc640 10 client.16002 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f979a7fc640 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa75e1c0 0x5644aa7605e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8aa2e7f97000dad server_cookie=d44381e366fe607b in_seq=0 out_seq=0 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f978a7fc640 1 -- 172.21.15.31:0/3401079387 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644aa6458a0 con 0x5644aa75e1c0 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.240+0000 7f978a7fc640 10 client.16002 handle_command_reply: tid=0 2024-09-21T02:54:52.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 client.16002 shutdown 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 msgr2=0x5644aa6e9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6e9cb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5644aa6d0320 tx=0x7f979400bce0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 shutdown_connections 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5644aa75e1c0 0x5644aa7605e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644aa32a070 0x5644aa6e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644aa3011b0 0x5644aa6e5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f978a7fc640 0 client.16002 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 --2- 172.21.15.31:0/3401079387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644aa2bdda0 0x5644aa6e9cb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 >> 172.21.15.31:0/3401079387 conn(0x5644aa31e460 msgr2=0x5644aa6df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 shutdown_connections 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 1 -- 172.21.15.31:0/3401079387 wait complete. 2024-09-21T02:54:52.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.241+0000 7f97afd69740 20 client.16002 trim_cache size 0 max 16384 2024-09-21T02:54:52.247 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:52.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97900b6680 msgr2=0x7f97900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97900b6680 0x7f97900b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f978c032fd0 tx=0x7f978c040040 comp rx=0 tx=0).stop 2024-09-21T02:54:52.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 msgr2=0x7f97a419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:52.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 0x7f97a419fcc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f979c00eaa0 tx=0x7f979c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 shutdown_connections 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97900b6680 0x7f97900b8b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97a4109e10 0x7f97a419fcc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a41094c0 0x7f97a419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 --2- 172.21.15.31:0/2460324952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97a4103b80 0x7f97a419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 >> 172.21.15.31:0/2460324952 conn(0x7f97a40ff610 msgr2=0x7f97a4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 shutdown_connections 2024-09-21T02:54:52.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:52.242+0000 7f97a97f5640 1 -- 172.21.15.31:0/2460324952 wait complete. 2024-09-21T02:54:52.920 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:52 smithi031 ceph-mon[31209]: pgmap v384: 97 pgs: 97 active+clean; 207 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-21T02:54:52.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:52 smithi165 ceph-mon[34818]: pgmap v384: 97 pgs: 97 active+clean; 207 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-21T02:54:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:52 smithi178 ceph-mon[34759]: pgmap v384: 97 pgs: 97 active+clean; 207 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-21T02:54:53.954 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:54:53.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:54:54.230 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 -- 172.21.15.31:0/275807957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4104840 msgr2=0x7f2aa4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/275807957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4104840 0x7f2aa4104c50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009900 tx=0x7f2a9402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 -- 172.21.15.31:0/275807957 shutdown_connections 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/275807957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 0x7f2aa4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/275807957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa4105270 0x7f2aa410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/275807957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4104840 0x7f2aa4104c50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 -- 172.21.15.31:0/275807957 >> 172.21.15.31:0/275807957 conn(0x7f2aa4079730 msgr2=0x7f2aa4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 -- 172.21.15.31:0/275807957 shutdown_connections 2024-09-21T02:54:54.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.776+0000 7f2aa94d8640 1 -- 172.21.15.31:0/275807957 wait complete. 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 Processor -- start 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 -- start start 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa4104840 0x7f2aa419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 0x7f2aa419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 0x7f2aa419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4113a90 con 0x7f2aa4105270 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4113910 con 0x7f2aa4104840 2024-09-21T02:54:54.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.777+0000 7f2aa94d8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4113c10 con 0x7f2aa410ddf0 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 0x7f2aa419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 0x7f2aa419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44982/0 (socket says 172.21.15.31:44982) 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3558134842 learned_addr learned my addr 172.21.15.31:0/3558134842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 0x7f2aa419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa4104840 0x7f2aa419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3558134842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 msgr2=0x7f2aa419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 0x7f2aa419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3558134842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa4104840 msgr2=0x7f2aa419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa4104840 0x7f2aa419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3558134842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa41a04f0 con 0x7f2aa4105270 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 0x7f2aa419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 0x7f2aa419ab20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d740 tx=0x7f2a8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.778+0000 7f2aa0ff9640 1 -- 172.21.15.31:0/3558134842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c017070 con 0x7f2aa4105270 2024-09-21T02:54:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.779+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa41a07e0 con 0x7f2aa4105270 2024-09-21T02:54:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.779+0000 7f2aa0ff9640 1 -- 172.21.15.31:0/3558134842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a8c00de50 con 0x7f2aa4105270 2024-09-21T02:54:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.779+0000 7f2aa0ff9640 1 -- 172.21.15.31:0/3558134842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00f620 con 0x7f2aa4105270 2024-09-21T02:54:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.779+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa41a6f40 con 0x7f2aa4105270 2024-09-21T02:54:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aa0ff9640 1 -- 172.21.15.31:0/3558134842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2a8c01a070 con 0x7f2aa4105270 2024-09-21T02:54:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 Processor -- start 2024-09-21T02:54:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 -- start start 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14507430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14137490 0x556b14507970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b14507eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556b1447de60 con 0x556b14160350 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b1446d290 con 0x556b14137490 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b144710a0 con 0x556b1413f670 2024-09-21T02:54:54.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.782+0000 7f2aa37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14507430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14507430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44996/0 (socket says 172.21.15.31:44996) 2024-09-21T02:54:54.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 -- 172.21.15.31:0/2893374978 learned_addr learned my addr 172.21.15.31:0/2893374978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:54.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b14507eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14137490 0x556b14507970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 -- 172.21.15.31:0/2893374978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 msgr2=0x556b14507eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b14507eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 -- 172.21.15.31:0/2893374978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14137490 msgr2=0x556b14507970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14137490 0x556b14507970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 -- 172.21.15.31:0/2893374978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b141cb930 con 0x556b14160350 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14137490 0x556b14507970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa0ff9640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2a900b6680 0x7f2a900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14507430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c011790 tx=0x7f2a9c011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bcb993a987e835 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.783+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2a900b6680 0x7f2a900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.784+0000 7f2aa0ff9640 1 -- 172.21.15.31:0/3558134842 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2a8c0e7090 con 0x7f2aa4105270 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.784+0000 7f2a997fa640 1 -- 172.21.15.31:0/2893374978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c003c30 con 0x556b14160350 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.784+0000 7f2a997fa640 1 -- 172.21.15.31:0/2893374978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c00fe50 con 0x556b14160350 2024-09-21T02:54:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.784+0000 7f2a997fa640 1 -- 172.21.15.31:0/2893374978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0193e0 con 0x556b14160350 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 -- 172.21.15.31:0/2893374978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 msgr2=0x556b14507430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2a900b6680 0x7f2a900b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a94000c00 tx=0x7f2a94031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14507430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c011790 tx=0x7f2a9c011c50 comp rx=0 tx=0).stop 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 -- 172.21.15.31:0/2893374978 shutdown_connections 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b14507eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14137490 0x556b14507970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/2893374978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14507430 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 -- 172.21.15.31:0/2893374978 >> 172.21.15.31:0/2893374978 conn(0x556b14154740 msgr2=0x556b14505320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 -- 172.21.15.31:0/2893374978 shutdown_connections 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.785+0000 7f2aafa4c740 1 -- 172.21.15.31:0/2893374978 wait complete. 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 -- start start 2024-09-21T02:54:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 Processor -- start 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 0x556b14513070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b145135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1451ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556b142ce890 con 0x556b14137490 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b14407d40 con 0x556b14160350 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b14515c10 con 0x556b1413f670 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aafa4c740 10 client.0 _ll_register_callbacks cb 0x556b1446bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aa2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 0x556b14513070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aa2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 0x556b14513070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45012/0 (socket says 172.21.15.31:45012) 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2aa2ffd640 1 -- 172.21.15.31:0/4137522298 learned_addr learned my addr 172.21.15.31:0/4137522298 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.786+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b145135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1451ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa2ffd640 1 -- 172.21.15.31:0/4137522298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 msgr2=0x556b145135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b145135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa2ffd640 1 -- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 msgr2=0x556b1451ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1451ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa2ffd640 1 -- 172.21.15.31:0/4137522298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1451f4f0 con 0x556b14137490 2024-09-21T02:54:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 0x556b14513070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2aa406bc00 tx=0x7f2a94009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acee97b7149d3e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2a8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/4137522298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9404d070 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/4137522298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2a94005030 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9c00fe50 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/4137522298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a94059050 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aafa4c740 10 client.16014 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.787+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b14521e20 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/4137522298 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c00fe50 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/4137522298 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2a940035f0 con 0x556b14137490 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2aafa4c740 10 client.16014 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2aafa4c740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2aafa4c740 20 client.16014 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b1411ecd0 0x556b14525390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2aafa4c740 4 client.16014 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1447fdb0 con 0x556b1411ecd0 2024-09-21T02:54:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.788+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b1411ecd0 0x556b14525390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.789+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b1411ecd0 0x556b14525390 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf0d48a706413b0 server_cookie=b6180ca4f16edeab in_seq=0 out_seq=0 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.789+0000 7f2a8a7fc640 10 client.16014 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/4137522298 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b1447fdb0 con 0x556b1411ecd0 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2a8a7fc640 10 client.16014 handle_command_reply: tid=0 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 client.16014 shutdown 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 msgr2=0x556b14513070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 0x556b14513070 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2aa406bc00 tx=0x7f2a94009260 comp rx=0 tx=0).stop 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 shutdown_connections 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b1411ecd0 0x556b14525390 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b1413f670 0x556b145135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1451ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/4137522298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14137490 0x556b14513070 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2a8a7fc640 0 client.16014 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 >> 172.21.15.31:0/4137522298 conn(0x556b14154740 msgr2=0x556b1450a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 shutdown_connections 2024-09-21T02:54:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 1 -- 172.21.15.31:0/4137522298 wait complete. 2024-09-21T02:54:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.790+0000 7f2aafa4c740 20 client.16014 trim_cache size 0 max 16384 2024-09-21T02:54:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.808+0000 7f2aafa4c740 1 Processor -- start 2024-09-21T02:54:54.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.808+0000 7f2aafa4c740 1 -- start start 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b1450cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b1413f670 0x556b1451ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14512700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41af750 con 0x556b14160350 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b1290 con 0x556b1413f670 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41afe20 con 0x556b14137490 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aa37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14512700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aa37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14512700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45014/0 (socket says 172.21.15.31:45014) 2024-09-21T02:54:54.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aa37fe640 1 -- 172.21.15.31:0/3726623841 learned_addr learned my addr 172.21.15.31:0/3726623841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b1450cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.809+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b1413f670 0x556b1451ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2aa37fe640 1 -- 172.21.15.31:0/3726623841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 msgr2=0x556b1450cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b1450cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2aa37fe640 1 -- 172.21.15.31:0/3726623841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b1413f670 msgr2=0x556b1451ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b1413f670 0x556b1451ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2aa37fe640 1 -- 172.21.15.31:0/3726623841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b14513a30 con 0x556b14160350 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b1413f670 0x556b1451ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:54.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14512700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c011730 tx=0x7f2a9c011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1117ee86314c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2a98ff9640 1 -- 172.21.15.31:0/3726623841 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c01f040 con 0x556b14160350 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2a98ff9640 1 -- 172.21.15.31:0/3726623841 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c0111b0 con 0x556b14160350 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.810+0000 7f2a98ff9640 1 -- 172.21.15.31:0/3726623841 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0052b0 con 0x556b14160350 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3726623841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 msgr2=0x556b14512700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14512700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c011730 tx=0x7f2a9c011040 comp rx=0 tx=0).stop 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3726623841 shutdown_connections 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b14160350 0x556b14512700 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b1413f670 0x556b1451ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3726623841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b1450cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3726623841 >> 172.21.15.31:0/3726623841 conn(0x556b14154740 msgr2=0x556b1450c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3726623841 shutdown_connections 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.811+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3726623841 wait complete. 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 -- start start 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 Processor -- start 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b145172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 0x556b14517880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1458ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b0bc0 con 0x556b1413f670 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b04f0 con 0x556b14160350 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41ad730 con 0x556b14137490 2024-09-21T02:54:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aafa4c740 10 client.0 _ll_register_callbacks cb 0x7f2a94031520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 0x556b14517880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 0x556b14517880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45018/0 (socket says 172.21.15.31:45018) 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3221274374 learned_addr learned my addr 172.21.15.31:0/3221274374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.812+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b145172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1458ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3221274374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 msgr2=0x556b145172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b145172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 msgr2=0x556b1458ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1458ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a9bfff640 1 -- 172.21.15.31:0/3221274374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1458f290 con 0x556b1413f670 2024-09-21T02:54:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2aa37fe640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1458ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:54:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a9bfff640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 0x556b14517880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0e48b0 tx=0x7f2a8c0eabe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191581edc1b4b42e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/3221274374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c021040 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/3221274374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2a8c0a9220 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9c0111b0 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/3221274374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a8c0acd80 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2aafa4c740 10 client.16020 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.813+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b14590480 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/3221274374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c0111b0 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/3221274374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2a8c0981a0 con 0x556b1413f670 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aafa4c740 10 client.16020 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aafa4c740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:54.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aafa4c740 20 client.16020 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b140f4080 0x556b14592c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aafa4c740 4 client.16020 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b1447b860 con 0x556b140f4080 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.814+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b140f4080 0x556b14592c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.815+0000 7f2a8a7fc640 10 client.16020 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.815+0000 7f2aa2ffd640 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b140f4080 0x556b14592c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f24fc478dca36d3 server_cookie=b94a9d023dec757a in_seq=0 out_seq=0 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.816+0000 7f2a8a7fc640 1 -- 172.21.15.31:0/3221274374 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b1447b860 con 0x556b140f4080 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.816+0000 7f2a8a7fc640 10 client.16020 handle_command_reply: tid=0 2024-09-21T02:54:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.816+0000 7f2aafa4c740 1 client.16020 shutdown 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 msgr2=0x556b14517880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 0x556b14517880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0e48b0 tx=0x7f2a8c0eabe0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 shutdown_connections 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x556b140f4080 0x556b14592c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b14160350 0x556b1458ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2a8a7fc640 0 client.16020 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b1413f670 0x556b14517880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 --2- 172.21.15.31:0/3221274374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b14137490 0x556b145172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 >> 172.21.15.31:0/3221274374 conn(0x556b14154740 msgr2=0x556b14508460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 shutdown_connections 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 1 -- 172.21.15.31:0/3221274374 wait complete. 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.817+0000 7f2aafa4c740 20 client.16020 trim_cache size 0 max 16384 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423", 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:54:54.821 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2a900b6680 msgr2=0x7f2a900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2a900b6680 0x7f2a900b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a94000c00 tx=0x7f2a94031400 comp rx=0 tx=0).stop 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 msgr2=0x7f2aa419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 0x7f2aa419ab20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d740 tx=0x7f2a8c00dc00 comp rx=0 tx=0).stop 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 shutdown_connections 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2a900b6680 0x7f2a900b8b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2aa410ddf0 0x7f2aa419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa4105270 0x7f2aa419ab20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 --2- 172.21.15.31:0/3558134842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2aa4104840 0x7f2aa419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:54.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 >> 172.21.15.31:0/3558134842 conn(0x7f2aa4079730 msgr2=0x7f2aa4107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:54.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 shutdown_connections 2024-09-21T02:54:54.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:54.818+0000 7f2aa94d8640 1 -- 172.21.15.31:0/3558134842 wait complete. 2024-09-21T02:54:54.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:54 smithi165 ceph-mon[34818]: pgmap v385: 97 pgs: 97 active+clean; 231 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-21T02:54:54.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:54 smithi178 ceph-mon[34759]: pgmap v385: 97 pgs: 97 active+clean; 231 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-21T02:54:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:54 smithi031 ceph-mon[31209]: pgmap v385: 97 pgs: 97 active+clean; 231 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-21T02:54:55.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:54:55.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:55 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:54:55.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:55 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:55.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:55 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:54:55.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:55 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:55.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:55 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:54:55.851 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:55.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:55 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:54:55.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:55 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:55.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:55 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:54:55.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:55 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:55.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:55 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:54:55.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:55 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:54:55.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:55 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:55.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:55 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:54:55.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:55 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:54:55.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:55 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 -- 172.21.15.31:0/3731833504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418102ae0 msgr2=0x7f9418102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 --2- 172.21.15.31:0/3731833504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418102ae0 0x7f9418102ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9408009900 tx=0x7f940802a3f0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 -- 172.21.15.31:0/3731833504 shutdown_connections 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 --2- 172.21.15.31:0/3731833504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9418108d20 0x7f941810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 --2- 172.21.15.31:0/3731833504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f94181087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 --2- 172.21.15.31:0/3731833504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418102ae0 0x7f9418102ef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 -- 172.21.15.31:0/3731833504 >> 172.21.15.31:0/3731833504 conn(0x7f9418079760 msgr2=0x7f9418079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.342+0000 7f941df89640 1 -- 172.21.15.31:0/3731833504 shutdown_connections 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.343+0000 7f941df89640 1 -- 172.21.15.31:0/3731833504 wait complete. 2024-09-21T02:54:56.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.343+0000 7f941df89640 1 Processor -- start 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.343+0000 7f941df89640 1 -- start start 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f941df89640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9418102ae0 0x7f941819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f941df89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f941819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f941df89640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418108d20 0x7f941819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f941df89640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94181129e0 con 0x7f94181083d0 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f941df89640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9418112860 con 0x7f9418102ae0 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f941df89640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9418112b60 con 0x7f9418108d20 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f941819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f941819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45040/0 (socket says 172.21.15.31:45040) 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 -- 172.21.15.31:0/264778821 learned_addr learned my addr 172.21.15.31:0/264778821 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f94177fe640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9418102ae0 0x7f941819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9417fff640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418108d20 0x7f941819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 -- 172.21.15.31:0/264778821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418108d20 msgr2=0x7f941819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418108d20 0x7f941819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 -- 172.21.15.31:0/264778821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9418102ae0 msgr2=0x7f941819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9418102ae0 0x7f941819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.344+0000 7f9416ffd640 1 -- 172.21.15.31:0/264778821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a0390 con 0x7f94181083d0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.345+0000 7f9416ffd640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f941819a9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f940c00d770 tx=0x7f940c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.345+0000 7f9414ff9640 1 -- 172.21.15.31:0/264778821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c017070 con 0x7f94181083d0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.345+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a0680 con 0x7f94181083d0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.345+0000 7f9414ff9640 1 -- 172.21.15.31:0/264778821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f940c009c30 con 0x7f94181083d0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.345+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a6de0 con 0x7f94181083d0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.345+0000 7f9414ff9640 1 -- 172.21.15.31:0/264778821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00f620 con 0x7f94181083d0 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 Processor -- start 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 -- start start 2024-09-21T02:54:56.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e26070 0x5592e21cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 0x5592e21cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5592e2143e50 con 0x5592e1dfd1b0 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5592e2133280 con 0x5592e1e05390 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94244fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5592e2137090 con 0x5592e1e26070 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 0x5592e21cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 0x5592e21cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45050/0 (socket says 172.21.15.31:45050) 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f94177fe640 1 -- 172.21.15.31:0/3487371642 learned_addr learned my addr 172.21.15.31:0/3487371642 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.348+0000 7f9417fff640 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e26070 0x5592e21cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f94177fe640 1 -- 172.21.15.31:0/3487371642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e26070 msgr2=0x5592e21cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f94177fe640 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e26070 0x5592e21cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f94177fe640 1 -- 172.21.15.31:0/3487371642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 msgr2=0x5592e21ce0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f94177fe640 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21ce0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f94177fe640 1 -- 172.21.15.31:0/3487371642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e1e91650 con 0x5592e1dfd1b0 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f94177fe640 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 0x5592e21cdb90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9408009230 tx=0x7f9408004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ba2b45cdfce7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f93fdffb640 1 -- 172.21.15.31:0/3487371642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408035040 con 0x5592e1dfd1b0 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f93fdffb640 1 -- 172.21.15.31:0/3487371642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9408043070 con 0x5592e1dfd1b0 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.349+0000 7f93fdffb640 1 -- 172.21.15.31:0/3487371642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408004770 con 0x5592e1dfd1b0 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 -- 172.21.15.31:0/3487371642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 msgr2=0x5592e21cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 0x5592e21cdb90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9408009230 tx=0x7f9408004180 comp rx=0 tx=0).stop 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f9414ff9640 1 -- 172.21.15.31:0/264778821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f940c01a070 con 0x7f94181083d0 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 -- 172.21.15.31:0/3487371642 shutdown_connections 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1dfd1b0 0x5592e21cdb90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 --2- 172.21.15.31:0/3487371642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e26070 0x5592e21cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.350+0000 7f94244fd740 1 -- 172.21.15.31:0/3487371642 >> 172.21.15.31:0/3487371642 conn(0x5592e1e1a460 msgr2=0x5592e21cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:56.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.351+0000 7f94244fd740 1 -- 172.21.15.31:0/3487371642 shutdown_connections 2024-09-21T02:54:56.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.351+0000 7f94244fd740 1 -- 172.21.15.31:0/3487371642 wait complete. 2024-09-21T02:54:56.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.351+0000 7f94244fd740 1 -- start start 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f94244fd740 1 Processor -- start 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f9414ff9640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94000b6610 0x7f94000b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f94244fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1dfd1b0 0x5592e21d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f94244fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21d9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f94244fd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e26070 0x5592e21e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f9417fff640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94000b6610 0x7f94000b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f9414ff9640 1 -- 172.21.15.31:0/264778821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f940c0e7110 con 0x7f94181083d0 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.352+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94177fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1dfd1b0 0x5592e21d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21d9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:45824/0 (socket says 172.21.15.31:45824) 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f9416ffd640 1 -- 172.21.15.31:0/1789180814 learned_addr learned my addr 172.21.15.31:0/1789180814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:56.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f9417fff640 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e26070 0x5592e21e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5592e1f94700 con 0x5592e1e26070 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5592e20ce110 con 0x5592e1e05390 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5592e21dbdd0 con 0x5592e1dfd1b0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94244fd740 10 client.0 _ll_register_callbacks cb 0x5592e20e7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f9417fff640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94000b6610 0x7f94000b8b20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9410005fd0 tx=0x7f9410015a60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94177fe640 1 -- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 msgr2=0x5592e21d9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94177fe640 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21d9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94177fe640 1 -- 172.21.15.31:0/1789180814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e26070 msgr2=0x5592e21e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94177fe640 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e26070 0x5592e21e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.353+0000 7f94177fe640 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e20fbc40 con 0x5592e1dfd1b0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f94177fe640 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1dfd1b0 0x5592e21d8d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9408005ab0 tx=0x7f940803fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3ed6442fd16c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f93f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:56.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f93f6ffd640 1 -- 172.21.15.31:0/1789180814 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408055060 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9408043070 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f93f6ffd640 1 -- 172.21.15.31:0/1789180814 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9408005240 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f93f6ffd640 1 -- 172.21.15.31:0/1789180814 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9408035050 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f94244fd740 10 client.25711 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592e21e7af0 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.354+0000 7f93f6ffd640 1 -- 172.21.15.31:0/1789180814 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9408043070 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f93f6ffd640 1 -- 172.21.15.31:0/1789180814 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f94080035f0 con 0x5592e1dfd1b0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f94244fd740 10 client.25711 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f94244fd740 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f94244fd740 20 client.25711 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f94244fd740 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1de49f0 0x5592e21eae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f94244fd740 4 client.25711 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e2145da0 con 0x5592e1de49f0 2024-09-21T02:54:56.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.355+0000 7f9416ffd640 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1de49f0 0x5592e21eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.356+0000 7f93f6ffd640 10 client.25711 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.356+0000 7f9416ffd640 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1de49f0 0x5592e21eae50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307d5dca2b3c2f6d server_cookie=ad66d1263cc1a60a in_seq=0 out_seq=0 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.356+0000 7f93f6ffd640 1 -- 172.21.15.31:0/1789180814 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5592e2145da0 con 0x5592e1de49f0 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.356+0000 7f93f6ffd640 10 client.25711 handle_command_reply: tid=0 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.356+0000 7f94244fd740 1 client.25711 shutdown 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1dfd1b0 msgr2=0x5592e21d8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1dfd1b0 0x5592e21d8d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9408005ab0 tx=0x7f940803fc00 comp rx=0 tx=0).stop 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 shutdown_connections 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1de49f0 0x5592e21eae50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e26070 0x5592e21e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f93f6ffd640 0 client.25711 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1e05390 0x5592e21d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 --2- 172.21.15.31:0/1789180814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1dfd1b0 0x5592e21d8d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 >> 172.21.15.31:0/1789180814 conn(0x5592e1e1a460 msgr2=0x5592e21d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:56.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 shutdown_connections 2024-09-21T02:54:56.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 1 -- 172.21.15.31:0/1789180814 wait complete. 2024-09-21T02:54:56.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.357+0000 7f94244fd740 20 client.25711 trim_cache size 0 max 16384 2024-09-21T02:54:56.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.375+0000 7f94244fd740 1 Processor -- start 2024-09-21T02:54:56.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.375+0000 7f94244fd740 1 -- start start 2024-09-21T02:54:56.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94244fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1de49f0 0x5592e21cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94244fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94244fd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 0x5592e21da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94244fd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ad440 con 0x5592e1e05390 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94244fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94181aef80 con 0x5592e1dfd1b0 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94244fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94181adb10 con 0x5592e1de49f0 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 0x5592e21da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 0x5592e21da970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45064/0 (socket says 172.21.15.31:45064) 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 -- 172.21.15.31:0/1871722256 learned_addr learned my addr 172.21.15.31:0/1871722256 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f94177fe640 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1de49f0 0x5592e21cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9416ffd640 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 -- 172.21.15.31:0/1871722256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1de49f0 msgr2=0x5592e21cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1de49f0 0x5592e21cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 -- 172.21.15.31:0/1871722256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 msgr2=0x5592e21e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.376+0000 7f9417fff640 1 -- 172.21.15.31:0/1871722256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e21ce470 con 0x5592e1e05390 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.377+0000 7f9417fff640 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 0x5592e21da970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9410011ba0 tx=0x7f941001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffcb0a8ed72e47f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.377+0000 7f93fd7fa640 1 -- 172.21.15.31:0/1871722256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94100140d0 con 0x5592e1e05390 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.377+0000 7f93fd7fa640 1 -- 172.21.15.31:0/1871722256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9410003e20 con 0x5592e1e05390 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.377+0000 7f93fd7fa640 1 -- 172.21.15.31:0/1871722256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94100041b0 con 0x5592e1e05390 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 -- 172.21.15.31:0/1871722256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 msgr2=0x5592e21da970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 0x5592e21da970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9410011ba0 tx=0x7f941001c000 comp rx=0 tx=0).stop 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 -- 172.21.15.31:0/1871722256 shutdown_connections 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1e05390 0x5592e21da970 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 --2- 172.21.15.31:0/1871722256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1de49f0 0x5592e21cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 -- 172.21.15.31:0/1871722256 >> 172.21.15.31:0/1871722256 conn(0x5592e1e1a460 msgr2=0x5592e21da260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:56.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 -- 172.21.15.31:0/1871722256 shutdown_connections 2024-09-21T02:54:56.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 -- 172.21.15.31:0/1871722256 wait complete. 2024-09-21T02:54:56.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.378+0000 7f94244fd740 1 -- start start 2024-09-21T02:54:56.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 Processor -- start 2024-09-21T02:54:56.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1de49f0 0x5592e21ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21ec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e05390 0x5592e21ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ae8b0 con 0x5592e1de49f0 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ae1e0 con 0x5592e1dfd1b0 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94181ab420 con 0x5592e1e05390 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94244fd740 10 client.0 _ll_register_callbacks cb 0x7f9410009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1de49f0 0x5592e21ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f9416ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21ec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1de49f0 0x5592e21ebe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45066/0 (socket says 172.21.15.31:45066) 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.379+0000 7f94177fe640 1 -- 172.21.15.31:0/80913963 learned_addr learned my addr 172.21.15.31:0/80913963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9417fff640 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e05390 0x5592e21ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9416ffd640 1 -- 172.21.15.31:0/80913963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e05390 msgr2=0x5592e21ec8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9416ffd640 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e05390 0x5592e21ec8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9416ffd640 1 -- 172.21.15.31:0/80913963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1de49f0 msgr2=0x5592e21ebe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:56.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9416ffd640 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1de49f0 0x5592e21ebe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9416ffd640 1 -- 172.21.15.31:0/80913963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e22545d0 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f9416ffd640 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21ec380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f940c0e48e0 tx=0x7f940c00df10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f72ac5c2d593c309 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f93f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f93f6ffd640 1 -- 172.21.15.31:0/80913963 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c021040 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f93f6ffd640 1 -- 172.21.15.31:0/80913963 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f940c011030 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.380+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410003e20 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f93f6ffd640 1 -- 172.21.15.31:0/80913963 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f940c011230 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 10 client.25658 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592e2255890 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f93f6ffd640 1 -- 172.21.15.31:0/80913963 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9410003e20 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f93f6ffd640 1 -- 172.21.15.31:0/80913963 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f940c0a8980 con 0x5592e1dfd1b0 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 10 client.25658 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:54:56.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 20 client.25658 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:56.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1e26070 0x5592e2258000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:56.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 4 client.25658 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:54:56.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.381+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592e21d48f0 con 0x5592e1e26070 2024-09-21T02:54:56.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.382+0000 7f94177fe640 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1e26070 0x5592e2258000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:56.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.382+0000 7f93f6ffd640 10 client.25658 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:56.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.382+0000 7f94177fe640 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1e26070 0x5592e2258000 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394e10e70337d5da server_cookie=58bb749f12de7493 in_seq=0 out_seq=0 2024-09-21T02:54:56.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.382+0000 7f93f6ffd640 1 -- 172.21.15.31:0/80913963 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x5592e21d48f0 con 0x5592e1e26070 2024-09-21T02:54:56.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.382+0000 7f93f6ffd640 10 client.25658 handle_command_reply: tid=0 2024-09-21T02:54:56.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.382+0000 7f94244fd740 1 client.25658 shutdown 2024-09-21T02:54:56.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 msgr2=0x5592e21ec380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21ec380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f940c0e48e0 tx=0x7f940c00df10 comp rx=0 tx=0).stop 2024-09-21T02:54:56.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 shutdown_connections 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5592e1e26070 0x5592e2258000 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5592e1e05390 0x5592e21ec8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592e1dfd1b0 0x5592e21ec380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f93f6ffd640 0 client.25658 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 --2- 172.21.15.31:0/80913963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5592e1de49f0 0x5592e21ebe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 >> 172.21.15.31:0/80913963 conn(0x5592e1e1a460 msgr2=0x5592e21cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 shutdown_connections 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 1 -- 172.21.15.31:0/80913963 wait complete. 2024-09-21T02:54:56.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.383+0000 7f94244fd740 20 client.25658 trim_cache size 0 max 16384 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:54:56.389 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:56.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:54:56.391 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:54:56.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94000b6610 msgr2=0x7f94000b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94000b6610 0x7f94000b8b20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9410005fd0 tx=0x7f9410015a60 comp rx=0 tx=0).stop 2024-09-21T02:54:56.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 msgr2=0x7f941819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:56.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f941819a9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f940c00d770 tx=0x7f940c00dc30 comp rx=0 tx=0).stop 2024-09-21T02:54:56.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 shutdown_connections 2024-09-21T02:54:56.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f94000b6610 0x7f94000b8b20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9418108d20 0x7f941819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94181083d0 0x7f941819a9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 --2- 172.21.15.31:0/264778821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9418102ae0 0x7f941819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:56.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 >> 172.21.15.31:0/264778821 conn(0x7f9418079760 msgr2=0x7f941806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:56.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 shutdown_connections 2024-09-21T02:54:56.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:56.384+0000 7f941df89640 1 -- 172.21.15.31:0/264778821 wait complete. 2024-09-21T02:54:56.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:56 smithi031 ceph-mon[31209]: pgmap v386: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 20 op/s 2024-09-21T02:54:56.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:56 smithi165 ceph-mon[34818]: pgmap v386: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 20 op/s 2024-09-21T02:54:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:56 smithi178 ceph-mon[34759]: pgmap v386: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 20 op/s 2024-09-21T02:54:57.107 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:54:57.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:54:57.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 -- 172.21.15.31:0/3964961011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 msgr2=0x7f4994103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 --2- 172.21.15.31:0/3964961011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f4994103f90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4984009900 tx=0x7f498402a410 comp rx=0 tx=0).stop 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 -- 172.21.15.31:0/3964961011 shutdown_connections 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 --2- 172.21.15.31:0/3964961011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49941094c0 0x7f49941098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 --2- 172.21.15.31:0/3964961011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4994109e10 0x7f4994110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 --2- 172.21.15.31:0/3964961011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f4994103f90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 -- 172.21.15.31:0/3964961011 >> 172.21.15.31:0/3964961011 conn(0x7f49940ff610 msgr2=0x7f4994101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 -- 172.21.15.31:0/3964961011 shutdown_connections 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.900+0000 7f499a0ab640 1 -- 172.21.15.31:0/3964961011 wait complete. 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 Processor -- start 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 -- start start 2024-09-21T02:54:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f499419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49941094c0 0x7f499419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4994109e10 0x7f499419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4994113ad0 con 0x7f4994103b80 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4994113950 con 0x7f49941094c0 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.901+0000 7f499a0ab640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4994113c50 con 0x7f4994109e10 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f499419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f499419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45094/0 (socket says 172.21.15.31:45094) 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 -- 172.21.15.31:0/4022923761 learned_addr learned my addr 172.21.15.31:0/4022923761 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f498affd640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49941094c0 0x7f499419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f4993fff640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4994109e10 0x7f499419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 -- 172.21.15.31:0/4022923761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4994109e10 msgr2=0x7f499419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4994109e10 0x7f499419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 -- 172.21.15.31:0/4022923761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49941094c0 msgr2=0x7f499419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49941094c0 0x7f499419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 -- 172.21.15.31:0/4022923761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49941a0430 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f498affd640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49941094c0 0x7f499419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49937fe640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f499419a4c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4984005940 tx=0x7f49840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.902+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498403f030 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.903+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49840044c0 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.903+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49941a06c0 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.903+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984035070 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.903+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941a6e80 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.904+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4994103f90 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.906+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4984042030 con 0x7f4994103b80 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.906+0000 7f49917fa640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49800b6680 0x7f49800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.907+0000 7f498affd640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49800b6680 0x7f49800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.907+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4984107cf0 con 0x7f4994103b80 2024-09-21T02:54:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.907+0000 7f498affd640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49800b6680 0x7f49800b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4994108f60 tx=0x7f497c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:57.915+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49840d2640 con 0x7f4994103b80 2024-09-21T02:54:58.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.148+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f499419b5f0 con 0x7f4994103b80 2024-09-21T02:54:58.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.148+0000 7f49917fa640 1 -- 172.21.15.31:0/4022923761 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f49840d7580 con 0x7f4994103b80 2024-09-21T02:54:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:54:58.150 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:54:58.150 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:54:58.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49800b6680 msgr2=0x7f49800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49800b6680 0x7f49800b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4994108f60 tx=0x7f497c005f90 comp rx=0 tx=0).stop 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 msgr2=0x7f499419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f499419a4c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4984005940 tx=0x7f49840040c0 comp rx=0 tx=0).stop 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 shutdown_connections 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f49800b6680 0x7f49800b8b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4994109e10 0x7f499419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49941094c0 0x7f499419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 --2- 172.21.15.31:0/4022923761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4994103b80 0x7f499419a4c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 >> 172.21.15.31:0/4022923761 conn(0x7f49940ff610 msgr2=0x7f4994101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 shutdown_connections 2024-09-21T02:54:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:58.151+0000 7f499a0ab640 1 -- 172.21.15.31:0/4022923761 wait complete. 2024-09-21T02:54:58.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:58 smithi031 ceph-mon[31209]: pgmap v387: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-21T02:54:58.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:54:58 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4022923761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:58.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:58 smithi165 ceph-mon[34818]: pgmap v387: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-21T02:54:58.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:54:58 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/4022923761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:58.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:54:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:58 smithi178 ceph-mon[34759]: pgmap v387: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-21T02:54:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:54:58 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/4022923761' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:54:59.246 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:54:59.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 -- 172.21.15.31:0/680978453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 msgr2=0x7f9b3c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/680978453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c103f90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009900 tx=0x7f9b2c02a410 comp rx=0 tx=0).stop 2024-09-21T02:54:59.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 -- 172.21.15.31:0/680978453 shutdown_connections 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/680978453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 0x7f9b3c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/680978453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c109e10 0x7f9b3c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/680978453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c103f90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 -- 172.21.15.31:0/680978453 >> 172.21.15.31:0/680978453 conn(0x7f9b3c0ff610 msgr2=0x7f9b3c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 -- 172.21.15.31:0/680978453 shutdown_connections 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.804+0000 7f9b40bdc640 1 -- 172.21.15.31:0/680978453 wait complete. 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.805+0000 7f9b40bdc640 1 Processor -- start 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.805+0000 7f9b40bdc640 1 -- start start 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b40bdc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b40bdc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 0x7f9b3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b40bdc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c109e10 0x7f9b3c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b40bdc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c113ad0 con 0x7f9b3c103b80 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b40bdc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c113950 con 0x7f9b3c109e10 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b40bdc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c113c50 con 0x7f9b3c1094c0 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 0x7f9b3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54190/0 (socket says 172.21.15.31:54190) 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b3a59c640 1 -- 172.21.15.31:0/3931800044 learned_addr learned my addr 172.21.15.31:0/3931800044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c109e10 0x7f9b3c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b33fff640 1 -- 172.21.15.31:0/3931800044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c109e10 msgr2=0x7f9b3c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c109e10 0x7f9b3c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b33fff640 1 -- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 msgr2=0x7f9b3c077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.806+0000 7f9b33fff640 1 -- 172.21.15.31:0/3931800044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c076470 con 0x7f9b3c1094c0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.807+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 0x7f9b3c075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c06aa90 tx=0x7f9b2400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.807+0000 7f9b337fe640 1 -- 172.21.15.31:0/3931800044 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cc40 con 0x7f9b3c1094c0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.807+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c1ace40 con 0x7f9b3c1094c0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.807+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c1ad2f0 con 0x7f9b3c1094c0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.807+0000 7f9b337fe640 1 -- 172.21.15.31:0/3931800044 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2400cde0 con 0x7f9b3c1094c0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.807+0000 7f9b337fe640 1 -- 172.21.15.31:0/3931800044 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24018640 con 0x7f9b3c1094c0 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.810+0000 7f9b47150740 1 Processor -- start 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.810+0000 7f9b47150740 1 -- start start 2024-09-21T02:54:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.810+0000 7f9b337fe640 1 -- 172.21.15.31:0/3931800044 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9b24018800 con 0x7f9b3c1094c0 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.810+0000 7f9b47150740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbe56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.810+0000 7f9b47150740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbe5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b47150740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbe6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b47150740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c37bb5bea0 con 0x55c37b8151b0 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b47150740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c37bb4b2d0 con 0x55c37b83e070 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b47150740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c37bb4f0e0 con 0x55c37b81d390 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbe5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbe5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54198/0 (socket says 172.21.15.31:54198) 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 -- 172.21.15.31:0/4178719808 learned_addr learned my addr 172.21.15.31:0/4178719808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbe56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b33fff640 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbe6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 -- 172.21.15.31:0/4178719808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 msgr2=0x55c37bbe6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbe6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 -- 172.21.15.31:0/4178719808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 msgr2=0x55c37bbe56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbe56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 -- 172.21.15.31:0/4178719808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37b8a9650 con 0x55c37b8151b0 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.811+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbe5be0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000c00 tx=0x7f9b2c031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46df4196bb568bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.812+0000 7f9b337fe640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9b280b6680 0x7f9b280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.812+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9b280b6680 0x7f9b280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.812+0000 7f9b30ff9640 1 -- 172.21.15.31:0/4178719808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c040070 con 0x55c37b8151b0 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.812+0000 7f9b337fe640 1 -- 172.21.15.31:0/3931800044 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9b240e7a80 con 0x7f9b3c1094c0 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.812+0000 7f9b30ff9640 1 -- 172.21.15.31:0/4178719808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2c004030 con 0x55c37b8151b0 2024-09-21T02:54:59.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.812+0000 7f9b30ff9640 1 -- 172.21.15.31:0/4178719808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c035040 con 0x55c37b8151b0 2024-09-21T02:54:59.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 -- 172.21.15.31:0/4178719808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 msgr2=0x55c37bbe5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9b280b6680 0x7f9b280b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0777d0 tx=0x7f9b34009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:59.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbe5be0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000c00 tx=0x7f9b2c031b10 comp rx=0 tx=0).stop 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 -- 172.21.15.31:0/4178719808 shutdown_connections 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbe6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbe5be0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 --2- 172.21.15.31:0/4178719808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbe56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.813+0000 7f9b47150740 1 -- 172.21.15.31:0/4178719808 >> 172.21.15.31:0/4178719808 conn(0x55c37b832460 msgr2=0x55c37bbe3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 -- 172.21.15.31:0/4178719808 shutdown_connections 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 -- 172.21.15.31:0/4178719808 wait complete. 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 -- start start 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 Processor -- start 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbf1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbfd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.814+0000 7f9b47150740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c37b9ac700 con 0x55c37b8151b0 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b47150740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c37bae6160 con 0x55c37b83e070 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b47150740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c37bbf3e20 con 0x55c37b81d390 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54204/0 (socket says 172.21.15.31:54204) 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b3a59c640 1 -- 172.21.15.31:0/3955159822 learned_addr learned my addr 172.21.15.31:0/3955159822 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b47150740 10 client.0 _ll_register_callbacks cb 0x55c37baff9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbf1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbfd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 -- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 msgr2=0x55c37bbfd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbfd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 -- 172.21.15.31:0/3955159822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 msgr2=0x55c37bbf0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 -- 172.21.15.31:0/3955159822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c37bb13c90 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b33fff640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbf1300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b240e51f0 tx=0x7f9b240eb190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82321ffe48de8a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.815+0000 7f9b21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b21ffb640 1 -- 172.21.15.31:0/3955159822 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240ef040 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2c004030 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b21ffb640 1 -- 172.21.15.31:0/3955159822 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9b24010430 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b21ffb640 1 -- 172.21.15.31:0/3955159822 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b240ad770 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 10 client.25735 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c37bbffb60 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b21ffb640 1 -- 172.21.15.31:0/3955159822 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2c004030 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b21ffb640 1 -- 172.21.15.31:0/3955159822 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9b24010630 con 0x55c37b81d390 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 10 client.25735 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 10 client.25735 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:54:59.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 20 client.25735 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7fc9f0 0x55c37bc03000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 4 client.25735 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.816+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c37bb5ddf0 con 0x55c37b7fc9f0 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.817+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7fc9f0 0x55c37bc03000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.817+0000 7f9b21ffb640 10 client.25735 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.817+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7fc9f0 0x55c37bc03000 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea0842886072d727 server_cookie=6f9bd79e82642a0b in_seq=0 out_seq=0 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b21ffb640 1 -- 172.21.15.31:0/3955159822 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c37bb5ddf0 con 0x55c37b7fc9f0 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b21ffb640 10 client.25735 handle_command_reply: tid=0 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 client.25735 shutdown 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 msgr2=0x55c37bbf1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbf1300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b240e51f0 tx=0x7f9b240eb190 comp rx=0 tx=0).stop 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 shutdown_connections 2024-09-21T02:54:59.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7fc9f0 0x55c37bc03000 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbf1300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbfd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 --2- 172.21.15.31:0/3955159822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b21ffb640 0 client.25735 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.818+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 >> 172.21.15.31:0/3955159822 conn(0x55c37b832460 msgr2=0x55c37bbe8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.819+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 shutdown_connections 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.819+0000 7f9b47150740 1 -- 172.21.15.31:0/3955159822 wait complete. 2024-09-21T02:54:59.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.819+0000 7f9b47150740 20 client.25735 trim_cache size 0 max 16384 2024-09-21T02:54:59.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.832+0000 7f9b47150740 1 Processor -- start 2024-09-21T02:54:59.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.832+0000 7f9b47150740 1 -- start start 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.832+0000 7f9b47150740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.832+0000 7f9b47150740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbedfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.832+0000 7f9b47150740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b47150740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1b5d50 con 0x55c37b8151b0 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b47150740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1b7890 con 0x55c37b83e070 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b47150740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1b6420 con 0x55c37b81d390 2024-09-21T02:54:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54206/0 (socket says 172.21.15.31:54206) 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbee4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42532/0 (socket says 172.21.15.31:42532) 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 -- 172.21.15.31:0/646565317 learned_addr learned my addr 172.21.15.31:0/646565317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b33fff640 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbedfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 -- 172.21.15.31:0/646565317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 msgr2=0x55c37bbedfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbedfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 -- 172.21.15.31:0/646565317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 msgr2=0x55c37bbee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 -- 172.21.15.31:0/646565317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37bbf2080 con 0x55c37b8151b0 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:54:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.833+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf29f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009a20 tx=0x7f9b2c005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b341df4889e950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.834+0000 7f9b23fff640 1 -- 172.21.15.31:0/646565317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c040040 con 0x55c37b8151b0 2024-09-21T02:54:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.834+0000 7f9b23fff640 1 -- 172.21.15.31:0/646565317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2c0053d0 con 0x55c37b8151b0 2024-09-21T02:54:59.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.834+0000 7f9b23fff640 1 -- 172.21.15.31:0/646565317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c035040 con 0x55c37b8151b0 2024-09-21T02:54:59.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 -- 172.21.15.31:0/646565317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 msgr2=0x55c37bbf29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf29f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009a20 tx=0x7f9b2c005160 comp rx=0 tx=0).stop 2024-09-21T02:54:59.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 -- 172.21.15.31:0/646565317 shutdown_connections 2024-09-21T02:54:59.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b81d390 0x55c37bbedfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 --2- 172.21.15.31:0/646565317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b8151b0 0x55c37bbf29f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 -- 172.21.15.31:0/646565317 >> 172.21.15.31:0/646565317 conn(0x55c37b832460 msgr2=0x55c37bbf22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 -- 172.21.15.31:0/646565317 shutdown_connections 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.835+0000 7f9b47150740 1 -- 172.21.15.31:0/646565317 wait complete. 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.836+0000 7f9b47150740 1 -- start start 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.836+0000 7f9b47150740 1 Processor -- start 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b8151b0 0x55c37bbf9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 0x55c37bbf6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbf6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1b71c0 con 0x55c37b81d390 2024-09-21T02:54:59.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1b6af0 con 0x55c37b83e070 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c1b3d30 con 0x55c37b8151b0 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b47150740 10 client.0 _ll_register_callbacks cb 0x7f9b240056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 0x55c37bbf6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 0x55c37bbf6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54208/0 (socket says 172.21.15.31:54208) 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 -- 172.21.15.31:0/2444452426 learned_addr learned my addr 172.21.15.31:0/2444452426 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b8151b0 0x55c37bbf9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbf6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 -- 172.21.15.31:0/2444452426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b8151b0 msgr2=0x55c37bbf9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b8151b0 0x55c37bbf9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 -- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 msgr2=0x55c37bbf6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbf6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 -- 172.21.15.31:0/2444452426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c37bbf74e0 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b3ad9d640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbf6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b33fff640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 0x55c37bbf6a60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9b24023040 tx=0x7f9b240106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b95f45255608df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.837+0000 7f9b21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b21ffb640 1 -- 172.21.15.31:0/2444452426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240a9110 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b21ffb640 1 -- 172.21.15.31:0/2444452426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9b240c3aa0 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2c0053d0 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b21ffb640 1 -- 172.21.15.31:0/2444452426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b240ad110 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 10 client.16062 fetch_fsmap learned FSMap version 27 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c37bc6d610 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b21ffb640 1 -- 172.21.15.31:0/2444452426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2c0053d0 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b21ffb640 1 -- 172.21.15.31:0/2444452426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9b240ad9a0 con 0x55c37b81d390 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 10 client.16062 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:54:59.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 20 client.16062 populate_metadata read hostname 'smithi031' 2024-09-21T02:54:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7d1da0 0x55c37bc6fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 4 client.16062 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:54:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.838+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c37bb598a0 con 0x55c37b7d1da0 2024-09-21T02:54:59.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.839+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7d1da0 0x55c37bc6fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.839+0000 7f9b3a59c640 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7d1da0 0x55c37bc6fc10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6aec9c23fc7a24d server_cookie=de7e64907e0277fb in_seq=0 out_seq=0 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.839+0000 7f9b21ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.839+0000 7f9b21ffb640 1 -- 172.21.15.31:0/2444452426 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c37bb598a0 con 0x55c37b7d1da0 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.839+0000 7f9b21ffb640 10 client.16062 handle_command_reply: tid=0 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.839+0000 7f9b47150740 1 client.16062 shutdown 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 msgr2=0x55c37bbf6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 0x55c37bbf6a60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9b24023040 tx=0x7f9b240106e0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 shutdown_connections 2024-09-21T02:54:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c37b83e070 0x55c37bbf6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c37b81d390 0x55c37bbf6a60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55c37b7d1da0 0x55c37bc6fc10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 --2- 172.21.15.31:0/2444452426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c37b8151b0 0x55c37bbf9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b21ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 >> 172.21.15.31:0/2444452426 conn(0x55c37b832460 msgr2=0x55c37bbf2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 shutdown_connections 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 1 -- 172.21.15.31:0/2444452426 wait complete. 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.840+0000 7f9b47150740 20 client.16062 trim_cache size 0 max 16384 2024-09-21T02:54:59.846 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9b280b6680 msgr2=0x7f9b280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9b280b6680 0x7f9b280b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0777d0 tx=0x7f9b34009500 comp rx=0 tx=0).stop 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 msgr2=0x7f9b3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 0x7f9b3c075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c06aa90 tx=0x7f9b2400ee50 comp rx=0 tx=0).stop 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 shutdown_connections 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9b280b6680 0x7f9b280b8b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b3c1094c0 0x7f9b3c075710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c109e10 0x7f9b3c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 --2- 172.21.15.31:0/3931800044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b3c103b80 0x7f9b3c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 >> 172.21.15.31:0/3931800044 conn(0x7f9b3c0ff610 msgr2=0x7f9b3c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 shutdown_connections 2024-09-21T02:54:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:54:59.841+0000 7f9b40bdc640 1 -- 172.21.15.31:0/3931800044 wait complete. 2024-09-21T02:55:00.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:55:00.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:00 smithi031 ceph-mon[31209]: pgmap v388: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-21T02:55:00.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:00 smithi165 ceph-mon[34818]: pgmap v388: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-21T02:55:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:00.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:00 smithi178 ceph-mon[34759]: pgmap v388: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 -- 172.21.15.31:0/3264524100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 msgr2=0x7f16341098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 --2- 172.21.15.31:0/3264524100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f16341098c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1628009900 tx=0x7f162802a410 comp rx=0 tx=0).stop 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 -- 172.21.15.31:0/3264524100 shutdown_connections 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 --2- 172.21.15.31:0/3264524100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634109e00 0x7f1634110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 --2- 172.21.15.31:0/3264524100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f16341098c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 --2- 172.21.15.31:0/3264524100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634103bc0 0x7f1634103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 -- 172.21.15.31:0/3264524100 >> 172.21.15.31:0/3264524100 conn(0x7f163407b910 msgr2=0x7f163407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 -- 172.21.15.31:0/3264524100 shutdown_connections 2024-09-21T02:55:01.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.484+0000 7f1638df5640 1 -- 172.21.15.31:0/3264524100 wait complete. 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.485+0000 7f1638df5640 1 Processor -- start 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.485+0000 7f1638df5640 1 -- start start 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1638df5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 0x7f163419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1638df5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f163419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1638df5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634109e00 0x7f163419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1638df5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1634113ac0 con 0x7f16341094b0 2024-09-21T02:55:01.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1638df5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1634113940 con 0x7f1634103bc0 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1638df5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1634113c40 con 0x7f1634109e00 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f163419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 0x7f163419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634109e00 0x7f163419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 0x7f163419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42560/0 (socket says 172.21.15.31:42560) 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 -- 172.21.15.31:0/2420503776 learned_addr learned my addr 172.21.15.31:0/2420503776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 -- 172.21.15.31:0/2420503776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634109e00 msgr2=0x7f163419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634109e00 0x7f163419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 -- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 msgr2=0x7f163419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f163419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f163259c640 1 -- 172.21.15.31:0/2420503776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16341a03e0 con 0x7f1634103bc0 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1632d9d640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634109e00 0x7f163419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.486+0000 7f1631d9b640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f163419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.487+0000 7f163259c640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 0x7f163419a470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f162400d770 tx=0x7f162400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.487+0000 7f16237fe640 1 -- 172.21.15.31:0/2420503776 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624017070 con 0x7f1634103bc0 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.487+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16341a06d0 con 0x7f1634103bc0 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.487+0000 7f16237fe640 1 -- 172.21.15.31:0/2420503776 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16240054d0 con 0x7f1634103bc0 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.487+0000 7f16237fe640 1 -- 172.21.15.31:0/2420503776 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624004c30 con 0x7f1634103bc0 2024-09-21T02:55:01.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.487+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16340755f0 con 0x7f1634103bc0 2024-09-21T02:55:01.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 Processor -- start 2024-09-21T02:55:01.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 -- start start 2024-09-21T02:55:01.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f16237fe640 1 -- 172.21.15.31:0/2420503776 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1624010460 con 0x7f1634103bc0 2024-09-21T02:55:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b677865ea0 con 0x55b677527390 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.490+0000 7f163f369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6778552d0 con 0x55b677548070 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f163f369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b6778590e0 con 0x55b67751f1b0 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42576/0 (socket says 172.21.15.31:42576) 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 -- 172.21.15.31:0/3433136867 learned_addr learned my addr 172.21.15.31:0/3433136867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54232/0 (socket says 172.21.15.31:54232) 2024-09-21T02:55:01.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f163259c640 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 -- 172.21.15.31:0/3433136867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 msgr2=0x55b6778efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778efbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 -- 172.21.15.31:0/3433136867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 msgr2=0x55b6778f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f16237fe640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f161c0b6680 0x7f161c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.491+0000 7f1632d9d640 1 -- 172.21.15.31:0/3433136867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6775b3650 con 0x55b677548070 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f1631d9b640 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f1632d9d640 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778ef6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f162c010a80 tx=0x7f162c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10878de82509531 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f16237fe640 1 -- 172.21.15.31:0/2420503776 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f16240e70f0 con 0x7f1634103bc0 2024-09-21T02:55:01.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f163259c640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f161c0b6680 0x7f161c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f1620ff9640 1 -- 172.21.15.31:0/3433136867 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c003c30 con 0x55b677548070 2024-09-21T02:55:01.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f1620ff9640 1 -- 172.21.15.31:0/3433136867 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c01f070 con 0x55b677548070 2024-09-21T02:55:01.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.492+0000 7f1620ff9640 1 -- 172.21.15.31:0/3433136867 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c004650 con 0x55b677548070 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 -- 172.21.15.31:0/3433136867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 msgr2=0x55b6778ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778ef6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f162c010a80 tx=0x7f162c009500 comp rx=0 tx=0).stop 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163259c640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f161c0b6680 0x7f161c0b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16240e49b0 tx=0x7f16240eacd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 -- 172.21.15.31:0/3433136867 shutdown_connections 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 --2- 172.21.15.31:0/3433136867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778ef6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 -- 172.21.15.31:0/3433136867 >> 172.21.15.31:0/3433136867 conn(0x55b67753c460 msgr2=0x55b6778ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 -- 172.21.15.31:0/3433136867 shutdown_connections 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.493+0000 7f163f369740 1 -- 172.21.15.31:0/3433136867 wait complete. 2024-09-21T02:55:01.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 -- start start 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 Processor -- start 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b6776b6700 con 0x55b677527390 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6777f0160 con 0x55b677548070 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b6778fde80 con 0x55b67751f1b0 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f163f369740 10 client.0 _ll_register_callbacks cb 0x55b6778099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778fb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54242/0 (socket says 172.21.15.31:54242) 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778facd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60286/0 (socket says 172.21.15.31:60286) 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.494+0000 7f1631d9b640 1 -- 172.21.15.31:0/1202782323 learned_addr learned my addr 172.21.15.31:0/1202782323 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:01.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1632d9d640 1 -- 172.21.15.31:0/1202782323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 msgr2=0x55b6778fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1632d9d640 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1632d9d640 1 -- 172.21.15.31:0/1202782323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 msgr2=0x55b6778fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1632d9d640 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1632d9d640 1 -- 172.21.15.31:0/1202782323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67781dc90 con 0x55b67751f1b0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1631d9b640 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1632d9d640 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778facd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f162c009fd0 tx=0x7f162c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffaf5ceb254136fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.495+0000 7f1619ffb640 1 -- 172.21.15.31:0/1202782323 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c004a40 con 0x55b67751f1b0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f1619ffb640 1 -- 172.21.15.31:0/1202782323 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f162c004be0 con 0x55b67751f1b0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f162c01f070 con 0x55b67751f1b0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f1619ffb640 1 -- 172.21.15.31:0/1202782323 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f162c036050 con 0x55b67751f1b0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 10 client.25753 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b677909b20 con 0x55b67751f1b0 2024-09-21T02:55:01.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f1619ffb640 1 -- 172.21.15.31:0/1202782323 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f162c01f070 con 0x55b67751f1b0 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f1619ffb640 1 -- 172.21.15.31:0/1202782323 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f162c025930 con 0x55b67751f1b0 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 10 client.25753 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 10 client.25753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 20 client.25753 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b6775069f0 0x55b67790cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 4 client.25753 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.496+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b677867df0 con 0x55b6775069f0 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.497+0000 7f1631d9b640 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b6775069f0 0x55b67790cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.497+0000 7f1631d9b640 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b6775069f0 0x55b67790cf90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46fb48791aba2d62 server_cookie=5d76f959e5fe887c in_seq=0 out_seq=0 2024-09-21T02:55:01.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.497+0000 7f1619ffb640 10 client.25753 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:01.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f1619ffb640 1 -- 172.21.15.31:0/1202782323 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b677867df0 con 0x55b6775069f0 2024-09-21T02:55:01.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f1619ffb640 10 client.25753 handle_command_reply: tid=0 2024-09-21T02:55:01.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 client.25753 shutdown 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 msgr2=0x55b6778facd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778facd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f162c009fd0 tx=0x7f162c010800 comp rx=0 tx=0).stop 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 shutdown_connections 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b6775069f0 0x55b67790cf90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b677548070 0x55b6778fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b6778fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f1619ffb640 0 client.25753 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 --2- 172.21.15.31:0/1202782323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b67751f1b0 0x55b6778facd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 >> 172.21.15.31:0/1202782323 conn(0x55b67753c460 msgr2=0x55b6778f2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 shutdown_connections 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 1 -- 172.21.15.31:0/1202782323 wait complete. 2024-09-21T02:55:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.498+0000 7f163f369740 20 client.25753 trim_cache size 0 max 16384 2024-09-21T02:55:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.513+0000 7f163f369740 1 Processor -- start 2024-09-21T02:55:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.513+0000 7f163f369740 1 -- start start 2024-09-21T02:55:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f163f369740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6775069f0 0x55b6778f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f163f369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f163f369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b677527390 0x55b6778ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f163f369740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f16341ad4e0 con 0x55b6775069f0 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f163f369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16341af020 con 0x55b67751f1b0 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f163f369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16341adbb0 con 0x55b677527390 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6775069f0 0x55b6778f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6775069f0 0x55b6778f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54250/0 (socket says 172.21.15.31:54250) 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42606/0 (socket says 172.21.15.31:42606) 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1632d9d640 1 -- 172.21.15.31:0/932200232 learned_addr learned my addr 172.21.15.31:0/932200232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1632d9d640 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b677527390 0x55b6778ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 -- 172.21.15.31:0/932200232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b677527390 msgr2=0x55b6778ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b677527390 0x55b6778ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 -- 172.21.15.31:0/932200232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6775069f0 msgr2=0x55b6778f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6775069f0 0x55b6778f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.514+0000 7f1631d9b640 1 -- 172.21.15.31:0/932200232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6778f2500 con 0x55b67751f1b0 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.515+0000 7f1631d9b640 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fc900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55b67790d5d0 tx=0x7f1628031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67623081215f040 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.515+0000 7f161bfff640 1 -- 172.21.15.31:0/932200232 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162804a020 con 0x55b67751f1b0 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.515+0000 7f161bfff640 1 -- 172.21.15.31:0/932200232 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16280054f0 con 0x55b67751f1b0 2024-09-21T02:55:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.515+0000 7f161bfff640 1 -- 172.21.15.31:0/932200232 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628035040 con 0x55b67751f1b0 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 -- 172.21.15.31:0/932200232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 msgr2=0x55b6778fc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fc900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55b67790d5d0 tx=0x7f1628031400 comp rx=0 tx=0).stop 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 -- 172.21.15.31:0/932200232 shutdown_connections 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b677527390 0x55b6778ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fc900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 --2- 172.21.15.31:0/932200232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6775069f0 0x55b6778f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 -- 172.21.15.31:0/932200232 >> 172.21.15.31:0/932200232 conn(0x55b67753c460 msgr2=0x55b6778fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 -- 172.21.15.31:0/932200232 shutdown_connections 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 -- 172.21.15.31:0/932200232 wait complete. 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.516+0000 7f163f369740 1 -- start start 2024-09-21T02:55:01.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 Processor -- start 2024-09-21T02:55:01.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6775069f0 0x55b6778fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b67796f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f16341ae950 con 0x55b677527390 2024-09-21T02:55:01.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16341ae280 con 0x55b67751f1b0 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16341ab4c0 con 0x55b6775069f0 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f163f369740 10 client.0 _ll_register_callbacks cb 0x7f1628009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b67796f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f1632d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b67796f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54266/0 (socket says 172.21.15.31:54266) 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f1631d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f1632d9d640 1 -- 172.21.15.31:0/251283064 learned_addr learned my addr 172.21.15.31:0/251283064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.517+0000 7f1632d9d640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6775069f0 0x55b6778fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1632d9d640 1 -- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6775069f0 msgr2=0x55b6778fe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1632d9d640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6775069f0 0x55b6778fe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1632d9d640 1 -- 172.21.15.31:0/251283064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 msgr2=0x55b6778fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1632d9d640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1632d9d640 1 -- 172.21.15.31:0/251283064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67796fe90 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1631d9b640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1632d9d640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b67796f820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f162c003f40 tx=0x7f162c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af75563d430f2772 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1619ffb640 1 -- 172.21.15.31:0/251283064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c025620 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1619ffb640 1 -- 172.21.15.31:0/251283064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f162c0257c0 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16280054f0 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f1619ffb640 1 -- 172.21.15.31:0/251283064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f162c0259c0 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f163f369740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.518+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b6779771b0 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f1619ffb640 1 -- 172.21.15.31:0/251283064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16280054f0 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f1619ffb640 1 -- 172.21.15.31:0/251283064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f162c039040 con 0x55b677527390 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f163f369740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f163f369740 10 client.16080 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:55:01.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f163f369740 20 client.16080 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f163f369740 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b677548070 0x55b6779797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f163f369740 4 client.16080 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6778638a0 con 0x55b677548070 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f1631d9b640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b677548070 0x55b6779797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f1631d9b640 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b677548070 0x55b6779797d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1db6edf5b2a60b3e server_cookie=f0c6884931248936 in_seq=0 out_seq=0 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.519+0000 7f1619ffb640 10 client.16080 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f1619ffb640 1 -- 172.21.15.31:0/251283064 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6778638a0 con 0x55b677548070 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f1619ffb640 10 client.16080 handle_command_reply: tid=0 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 client.16080 shutdown 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 msgr2=0x55b67796f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b67796f820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f162c003f40 tx=0x7f162c0096c0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 shutdown_connections 2024-09-21T02:55:01.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55b677548070 0x55b6779797d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b677527390 0x55b67796f820 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b67751f1b0 0x55b6778fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 --2- 172.21.15.31:0/251283064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6775069f0 0x55b6778fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 >> 172.21.15.31:0/251283064 conn(0x55b67753c460 msgr2=0x55b6778fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f1619ffb640 0 client.16080 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 shutdown_connections 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 1 -- 172.21.15.31:0/251283064 wait complete. 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.520+0000 7f163f369740 20 client.16080 trim_cache size 0 max 16384 2024-09-21T02:55:01.526 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f161c0b6680 msgr2=0x7f161c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f161c0b6680 0x7f161c0b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16240e49b0 tx=0x7f16240eacd0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 msgr2=0x7f163419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 0x7f163419a470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f162400d770 tx=0x7f162400dc30 comp rx=0 tx=0).stop 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 shutdown_connections 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f161c0b6680 0x7f161c0b8b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1634109e00 0x7f163419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16341094b0 0x7f163419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 --2- 172.21.15.31:0/2420503776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1634103bc0 0x7f163419a470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 >> 172.21.15.31:0/2420503776 conn(0x7f163407b910 msgr2=0x7f1634105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 shutdown_connections 2024-09-21T02:55:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:01.521+0000 7f1638df5640 1 -- 172.21.15.31:0/2420503776 wait complete. 2024-09-21T02:55:02.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:55:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:02.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:02 smithi165 ceph-mon[34818]: pgmap v389: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-21T02:55:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:02 smithi031 ceph-mon[31209]: pgmap v389: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-21T02:55:02.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:02 smithi178 ceph-mon[34759]: pgmap v389: 97 pgs: 97 active+clean; 259 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-21T02:55:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 -- 172.21.15.31:0/1492737356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294108cf0 msgr2=0x7f329410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 --2- 172.21.15.31:0/1492737356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294108cf0 0x7f329410f520 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f328c009ca0 tx=0x7f328c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 -- 172.21.15.31:0/1492737356 shutdown_connections 2024-09-21T02:55:03.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 --2- 172.21.15.31:0/1492737356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32941083a0 0x7f32941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 --2- 172.21.15.31:0/1492737356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294108cf0 0x7f329410f520 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 --2- 172.21.15.31:0/1492737356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3294102a60 0x7f3294102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 -- 172.21.15.31:0/1492737356 >> 172.21.15.31:0/1492737356 conn(0x7f3294079730 msgr2=0x7f3294079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 -- 172.21.15.31:0/1492737356 shutdown_connections 2024-09-21T02:55:03.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.075+0000 7f329914c640 1 -- 172.21.15.31:0/1492737356 wait complete. 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.076+0000 7f329914c640 1 Processor -- start 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 -- start start 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 0x7f32940ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32941083a0 0x7f32940ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3294108cf0 0x7f32941000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3294112960 con 0x7f3294102a60 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32941127e0 con 0x7f32941083a0 2024-09-21T02:55:03.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329914c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3294112ae0 con 0x7f3294108cf0 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 0x7f32940ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 0x7f32940ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54276/0 (socket says 172.21.15.31:54276) 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329359e640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3294108cf0 0x7f32941000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f3292d9d640 1 -- 172.21.15.31:0/1071298381 learned_addr learned my addr 172.21.15.31:0/1071298381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.077+0000 7f329259c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32941083a0 0x7f32940ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f3292d9d640 1 -- 172.21.15.31:0/1071298381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3294108cf0 msgr2=0x7f32941000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f3292d9d640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3294108cf0 0x7f32941000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f3292d9d640 1 -- 172.21.15.31:0/1071298381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32941083a0 msgr2=0x7f32940ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f3292d9d640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32941083a0 0x7f32940ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f3292d9d640 1 -- 172.21.15.31:0/1071298381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32941a8760 con 0x7f3294102a60 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f329259c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32941083a0 0x7f32940ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f329359e640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3294108cf0 0x7f32941000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:03.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f3292d9d640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 0x7f32940ff670 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f328400e9f0 tx=0x7f328400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f328bfff640 1 -- 172.21.15.31:0/1071298381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400cd80 con 0x7f3294102a60 2024-09-21T02:55:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32941a8a50 con 0x7f3294102a60 2024-09-21T02:55:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32941a8fe0 con 0x7f3294102a60 2024-09-21T02:55:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.078+0000 7f328bfff640 1 -- 172.21.15.31:0/1071298381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32840108e0 con 0x7f3294102a60 2024-09-21T02:55:03.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.079+0000 7f328bfff640 1 -- 172.21.15.31:0/1071298381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284018980 con 0x7f3294102a60 2024-09-21T02:55:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.081+0000 7f329f6c0740 1 Processor -- start 2024-09-21T02:55:03.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.081+0000 7f329f6c0740 1 -- start start 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f328bfff640 1 -- 172.21.15.31:0/1071298381 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3284018b20 con 0x7f3294102a60 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179bf070 0x558f17d666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179961b0 0x558f17d66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329f6c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558f17cdcea0 con 0x558f1799e390 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329f6c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f17ccc2d0 con 0x558f179bf070 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329f6c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f17cd00e0 con 0x558f179961b0 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179961b0 0x558f17d66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329359e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179bf070 0x558f17d666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d67120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54286/0 (socket says 172.21.15.31:54286) 2024-09-21T02:55:03.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.082+0000 7f329259c640 1 -- 172.21.15.31:0/3766981276 learned_addr learned my addr 172.21.15.31:0/3766981276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f329259c640 1 -- 172.21.15.31:0/3766981276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179961b0 msgr2=0x558f17d66be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f329259c640 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179961b0 0x558f17d66be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f329259c640 1 -- 172.21.15.31:0/3766981276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179bf070 msgr2=0x558f17d666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f329259c640 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179bf070 0x558f17d666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f329259c640 1 -- 172.21.15.31:0/3766981276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f17a2a650 con 0x558f1799e390 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f3292d9d640 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179961b0 0x558f17d66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f329259c640 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d67120 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f327c009fd0 tx=0x7f327c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1035a4e922e538a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f32897fa640 1 -- 172.21.15.31:0/3766981276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c004310 con 0x558f1799e390 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f32897fa640 1 -- 172.21.15.31:0/3766981276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c0044b0 con 0x558f1799e390 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f32897fa640 1 -- 172.21.15.31:0/3766981276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c0035f0 con 0x558f1799e390 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.083+0000 7f328bfff640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32800b6700 0x7f32800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.084+0000 7f329359e640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32800b6700 0x7f32800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.084+0000 7f328bfff640 1 -- 172.21.15.31:0/1071298381 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f32840e77e0 con 0x7f3294102a60 2024-09-21T02:55:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.084+0000 7f329f6c0740 1 -- 172.21.15.31:0/3766981276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 msgr2=0x558f17d67120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.084+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d67120 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f327c009fd0 tx=0x7f327c0118d0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.084+0000 7f329359e640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32800b6700 0x7f32800b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f328c006010 tx=0x7f328c032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 -- 172.21.15.31:0/3766981276 shutdown_connections 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179961b0 0x558f17d66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d67120 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3766981276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179bf070 0x558f17d666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 -- 172.21.15.31:0/3766981276 >> 172.21.15.31:0/3766981276 conn(0x558f179b3460 msgr2=0x558f17d64590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 -- 172.21.15.31:0/3766981276 shutdown_connections 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 -- 172.21.15.31:0/3766981276 wait complete. 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 -- start start 2024-09-21T02:55:03.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.085+0000 7f329f6c0740 1 Processor -- start 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 0x558f17d71ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1799e390 0x558f17d721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d72720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558f17b2d700 con 0x558f179961b0 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f17c67160 con 0x558f1799e390 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f17d74e80 con 0x558f179bf070 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329f6c0740 10 client.0 _ll_register_callbacks cb 0x558f17c809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:03.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 0x558f17d71ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 0x558f17d71ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54302/0 (socket says 172.21.15.31:54302) 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1799e390 0x558f17d721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 -- 172.21.15.31:0/3050624132 learned_addr learned my addr 172.21.15.31:0/3050624132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329359e640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d72720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 -- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 msgr2=0x558f17d72720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d72720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 -- 172.21.15.31:0/3050624132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1799e390 msgr2=0x558f17d721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1799e390 0x558f17d721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 -- 172.21.15.31:0/3050624132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f17c94c90 con 0x558f179961b0 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f329259c640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1799e390 0x558f17d721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f3292d9d640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 0x558f17d71ca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f32840e4fb0 tx=0x7f3284010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e434ec690d0750 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:03.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f327a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.086+0000 7f327a7fc640 1 -- 172.21.15.31:0/3050624132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32840a9d50 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.087+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c0044b0 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.087+0000 7f327a7fc640 1 -- 172.21.15.31:0/3050624132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f32840b6250 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.087+0000 7f327a7fc640 1 -- 172.21.15.31:0/3050624132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32840ace70 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.087+0000 7f329f6c0740 10 client.16092 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.087+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f17d80b30 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.087+0000 7f327a7fc640 1 -- 172.21.15.31:0/3050624132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c0044b0 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f327a7fc640 1 -- 172.21.15.31:0/3050624132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f32840ac3d0 con 0x558f179961b0 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329f6c0740 10 client.16092 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329f6c0740 10 client.16092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329f6c0740 20 client.16092 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f1797d9f0 0x558f17d84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329f6c0740 4 client.16092 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f17cdedf0 con 0x558f1797d9f0 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.088+0000 7f329259c640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f1797d9f0 0x558f17d84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.089+0000 7f327a7fc640 10 client.16092 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.089+0000 7f329259c640 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f1797d9f0 0x558f17d84140 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8e66131565737a9 server_cookie=71d44da60b1c20fd in_seq=0 out_seq=0 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f327a7fc640 1 -- 172.21.15.31:0/3050624132 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f17cdedf0 con 0x558f1797d9f0 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f327a7fc640 10 client.16092 handle_command_reply: tid=0 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 client.16092 shutdown 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 msgr2=0x558f17d71ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 0x558f17d71ca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f32840e4fb0 tx=0x7f3284010800 comp rx=0 tx=0).stop 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 shutdown_connections 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f1797d9f0 0x558f17d84140 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d72720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1799e390 0x558f17d721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 --2- 172.21.15.31:0/3050624132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179961b0 0x558f17d71ca0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 >> 172.21.15.31:0/3050624132 conn(0x558f179b3460 msgr2=0x558f17d69830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f327a7fc640 0 client.16092 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:03.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 shutdown_connections 2024-09-21T02:55:03.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.090+0000 7f329f6c0740 1 -- 172.21.15.31:0/3050624132 wait complete. 2024-09-21T02:55:03.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.091+0000 7f329f6c0740 20 client.16092 trim_cache size 0 max 16384 2024-09-21T02:55:03.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.105+0000 7f329f6c0740 1 Processor -- start 2024-09-21T02:55:03.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.105+0000 7f329f6c0740 1 -- start start 2024-09-21T02:55:03.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d63bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d64100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329f6c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f327400e7a0 con 0x558f1799e390 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329f6c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32740102e0 con 0x558f179961b0 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329f6c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f327400ee70 con 0x558f179bf070 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d63bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d63bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54312/0 (socket says 172.21.15.31:54312) 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 -- 172.21.15.31:0/1764638301 learned_addr learned my addr 172.21.15.31:0/1764638301 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329359e640 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 -- 172.21.15.31:0/1764638301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 msgr2=0x558f17d64100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d64100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 -- 172.21.15.31:0/1764638301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 msgr2=0x558f17d67de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d67de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329259c640 1 -- 172.21.15.31:0/1764638301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f17d72d40 con 0x558f1799e390 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f329359e640 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d64100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.106+0000 7f3292d9d640 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d67de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.107+0000 7f329259c640 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d63bc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f327c009fa0 tx=0x7f327c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8692ed54161e6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.107+0000 7f3288ff9640 1 -- 172.21.15.31:0/1764638301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c009b30 con 0x558f1799e390 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.107+0000 7f3288ff9640 1 -- 172.21.15.31:0/1764638301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c032020 con 0x558f1799e390 2024-09-21T02:55:03.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.107+0000 7f3288ff9640 1 -- 172.21.15.31:0/1764638301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c00f740 con 0x558f1799e390 2024-09-21T02:55:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 -- 172.21.15.31:0/1764638301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 msgr2=0x558f17d63bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d63bc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f327c009fa0 tx=0x7f327c005d30 comp rx=0 tx=0).stop 2024-09-21T02:55:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 -- 172.21.15.31:0/1764638301 shutdown_connections 2024-09-21T02:55:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f179bf070 0x558f17d64100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f1799e390 0x558f17d63bc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 --2- 172.21.15.31:0/1764638301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d67de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 -- 172.21.15.31:0/1764638301 >> 172.21.15.31:0/1764638301 conn(0x558f179b3460 msgr2=0x558f17d676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 -- 172.21.15.31:0/1764638301 shutdown_connections 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.108+0000 7f329f6c0740 1 -- 172.21.15.31:0/1764638301 wait complete. 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.109+0000 7f329f6c0740 1 -- start start 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.109+0000 7f329f6c0740 1 Processor -- start 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.109+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d75bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.109+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f1799e390 0x558f17d76170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.109+0000 7f329f6c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179bf070 0x558f17d7a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329f6c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f327400fc10 con 0x558f179bf070 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329f6c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f327400f540 con 0x558f179961b0 2024-09-21T02:55:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329f6c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f327400c780 con 0x558f1799e390 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329f6c0740 10 client.0 _ll_register_callbacks cb 0x7f328400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d75bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f1799e390 0x558f17d76170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f1799e390 0x558f17d76170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60370/0 (socket says 172.21.15.31:60370) 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329259c640 1 -- 172.21.15.31:0/968878247 learned_addr learned my addr 172.21.15.31:0/968878247 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f329359e640 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179bf070 0x558f17d7a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 -- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f1799e390 msgr2=0x558f17d76170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f1799e390 0x558f17d76170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 -- 172.21.15.31:0/968878247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179bf070 msgr2=0x558f17d7a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179bf070 0x558f17d7a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 -- 172.21.15.31:0/968878247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f17d7a6a0 con 0x558f179961b0 2024-09-21T02:55:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f3292d9d640 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d75bd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f328400c6a0 tx=0x7f3284010d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19182b811f403354 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.110+0000 7f327a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f327a7fc640 1 -- 172.21.15.31:0/968878247 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32840ad050 con 0x558f179961b0 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f327a7fc640 1 -- 172.21.15.31:0/968878247 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f32840a9050 con 0x558f179961b0 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c032020 con 0x558f179961b0 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f327a7fc640 1 -- 172.21.15.31:0/968878247 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32840ad4d0 con 0x558f179961b0 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f329f6c0740 10 client.25715 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f17dee480 con 0x558f179961b0 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f327a7fc640 1 -- 172.21.15.31:0/968878247 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c032020 con 0x558f179961b0 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f327a7fc640 1 -- 172.21.15.31:0/968878247 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f32840a9250 con 0x558f179961b0 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f329f6c0740 10 client.25715 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f329f6c0740 10 client.25715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.111+0000 7f329f6c0740 20 client.25715 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.112+0000 7f329f6c0740 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f17952da0 0x558f17df0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.112+0000 7f329f6c0740 4 client.25715 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.112+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f17cda8a0 con 0x558f17952da0 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.112+0000 7f329259c640 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f17952da0 0x558f17df0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.112+0000 7f327a7fc640 10 client.25715 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:03.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.112+0000 7f329259c640 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f17952da0 0x558f17df0bc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10ffc09fc983ba2 server_cookie=2a991b4fa93c4279 in_seq=0 out_seq=0 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f327a7fc640 1 -- 172.21.15.31:0/968878247 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f17cda8a0 con 0x558f17952da0 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f327a7fc640 10 client.25715 handle_command_reply: tid=0 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 client.25715 shutdown 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 msgr2=0x558f17d75bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d75bd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f328400c6a0 tx=0x7f3284010d10 comp rx=0 tx=0).stop 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 shutdown_connections 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x558f17952da0 0x558f17df0bc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f1799e390 0x558f17d76170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f327a7fc640 0 client.25715 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f179bf070 0x558f17d7a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 --2- 172.21.15.31:0/968878247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f179961b0 0x558f17d75bd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 >> 172.21.15.31:0/968878247 conn(0x558f179b3460 msgr2=0x558f17d708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 shutdown_connections 2024-09-21T02:55:03.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 1 -- 172.21.15.31:0/968878247 wait complete. 2024-09-21T02:55:03.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.113+0000 7f329f6c0740 20 client.25715 trim_cache size 0 max 16384 2024-09-21T02:55:03.119 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32800b6700 msgr2=0x7f32800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32800b6700 0x7f32800b8c10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f328c006010 tx=0x7f328c032ed0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 msgr2=0x7f32940ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 0x7f32940ff670 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f328400e9f0 tx=0x7f328400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 shutdown_connections 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f32800b6700 0x7f32800b8c10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3294108cf0 0x7f32941000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32941083a0 0x7f32940ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 --2- 172.21.15.31:0/1071298381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3294102a60 0x7f32940ff670 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 >> 172.21.15.31:0/1071298381 conn(0x7f3294079730 msgr2=0x7f329406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 shutdown_connections 2024-09-21T02:55:03.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:03.114+0000 7f329914c640 1 -- 172.21.15.31:0/1071298381 wait complete. 2024-09-21T02:55:04.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:55:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:04.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:04 smithi031 ceph-mon[31209]: pgmap v390: 97 pgs: 97 active+clean; 259 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 44 op/s 2024-09-21T02:55:04.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 -- 172.21.15.31:0/1802286586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 msgr2=0x7fbbc810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 --2- 172.21.15.31:0/1802286586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc810d8d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009900 tx=0x7fbbb802a410 comp rx=0 tx=0).stop 2024-09-21T02:55:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 -- 172.21.15.31:0/1802286586 shutdown_connections 2024-09-21T02:55:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 --2- 172.21.15.31:0/1802286586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbc810de10 0x7fbbc8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 --2- 172.21.15.31:0/1802286586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc810d8d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 --2- 172.21.15.31:0/1802286586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 0x7fbbc8104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 -- 172.21.15.31:0/1802286586 >> 172.21.15.31:0/1802286586 conn(0x7fbbc8079740 msgr2=0x7fbbc8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 -- 172.21.15.31:0/1802286586 shutdown_connections 2024-09-21T02:55:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.824+0000 7fbbd0472640 1 -- 172.21.15.31:0/1802286586 wait complete. 2024-09-21T02:55:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 Processor -- start 2024-09-21T02:55:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 -- start start 2024-09-21T02:55:04.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 0x7fbbc8102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc8102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbc810de10 0x7fbbc81031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8113a90 con 0x7fbbc8105290 2024-09-21T02:55:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8113910 con 0x7fbbc8104860 2024-09-21T02:55:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbd0472640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8113c10 con 0x7fbbc810de10 2024-09-21T02:55:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbcda0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc8102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 0x7fbbc8102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbcda0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc8102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54342/0 (socket says 172.21.15.31:54342) 2024-09-21T02:55:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbcda0d640 1 -- 172.21.15.31:0/361285417 learned_addr learned my addr 172.21.15.31:0/361285417 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.825+0000 7fbbcea0f640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbc810de10 0x7fbbc81031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbce20e640 1 -- 172.21.15.31:0/361285417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbc810de10 msgr2=0x7fbbc81031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbce20e640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbc810de10 0x7fbbc81031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbce20e640 1 -- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 msgr2=0x7fbbc8102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbce20e640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc8102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbce20e640 1 -- 172.21.15.31:0/361285417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc81a87a0 con 0x7fbbc8104860 2024-09-21T02:55:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbce20e640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 0x7fbbc8102770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400e9c0 tx=0x7fbbc400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbbf7fe640 1 -- 172.21.15.31:0/361285417 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400ccc0 con 0x7fbbc8104860 2024-09-21T02:55:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbbf7fe640 1 -- 172.21.15.31:0/361285417 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc40035f0 con 0x7fbbc8104860 2024-09-21T02:55:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81a8a90 con 0x7fbbc8104860 2024-09-21T02:55:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.826+0000 7fbbbf7fe640 1 -- 172.21.15.31:0/361285417 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400ce60 con 0x7fbbc8104860 2024-09-21T02:55:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.827+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81a8fd0 con 0x7fbbc8104860 2024-09-21T02:55:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.829+0000 7fbbbf7fe640 1 -- 172.21.15.31:0/361285417 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbbc4004000 con 0x7fbbc8104860 2024-09-21T02:55:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 Processor -- start 2024-09-21T02:55:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 -- start start 2024-09-21T02:55:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fce2070 0x55fa800896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80089be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcc1390 0x55fa8008a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ffffea0 con 0x55fa7fcb91b0 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ffef2d0 con 0x55fa7fce2070 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.830+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fff30e0 con 0x55fa7fcc1390 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcea0f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fce2070 0x55fa800896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcea0f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fce2070 0x55fa800896a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42702/0 (socket says 172.21.15.31:42702) 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcea0f640 1 -- 172.21.15.31:0/279351784 learned_addr learned my addr 172.21.15.31:0/279351784 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcc1390 0x55fa8008a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbce20e640 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80089be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 -- 172.21.15.31:0/279351784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fce2070 msgr2=0x55fa800896a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fce2070 0x55fa800896a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 -- 172.21.15.31:0/279351784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 msgr2=0x55fa80089be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80089be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 -- 172.21.15.31:0/279351784 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7fd4d650 con 0x55fa7fcc1390 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbce20e640 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80089be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.831+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcc1390 0x55fa8008a120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8031440 tx=0x7fbbb8031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8262e9394df1e6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.832+0000 7fbbbcd9a640 1 -- 172.21.15.31:0/279351784 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8040070 con 0x55fa7fcc1390 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.832+0000 7fbbbcd9a640 1 -- 172.21.15.31:0/279351784 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004030 con 0x55fa7fcc1390 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.832+0000 7fbbbf7fe640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbb40b6680 0x7fbbb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.832+0000 7fbbbcd9a640 1 -- 172.21.15.31:0/279351784 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8035040 con 0x55fa7fcc1390 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.832+0000 7fbbcea0f640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbb40b6680 0x7fbbb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.832+0000 7fbbbf7fe640 1 -- 172.21.15.31:0/361285417 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbbc40e7c00 con 0x7fbbc8104860 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbcea0f640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbb40b6680 0x7fbbb40b8b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009fd0 tx=0x7fbbc0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 -- 172.21.15.31:0/279351784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcc1390 msgr2=0x55fa8008a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcc1390 0x55fa8008a120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8031440 tx=0x7fbbb8031b10 comp rx=0 tx=0).stop 2024-09-21T02:55:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 -- 172.21.15.31:0/279351784 shutdown_connections 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcc1390 0x55fa8008a120 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80089be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/279351784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fce2070 0x55fa800896a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 -- 172.21.15.31:0/279351784 >> 172.21.15.31:0/279351784 conn(0x55fa7fcd6460 msgr2=0x55fa80087590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 -- 172.21.15.31:0/279351784 shutdown_connections 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.833+0000 7fbbd69e6740 1 -- 172.21.15.31:0/279351784 wait complete. 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.834+0000 7fbbd69e6740 1 -- start start 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.834+0000 7fbbd69e6740 1 Processor -- start 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.834+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa80094dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.834+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80095310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fce2070 0x55fa800a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fe50700 con 0x55fa7fce2070 2024-09-21T02:55:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ff8a160 con 0x55fa7fcc1390 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fa80097e20 con 0x55fa7fcb91b0 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbd69e6740 10 client.0 _ll_register_callbacks cb 0x55fa7ffa39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa80094dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa80094dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60430/0 (socket says 172.21.15.31:60430) 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 -- 172.21.15.31:0/2146349803 learned_addr learned my addr 172.21.15.31:0/2146349803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbcea0f640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fce2070 0x55fa800a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80095310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 -- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 msgr2=0x55fa80095310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80095310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 -- 172.21.15.31:0/2146349803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fce2070 msgr2=0x55fa800a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fce2070 0x55fa800a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 -- 172.21.15.31:0/2146349803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7ffb7c90 con 0x55fa7fcb91b0 2024-09-21T02:55:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbce20e640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa80094dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40ada00 tx=0x7fbbc4003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e61c96332604ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.835+0000 7fbbb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/2146349803 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40a9050 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/2146349803 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbbc40a9ce0 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb8004030 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/2146349803 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbc40ad580 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 10 client.25789 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa800a3c10 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/2146349803 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004030 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/2146349803 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbc4098030 con 0x55fa7fcb91b0 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 10 client.25789 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 10 client.25789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 20 client.25789 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:04.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.836+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fca09f0 0x55fa800a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.837+0000 7fbbd69e6740 4 client.25789 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.837+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa80001df0 con 0x55fa7fca09f0 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.837+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fca09f0 0x55fa800a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.837+0000 7fbbb1ffb640 10 client.25789 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.837+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fca09f0 0x55fa800a71a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6f8ce5a15866c7a server_cookie=29f3f6b2f7c39713 in_seq=0 out_seq=0 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/2146349803 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa80001df0 con 0x55fa7fca09f0 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbb1ffb640 10 client.25789 handle_command_reply: tid=0 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbd69e6740 1 client.25789 shutdown 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 msgr2=0x55fa80094dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa80094dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40ada00 tx=0x7fbbc4003cd0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 shutdown_connections 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fca09f0 0x55fa800a71a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.838+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fce2070 0x55fa800a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80095310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbb1ffb640 0 client.25789 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/2146349803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa80094dd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 >> 172.21.15.31:0/2146349803 conn(0x55fa7fcd6460 msgr2=0x55fa8008c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 shutdown_connections 2024-09-21T02:55:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbd69e6740 1 -- 172.21.15.31:0/2146349803 wait complete. 2024-09-21T02:55:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.839+0000 7fbbd69e6740 20 client.25789 trim_cache size 0 max 16384 2024-09-21T02:55:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 Processor -- start 2024-09-21T02:55:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 -- start start 2024-09-21T02:55:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 0x55fa80095290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa800a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcc1390 0x55fa800a1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b1950 con 0x55fa7fcc1390 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b3490 con 0x55fa7fca09f0 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.853+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b2020 con 0x55fa7fcb91b0 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbcda0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa800a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 0x55fa80095290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 0x55fa80095290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42710/0 (socket says 172.21.15.31:42710) 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbcda0d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa800a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60444/0 (socket says 172.21.15.31:60444) 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 -- 172.21.15.31:0/156669845 learned_addr learned my addr 172.21.15.31:0/156669845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbcea0f640 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcc1390 0x55fa800a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 -- 172.21.15.31:0/156669845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 msgr2=0x55fa800a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa800a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 -- 172.21.15.31:0/156669845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcc1390 msgr2=0x55fa800a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcc1390 0x55fa800a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 -- 172.21.15.31:0/156669845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa80090900 con 0x55fa7fca09f0 2024-09-21T02:55:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbce20e640 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 0x55fa80095290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40ad770 tx=0x7fbbc40039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a779d25fbfe8dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:04.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.854+0000 7fbbb3fff640 1 -- 172.21.15.31:0/156669845 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40a9040 con 0x55fa7fca09f0 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.855+0000 7fbbb3fff640 1 -- 172.21.15.31:0/156669845 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc40ae170 con 0x55fa7fca09f0 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.855+0000 7fbbb3fff640 1 -- 172.21.15.31:0/156669845 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40e5ac0 con 0x55fa7fca09f0 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.855+0000 7fbbd69e6740 1 -- 172.21.15.31:0/156669845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 msgr2=0x55fa80095290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.855+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 0x55fa80095290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40ad770 tx=0x7fbbc40039b0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 -- 172.21.15.31:0/156669845 shutdown_connections 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fcb91b0 0x55fa800a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcc1390 0x55fa800a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/156669845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fca09f0 0x55fa80095290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 -- 172.21.15.31:0/156669845 >> 172.21.15.31:0/156669845 conn(0x55fa7fcd6460 msgr2=0x55fa80094b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 -- 172.21.15.31:0/156669845 shutdown_connections 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 -- 172.21.15.31:0/156669845 wait complete. 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 -- start start 2024-09-21T02:55:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 Processor -- start 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.856+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 0x55fa8009d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80098eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbd69e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80099550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b2dc0 con 0x55fa7fcb91b0 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b26f0 con 0x55fa7fcc1390 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbd69e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81af930 con 0x55fa7fca09f0 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbd69e6740 10 client.0 _ll_register_callbacks cb 0x7fbbc40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 0x55fa8009d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 0x55fa8009d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60450/0 (socket says 172.21.15.31:60450) 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 -- 172.21.15.31:0/102438054 learned_addr learned my addr 172.21.15.31:0/102438054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbcea0f640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80098eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 -- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 msgr2=0x55fa80099550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80099550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 -- 172.21.15.31:0/102438054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 msgr2=0x55fa80098eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80098eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 -- 172.21.15.31:0/102438054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa80099ce0 con 0x55fa7fca09f0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80098eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbce20e640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 0x55fa8009d9b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40ada00 tx=0x7fbbc40eb2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457c6d07c1c3a88a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.857+0000 7fbbb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/102438054 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40a9040 con 0x55fa7fca09f0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/102438054 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbbc40e5ac0 con 0x55fa7fca09f0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc40ae170 con 0x55fa7fca09f0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/102438054 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbc40cc070 con 0x55fa7fca09f0 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbd69e6740 10 client.25795 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa80111c10 con 0x55fa7fca09f0 2024-09-21T02:55:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.858+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/102438054 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc40ae170 con 0x55fa7fca09f0 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/102438054 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbc40ef040 con 0x55fa7fca09f0 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbd69e6740 10 client.25795 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbd69e6740 10 client.25795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbd69e6740 20 client.25795 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fce2070 0x55fa80114150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbd69e6740 4 client.25795 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa7fffd8a0 con 0x55fa7fce2070 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fce2070 0x55fa80114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbcda0d640 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fce2070 0x55fa80114150 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=985f109cb0fd8c39 server_cookie=33ec3b0b6480ac62 in_seq=0 out_seq=0 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.859+0000 7fbbb1ffb640 10 client.25795 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbb1ffb640 1 -- 172.21.15.31:0/102438054 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa7fffd8a0 con 0x55fa7fce2070 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbb1ffb640 10 client.25795 handle_command_reply: tid=0 2024-09-21T02:55:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 client.25795 shutdown 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 msgr2=0x55fa8009d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 0x55fa8009d9b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40ada00 tx=0x7fbbc40eb2e0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 shutdown_connections 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55fa7fce2070 0x55fa80114150 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa7fcc1390 0x55fa80099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fa7fcb91b0 0x55fa80098eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 --2- 172.21.15.31:0/102438054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fa7fca09f0 0x55fa8009d9b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbb1ffb640 0 client.25795 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 >> 172.21.15.31:0/102438054 conn(0x55fa7fcd6460 msgr2=0x55fa8008a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 shutdown_connections 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 1 -- 172.21.15.31:0/102438054 wait complete. 2024-09-21T02:55:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.860+0000 7fbbd69e6740 20 client.25795 trim_cache size 0 max 16384 2024-09-21T02:55:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbb40b6680 msgr2=0x7fbbb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbb40b6680 0x7fbbb40b8b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009fd0 tx=0x7fbbc0009500 comp rx=0 tx=0).stop 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 msgr2=0x7fbbc8102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 0x7fbbc8102770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400e9c0 tx=0x7fbbc400ee80 comp rx=0 tx=0).stop 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 shutdown_connections 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbbb40b6680 0x7fbbb40b8b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbc810de10 0x7fbbc81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbc8105290 0x7fbbc8102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 --2- 172.21.15.31:0/361285417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbc8104860 0x7fbbc8102770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 >> 172.21.15.31:0/361285417 conn(0x7fbbc8079740 msgr2=0x7fbbc80ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 shutdown_connections 2024-09-21T02:55:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:04.861+0000 7fbbd0472640 1 -- 172.21.15.31:0/361285417 wait complete. 2024-09-21T02:55:04.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:04 smithi165 ceph-mon[34818]: pgmap v390: 97 pgs: 97 active+clean; 259 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 44 op/s 2024-09-21T02:55:04.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:04 smithi178 ceph-mon[34759]: pgmap v390: 97 pgs: 97 active+clean; 259 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 44 op/s 2024-09-21T02:55:05.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:55:05.872 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.659+0000 7fab56f8f640 1 -- 172.21.15.31:0/4171514479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 msgr2=0x7fab50102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.659+0000 7fab56f8f640 1 --2- 172.21.15.31:0/4171514479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab50102380 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fab44009900 tx=0x7fab4402a410 comp rx=0 tx=0).stop 2024-09-21T02:55:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 -- 172.21.15.31:0/4171514479 shutdown_connections 2024-09-21T02:55:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 --2- 172.21.15.31:0/4171514479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 0x7fab50110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 --2- 172.21.15.31:0/4171514479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab501094b0 0x7fab501098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 --2- 172.21.15.31:0/4171514479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab50102380 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 -- 172.21.15.31:0/4171514479 >> 172.21.15.31:0/4171514479 conn(0x7fab50077c60 msgr2=0x7fab500780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:06.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 -- 172.21.15.31:0/4171514479 shutdown_connections 2024-09-21T02:55:06.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 -- 172.21.15.31:0/4171514479 wait complete. 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.660+0000 7fab56f8f640 1 Processor -- start 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 -- start start 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab5019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab501094b0 0x7fab5019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 0x7fab5019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab50113ac0 con 0x7fab50101f70 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab50113940 con 0x7fab501094b0 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab56f8f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab50113c40 con 0x7fab50109e00 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab5019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab5019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54404/0 (socket says 172.21.15.31:54404) 2024-09-21T02:55:06.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab54d2b640 1 -- 172.21.15.31:0/1983458604 learned_addr learned my addr 172.21.15.31:0/1983458604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab5552c640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 0x7fab5019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.661+0000 7fab4ffff640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab501094b0 0x7fab5019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab5552c640 1 -- 172.21.15.31:0/1983458604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab501094b0 msgr2=0x7fab5019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab5552c640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab501094b0 0x7fab5019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab5552c640 1 -- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 msgr2=0x7fab5019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab5552c640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab5019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab5552c640 1 -- 172.21.15.31:0/1983458604 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab501a03e0 con 0x7fab50109e00 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab54d2b640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab5019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab5552c640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 0x7fab5019fcb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab4800ec50 tx=0x7fab4800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab4dffb640 1 -- 172.21.15.31:0/1983458604 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48003b70 con 0x7fab50109e00 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab501a06d0 con 0x7fab50109e00 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab4dffb640 1 -- 172.21.15.31:0/1983458604 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab48003d10 con 0x7fab50109e00 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab4dffb640 1 -- 172.21.15.31:0/1983458604 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48010830 con 0x7fab50109e00 2024-09-21T02:55:06.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.662+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab501a6e30 con 0x7fab50109e00 2024-09-21T02:55:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 Processor -- start 2024-09-21T02:55:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab4dffb640 1 -- 172.21.15.31:0/1983458604 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fab480109d0 con 0x7fab50109e00 2024-09-21T02:55:06.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 -- start start 2024-09-21T02:55:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 0x561719c596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198891b0 0x561719c59be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561719bcfea0 con 0x561719891390 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561719bbf2d0 con 0x5617198b2070 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.666+0000 7fab5d503740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561719bc30e0 con 0x5617198891b0 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.667+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 0x561719c596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.667+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198891b0 0x561719c59be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.667+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198891b0 0x561719c59be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60460/0 (socket says 172.21.15.31:60460) 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.667+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 0x561719c596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42756/0 (socket says 172.21.15.31:42756) 2024-09-21T02:55:06.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.667+0000 7fab54d2b640 1 -- 172.21.15.31:0/3842382674 learned_addr learned my addr 172.21.15.31:0/3842382674 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 -- 172.21.15.31:0/3842382674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198891b0 msgr2=0x561719c59be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198891b0 0x561719c59be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 -- 172.21.15.31:0/3842382674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 msgr2=0x561719c5a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c5a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 -- 172.21.15.31:0/3842382674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171991d650 con 0x5617198b2070 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4dffb640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fab400b6680 0x7fab400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 0x561719c596a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab501091d0 tx=0x7fab3c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ba3f25f626c52b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab4dffb640 1 -- 172.21.15.31:0/1983458604 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fab480e7c40 con 0x7fab50109e00 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab3affd640 1 -- 172.21.15.31:0/3842382674 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004030 con 0x5617198b2070 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.668+0000 7fab54d2b640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fab400b6680 0x7fab400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.669+0000 7fab3affd640 1 -- 172.21.15.31:0/3842382674 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c0054d0 con 0x5617198b2070 2024-09-21T02:55:06.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.669+0000 7fab3affd640 1 -- 172.21.15.31:0/3842382674 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c004b20 con 0x5617198b2070 2024-09-21T02:55:06.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 -- 172.21.15.31:0/3842382674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 msgr2=0x561719c596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 0x561719c596a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab501091d0 tx=0x7fab3c009770 comp rx=0 tx=0).stop 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab54d2b640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fab400b6680 0x7fab400b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab440096c0 tx=0x7fab440058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 -- 172.21.15.31:0/3842382674 shutdown_connections 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198891b0 0x561719c59be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 --2- 172.21.15.31:0/3842382674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198b2070 0x561719c596a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 -- 172.21.15.31:0/3842382674 >> 172.21.15.31:0/3842382674 conn(0x5617198a6460 msgr2=0x561719c57590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 -- 172.21.15.31:0/3842382674 shutdown_connections 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 -- 172.21.15.31:0/3842382674 wait complete. 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.670+0000 7fab5d503740 1 -- start start 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 Processor -- start 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 0x561719c75510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561719a20700 con 0x5617198b2070 2024-09-21T02:55:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561719b5a160 con 0x5617198891b0 2024-09-21T02:55:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561719c6c1e0 con 0x561719891390 2024-09-21T02:55:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5d503740 10 client.0 _ll_register_callbacks cb 0x561719b739e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5552c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 0x561719c75510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5552c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 0x561719c75510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54418/0 (socket says 172.21.15.31:54418) 2024-09-21T02:55:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.671+0000 7fab5552c640 1 -- 172.21.15.31:0/231122953 learned_addr learned my addr 172.21.15.31:0/231122953 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab5552c640 1 -- 172.21.15.31:0/231122953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 msgr2=0x561719c69680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:55:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab4ffff640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab5552c640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c69680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab5552c640 1 -- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 msgr2=0x561719c69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab5552c640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c69140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab5552c640 1 -- 172.21.15.31:0/231122953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561719b87c90 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab54d2b640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab5552c640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 0x561719c75510 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fab480ada40 tx=0x7fab480ad4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5e1b62c53803ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab2ffff640 1 -- 172.21.15.31:0/231122953 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48004660 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.672+0000 7fab2ffff640 1 -- 172.21.15.31:0/231122953 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fab480acd10 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab3c0054d0 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab2ffff640 1 -- 172.21.15.31:0/231122953 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab480ad5c0 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 10 client.16128 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561719c77ed0 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab2ffff640 1 -- 172.21.15.31:0/231122953 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c0054d0 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab2ffff640 1 -- 172.21.15.31:0/231122953 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fab480cfac0 con 0x5617198b2070 2024-09-21T02:55:06.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 10 client.16128 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 20 client.16128 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198709f0 0x561719c7b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 4 client.16128 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.673+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561719bd1df0 con 0x5617198709f0 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.674+0000 7fab54d2b640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198709f0 0x561719c7b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.674+0000 7fab2ffff640 10 client.16128 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.674+0000 7fab54d2b640 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198709f0 0x561719c7b460 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340bfd05adf1acc6 server_cookie=ab64f3c033ec727f in_seq=0 out_seq=0 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab2ffff640 1 -- 172.21.15.31:0/231122953 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561719bd1df0 con 0x5617198709f0 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab2ffff640 10 client.16128 handle_command_reply: tid=0 2024-09-21T02:55:06.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 client.16128 shutdown 2024-09-21T02:55:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 msgr2=0x561719c75510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 0x561719c75510 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fab480ada40 tx=0x7fab480ad4c0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 shutdown_connections 2024-09-21T02:55:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198709f0 0x561719c7b460 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab2ffff640 0 client.16128 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198b2070 0x561719c75510 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 --2- 172.21.15.31:0/231122953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.675+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 >> 172.21.15.31:0/231122953 conn(0x5617198a6460 msgr2=0x561719c55bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.676+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 shutdown_connections 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.676+0000 7fab5d503740 1 -- 172.21.15.31:0/231122953 wait complete. 2024-09-21T02:55:06.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.676+0000 7fab5d503740 20 client.16128 trim_cache size 0 max 16384 2024-09-21T02:55:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 Processor -- start 2024-09-21T02:55:06.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 -- start start 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 0x561719c575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198891b0 0x561719c75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c66330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab501ad4d0 con 0x5617198891b0 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab501af010 con 0x5617198709f0 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.691+0000 7fab5d503740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab501adba0 con 0x561719891390 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 0x561719c575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198891b0 0x561719c75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 0x561719c575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42766/0 (socket says 172.21.15.31:42766) 2024-09-21T02:55:06.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab54d2b640 1 -- 172.21.15.31:0/3939774770 learned_addr learned my addr 172.21.15.31:0/3939774770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab5552c640 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c66330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 -- 172.21.15.31:0/3939774770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 msgr2=0x561719c66330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c66330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 -- 172.21.15.31:0/3939774770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 msgr2=0x561719c575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 0x561719c575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 -- 172.21.15.31:0/3939774770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561719c5bbc0 con 0x5617198891b0 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab54d2b640 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 0x561719c575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198891b0 0x561719c75410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0058c0 tx=0x7fab3c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8b8dcbf401af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab3a7fc640 1 -- 172.21.15.31:0/3939774770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c02c070 con 0x5617198891b0 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.692+0000 7fab3a7fc640 1 -- 172.21.15.31:0/3939774770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c0035f0 con 0x5617198891b0 2024-09-21T02:55:06.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab3a7fc640 1 -- 172.21.15.31:0/3939774770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c01e680 con 0x5617198891b0 2024-09-21T02:55:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 -- 172.21.15.31:0/3939774770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198891b0 msgr2=0x561719c75410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198891b0 0x561719c75410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0058c0 tx=0x7fab3c0051d0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 -- 172.21.15.31:0/3939774770 shutdown_connections 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561719891390 0x561719c66330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617198891b0 0x561719c75410 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 --2- 172.21.15.31:0/3939774770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198709f0 0x561719c575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 -- 172.21.15.31:0/3939774770 >> 172.21.15.31:0/3939774770 conn(0x5617198a6460 msgr2=0x561719c56eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.693+0000 7fab5d503740 1 -- 172.21.15.31:0/3939774770 shutdown_connections 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.694+0000 7fab5d503740 1 -- 172.21.15.31:0/3939774770 wait complete. 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.694+0000 7fab5d503740 1 -- start start 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.694+0000 7fab5d503740 1 Processor -- start 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.694+0000 7fab5d503740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198709f0 0x561719c6d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.694+0000 7fab5d503740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c6d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.694+0000 7fab5d503740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c6dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5d503740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab501ae940 con 0x561719891390 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5d503740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab501ae270 con 0x5617198891b0 2024-09-21T02:55:06.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5d503740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab501ab4b0 con 0x5617198709f0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5d503740 10 client.0 _ll_register_callbacks cb 0x7fab48004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5552c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c6dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab54d2b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198709f0 0x561719c6d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c6d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5552c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c6dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54442/0 (socket says 172.21.15.31:54442) 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab5552c640 1 -- 172.21.15.31:0/3042705919 learned_addr learned my addr 172.21.15.31:0/3042705919 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 -- 172.21.15.31:0/3042705919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198709f0 msgr2=0x561719c6d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198709f0 0x561719c6d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 -- 172.21.15.31:0/3042705919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 msgr2=0x561719c6dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c6dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 -- 172.21.15.31:0/3042705919 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561719ce4e20 con 0x5617198891b0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab4ffff640 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c6d980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009ec0 tx=0x7fab3c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16271e1a6be12d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.695+0000 7fab2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:55:06.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab2ffff640 1 -- 172.21.15.31:0/3042705919 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035060 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab2ffff640 1 -- 172.21.15.31:0/3042705919 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fab3c005340 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab3c0035f0 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab2ffff640 1 -- 172.21.15.31:0/3042705919 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab3c02c040 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab5d503740 10 client.25751 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561719ce60a0 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.696+0000 7fab2ffff640 1 -- 172.21.15.31:0/3042705919 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c0035f0 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab2ffff640 1 -- 172.21.15.31:0/3042705919 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fab3c02f070 con 0x5617198891b0 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab5d503740 10 client.25751 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab5d503740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab5d503740 20 client.25751 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab5d503740 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198b2070 0x561719ce84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:06.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab5d503740 4 client.25751 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561719bcd8a0 con 0x5617198b2070 2024-09-21T02:55:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab54d2b640 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198b2070 0x561719ce84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:06.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.697+0000 7fab2ffff640 10 client.25751 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:06.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab54d2b640 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198b2070 0x561719ce84e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965edecf8372fc44 server_cookie=5ef7f9bbd87acc5b in_seq=0 out_seq=0 2024-09-21T02:55:06.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab2ffff640 1 -- 172.21.15.31:0/3042705919 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561719bcd8a0 con 0x5617198b2070 2024-09-21T02:55:06.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab2ffff640 10 client.25751 handle_command_reply: tid=0 2024-09-21T02:55:06.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 client.25751 shutdown 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 msgr2=0x561719c6d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c6d980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009ec0 tx=0x7fab3c005d50 comp rx=0 tx=0).stop 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 shutdown_connections 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5617198b2070 0x561719ce84e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561719891390 0x561719c6dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617198891b0 0x561719c6d980 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 --2- 172.21.15.31:0/3042705919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617198709f0 0x561719c6d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 >> 172.21.15.31:0/3042705919 conn(0x5617198a6460 msgr2=0x561719c6a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab2ffff640 0 client.25751 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 shutdown_connections 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.698+0000 7fab5d503740 1 -- 172.21.15.31:0/3042705919 wait complete. 2024-09-21T02:55:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab5d503740 20 client.25751 trim_cache size 0 max 16384 2024-09-21T02:55:06.705 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:06.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fab400b6680 msgr2=0x7fab400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fab400b6680 0x7fab400b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab440096c0 tx=0x7fab440058d0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 msgr2=0x7fab5019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 0x7fab5019fcb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab4800ec50 tx=0x7fab4800c6a0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 shutdown_connections 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fab400b6680 0x7fab400b8b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab50109e00 0x7fab5019fcb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab501094b0 0x7fab5019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 --2- 172.21.15.31:0/1983458604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50101f70 0x7fab5019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 >> 172.21.15.31:0/1983458604 conn(0x7fab50077c60 msgr2=0x7fab50107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 shutdown_connections 2024-09-21T02:55:06.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:06.699+0000 7fab56f8f640 1 -- 172.21.15.31:0/1983458604 wait complete. 2024-09-21T02:55:06.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:06 smithi031 ceph-mon[31209]: pgmap v391: 97 pgs: 97 active+clean; 259 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 179 op/s 2024-09-21T02:55:06.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:06 smithi165 ceph-mon[34818]: pgmap v391: 97 pgs: 97 active+clean; 259 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 179 op/s 2024-09-21T02:55:06.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:06 smithi178 ceph-mon[34759]: pgmap v391: 97 pgs: 97 active+clean; 259 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 179 op/s 2024-09-21T02:55:08.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:55:08.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:55:08.906 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:08.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:08 smithi165 ceph-mon[34818]: pgmap v392: 97 pgs: 97 active+clean; 260 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-09-21T02:55:08.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:08 smithi178 ceph-mon[34759]: pgmap v392: 97 pgs: 97 active+clean; 260 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-09-21T02:55:09.031 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:08 smithi031 ceph-mon[31209]: pgmap v392: 97 pgs: 97 active+clean; 260 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-09-21T02:55:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 -- 172.21.15.31:0/3115652814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688103b80 msgr2=0x7fd688103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/3115652814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688103b80 0x7fd688103f90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c02a410 comp rx=0 tx=0).stop 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 -- 172.21.15.31:0/3115652814 shutdown_connections 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/3115652814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 0x7fd688110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/3115652814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6881094c0 0x7fd6881098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/3115652814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd688103b80 0x7fd688103f90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 -- 172.21.15.31:0/3115652814 >> 172.21.15.31:0/3115652814 conn(0x7fd6880ff590 msgr2=0x7fd6881019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 -- 172.21.15.31:0/3115652814 shutdown_connections 2024-09-21T02:55:09.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.451+0000 7fd68ef1f640 1 -- 172.21.15.31:0/3115652814 wait complete. 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 Processor -- start 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 -- start start 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 0x7fd688075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 0x7fd688075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 0x7fd688076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd688113ad0 con 0x7fd6881094c0 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd688113950 con 0x7fd688103b80 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ef1f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd688113c50 con 0x7fd688109e10 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ccbb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 0x7fd688075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd687fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 0x7fd688075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ccbb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 0x7fd688075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:47508/0 (socket says 172.21.15.31:47508) 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68ccbb640 1 -- 172.21.15.31:0/222298843 learned_addr learned my addr 172.21.15.31:0/222298843 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd687fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 0x7fd688075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40454/0 (socket says 172.21.15.31:40454) 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.452+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 0x7fd688076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ccbb640 1 -- 172.21.15.31:0/222298843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 msgr2=0x7fd688076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ccbb640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 0x7fd688076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ccbb640 1 -- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 msgr2=0x7fd688075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ccbb640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 0x7fd688075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ccbb640 1 -- 172.21.15.31:0/222298843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6881a8780 con 0x7fd688103b80 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd687fff640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 0x7fd688075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 0x7fd688076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ccbb640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 0x7fd688075710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005940 tx=0x7fd67c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd685ffb640 1 -- 172.21.15.31:0/222298843 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c03f030 con 0x7fd688103b80 2024-09-21T02:55:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd685ffb640 1 -- 172.21.15.31:0/222298843 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c0044c0 con 0x7fd688103b80 2024-09-21T02:55:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd685ffb640 1 -- 172.21.15.31:0/222298843 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c035070 con 0x7fd688103b80 2024-09-21T02:55:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.453+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6881a8a10 con 0x7fd688103b80 2024-09-21T02:55:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.454+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a8ef0 con 0x7fd688103b80 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 Processor -- start 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 -- start start 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd685ffb640 1 -- 172.21.15.31:0/222298843 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd67c042030 con 0x7fd688103b80 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 0x5618e5ba5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57dd670 0x5618e5ba5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618e5b1be60 con 0x5618e57dd670 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5618e5b0b290 con 0x5618e57d5490 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.456+0000 7fd695493740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5618e5b0f0a0 con 0x5618e57fe350 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 0x5618e5ba5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 0x5618e5ba5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:46580/0 (socket says 172.21.15.31:46580) 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 -- 172.21.15.31:0/3071766593 learned_addr learned my addr 172.21.15.31:0/3071766593 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd687fff640 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57dd670 0x5618e5ba5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68ccbb640 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 -- 172.21.15.31:0/3071766593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 msgr2=0x5618e5ba5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 -- 172.21.15.31:0/3071766593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57dd670 msgr2=0x5618e5ba5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57dd670 0x5618e5ba5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 -- 172.21.15.31:0/3071766593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e5869930 con 0x5618e57fe350 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd687fff640 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57dd670 0x5618e5ba5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.457+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 0x5618e5ba5430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6800117e0 tx=0x7fd680011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55df5b049e24fb7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.458+0000 7fd672ffd640 1 -- 172.21.15.31:0/3071766593 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680011310 con 0x5618e57fe350 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.458+0000 7fd672ffd640 1 -- 172.21.15.31:0/3071766593 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd680003e20 con 0x5618e57fe350 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.458+0000 7fd672ffd640 1 -- 172.21.15.31:0/3071766593 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800044f0 con 0x5618e57fe350 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.458+0000 7fd685ffb640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd6780b6680 0x7fd6780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.458+0000 7fd68ccbb640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd6780b6680 0x7fd6780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.458+0000 7fd685ffb640 1 -- 172.21.15.31:0/222298843 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd67c107a40 con 0x7fd688103b80 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 -- 172.21.15.31:0/3071766593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 msgr2=0x5618e5ba5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 0x5618e5ba5430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd6800117e0 tx=0x7fd680011ca0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd68ccbb640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd6780b6680 0x7fd6780b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd67c105260 tx=0x7fd67c10b590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 -- 172.21.15.31:0/3071766593 shutdown_connections 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57dd670 0x5618e5ba5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 --2- 172.21.15.31:0/3071766593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57fe350 0x5618e5ba5430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 -- 172.21.15.31:0/3071766593 >> 172.21.15.31:0/3071766593 conn(0x5618e57f2740 msgr2=0x5618e5ba3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 -- 172.21.15.31:0/3071766593 shutdown_connections 2024-09-21T02:55:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 -- 172.21.15.31:0/3071766593 wait complete. 2024-09-21T02:55:09.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.459+0000 7fd695493740 1 -- start start 2024-09-21T02:55:09.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 Processor -- start 2024-09-21T02:55:09.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57d5490 0x5618e5bb0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57fe350 0x5618e5bbcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618e596c890 con 0x5618e57d5490 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5618e5aa5d40 con 0x5618e57fe350 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5618e5bb3c10 con 0x5618e57dd670 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd695493740 10 client.0 _ll_register_callbacks cb 0x5618e5b09c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57d5490 0x5618e5bb0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57d5490 0x5618e5bb0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40468/0 (socket says 172.21.15.31:40468) 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd68d4bc640 1 -- 172.21.15.31:0/2505816898 learned_addr learned my addr 172.21.15.31:0/2505816898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd687fff640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.460+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57fe350 0x5618e5bbcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd687fff640 1 -- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57fe350 msgr2=0x5618e5bbcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd687fff640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57fe350 0x5618e5bbcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd687fff640 1 -- 172.21.15.31:0/2505816898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57d5490 msgr2=0x5618e5bb0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd687fff640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57d5490 0x5618e5bb0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd687fff640 1 -- 172.21.15.31:0/2505816898 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e5bbd4c0 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd687fff640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb1510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd68806aa90 tx=0x7fd674012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb80fec315c1808 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd667fff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd667fff640 1 -- 172.21.15.31:0/2505816898 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674004030 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680003e20 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd667fff640 1 -- 172.21.15.31:0/2505816898 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd6740041d0 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd667fff640 1 -- 172.21.15.31:0/2505816898 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6740043d0 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.461+0000 7fd695493740 10 client.25822 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618e5bbfea0 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd667fff640 1 -- 172.21.15.31:0/2505816898 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd680003e20 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd667fff640 1 -- 172.21.15.31:0/2505816898 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd67402b070 con 0x5618e57dd670 2024-09-21T02:55:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 10 client.25822 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 10 client.25822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 20 client.25822 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e57bccd0 0x5618e5bc3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 4 client.25822 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.462+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618e5b1ddb0 con 0x5618e57bccd0 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.463+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e57bccd0 0x5618e5bc3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.463+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e57bccd0 0x5618e5bc3530 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744424a5fa75497d server_cookie=2a929c215446b381 in_seq=0 out_seq=0 2024-09-21T02:55:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.463+0000 7fd667fff640 10 client.25822 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd667fff640 1 -- 172.21.15.31:0/2505816898 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618e5b1ddb0 con 0x5618e57bccd0 2024-09-21T02:55:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd667fff640 10 client.25822 handle_command_reply: tid=0 2024-09-21T02:55:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 client.25822 shutdown 2024-09-21T02:55:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 msgr2=0x5618e5bb1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb1510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd68806aa90 tx=0x7fd674012820 comp rx=0 tx=0).stop 2024-09-21T02:55:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 shutdown_connections 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e57bccd0 0x5618e5bc3530 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb1510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57fe350 0x5618e5bbcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 --2- 172.21.15.31:0/2505816898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57d5490 0x5618e5bb0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd667fff640 0 client.25822 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 >> 172.21.15.31:0/2505816898 conn(0x5618e57f2740 msgr2=0x5618e5ba85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 shutdown_connections 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 1 -- 172.21.15.31:0/2505816898 wait complete. 2024-09-21T02:55:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.464+0000 7fd695493740 20 client.25822 trim_cache size 0 max 16384 2024-09-21T02:55:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 Processor -- start 2024-09-21T02:55:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 -- start start 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5bbce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ad4f0 con 0x5618e57fe350 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881af030 con 0x5618e57d5490 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd695493740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881adbc0 con 0x5618e57dd670 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5bbce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd687fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.485+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5bbce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40482/0 (socket says 172.21.15.31:40482) 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd68d4bc640 1 -- 172.21.15.31:0/24885049 learned_addr learned my addr 172.21.15.31:0/24885049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd687fff640 1 -- 172.21.15.31:0/24885049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 msgr2=0x5618e5ba3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd687fff640 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd687fff640 1 -- 172.21.15.31:0/24885049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 msgr2=0x5618e5bbce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd687fff640 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5bbce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd687fff640 1 -- 172.21.15.31:0/24885049 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e5ba7720 con 0x5618e57dd670 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5bbce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd687fff640 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bab550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd674013010 tx=0x7fd6740109d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb179c1269f0ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd6727fc640 1 -- 172.21.15.31:0/24885049 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674031070 con 0x5618e57dd670 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd6727fc640 1 -- 172.21.15.31:0/24885049 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd674010d40 con 0x5618e57dd670 2024-09-21T02:55:09.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.486+0000 7fd6727fc640 1 -- 172.21.15.31:0/24885049 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6740047d0 con 0x5618e57dd670 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 -- 172.21.15.31:0/24885049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 msgr2=0x5618e5bab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bab550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd674013010 tx=0x7fd6740109d0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 -- 172.21.15.31:0/24885049 shutdown_connections 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bab550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5bbce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 --2- 172.21.15.31:0/24885049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5ba3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 -- 172.21.15.31:0/24885049 >> 172.21.15.31:0/24885049 conn(0x5618e57f2740 msgr2=0x5618e5baf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 -- 172.21.15.31:0/24885049 shutdown_connections 2024-09-21T02:55:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.487+0000 7fd695493740 1 -- 172.21.15.31:0/24885049 wait complete. 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.488+0000 7fd695493740 1 -- start start 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.488+0000 7fd695493740 1 Processor -- start 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.488+0000 7fd695493740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5bb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.488+0000 7fd695493740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.488+0000 7fd695493740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5c2cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.488+0000 7fd695493740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ae960 con 0x5618e57fe350 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd695493740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ae290 con 0x5618e57d5490 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd695493740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ab4d0 con 0x5618e57dd670 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd695493740 10 client.0 _ll_register_callbacks cb 0x7fd674009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5bb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd687fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5bb5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:47546/0 (socket says 172.21.15.31:47546) 2024-09-21T02:55:09.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 -- 172.21.15.31:0/4104766760 learned_addr learned my addr 172.21.15.31:0/4104766760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5c2cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 -- 172.21.15.31:0/4104766760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 msgr2=0x5618e5bb5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 -- 172.21.15.31:0/4104766760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 msgr2=0x5618e5c2cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5c2cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 -- 172.21.15.31:0/4104766760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e5c2d470 con 0x5618e57d5490 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd68d4bc640 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5bb5560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd680003c50 tx=0x7fd68001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5784ff56db1cb485 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.489+0000 7fd667fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd667fff640 1 -- 172.21.15.31:0/4104766760 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680023040 con 0x5618e57d5490 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd674010d40 con 0x5618e57d5490 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd667fff640 1 -- 172.21.15.31:0/4104766760 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd68000fba0 con 0x5618e57d5490 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd667fff640 1 -- 172.21.15.31:0/4104766760 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd68000fda0 con 0x5618e57d5490 2024-09-21T02:55:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 10 client.25769 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618e5c2e770 con 0x5618e57d5490 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd667fff640 1 -- 172.21.15.31:0/4104766760 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd674010d40 con 0x5618e57d5490 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd667fff640 1 -- 172.21.15.31:0/4104766760 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd680018020 con 0x5618e57d5490 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 10 client.25769 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 20 client.25769 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e5792080 0x5618e5c30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 4 client.25769 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.490+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618e5b19860 con 0x5618e5792080 2024-09-21T02:55:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.491+0000 7fd687fff640 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e5792080 0x5618e5c30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:09.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.491+0000 7fd687fff640 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e5792080 0x5618e5c30d30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa91020701f6e72c server_cookie=2d40035d563a833e in_seq=0 out_seq=0 2024-09-21T02:55:09.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.491+0000 7fd667fff640 10 client.25769 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd667fff640 1 -- 172.21.15.31:0/4104766760 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618e5b19860 con 0x5618e5792080 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd667fff640 10 client.25769 handle_command_reply: tid=0 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 client.25769 shutdown 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 msgr2=0x5618e5bb5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5bb5560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd680003c50 tx=0x7fd68001c040 comp rx=0 tx=0).stop 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 shutdown_connections 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5618e5792080 0x5618e5c30d30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd667fff640 0 client.25769 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5618e57dd670 0x5618e5bb5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618e57fe350 0x5618e5c2cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 --2- 172.21.15.31:0/4104766760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5618e57d5490 0x5618e5bb5560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 >> 172.21.15.31:0/4104766760 conn(0x5618e57f2740 msgr2=0x5618e5ba6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 shutdown_connections 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.493+0000 7fd695493740 1 -- 172.21.15.31:0/4104766760 wait complete. 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd695493740 20 client.25769 trim_cache size 0 max 16384 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "70992e9d-1ff5-484f-832c-e890cc2e84da", 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:55:09.498 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd6780b6680 msgr2=0x7fd6780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd6780b6680 0x7fd6780b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd67c105260 tx=0x7fd67c10b590 comp rx=0 tx=0).stop 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 msgr2=0x7fd688075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 0x7fd688075710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005940 tx=0x7fd67c0040c0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 shutdown_connections 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd6780b6680 0x7fd6780b8b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd688109e10 0x7fd688076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6881094c0 0x7fd688075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 --2- 172.21.15.31:0/222298843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd688103b80 0x7fd688075710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.494+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 >> 172.21.15.31:0/222298843 conn(0x7fd6880ff590 msgr2=0x7fd688102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:09.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.495+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 shutdown_connections 2024-09-21T02:55:09.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:09.495+0000 7fd68ef1f640 1 -- 172.21.15.31:0/222298843 wait complete. 2024-09-21T02:55:10.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:55:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:10 smithi165 ceph-mon[34818]: pgmap v393: 97 pgs: 97 active+clean; 260 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.3 MiB/s wr, 386 op/s 2024-09-21T02:55:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:10 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:55:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:10 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:10 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:55:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:10 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:10.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:10 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:55:10.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:10 smithi178 ceph-mon[34759]: pgmap v393: 97 pgs: 97 active+clean; 260 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.3 MiB/s wr, 386 op/s 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:10 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:10 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:10 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:10 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:10 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:10 smithi031 ceph-mon[31209]: pgmap v393: 97 pgs: 97 active+clean; 260 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.3 MiB/s wr, 386 op/s 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:10 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:55:10.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:10 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:10.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:10 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:55:10.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:10 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:10.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:10 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:55:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 -- 172.21.15.31:0/3344109015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8601094a0 msgr2=0x7fe8601098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 --2- 172.21.15.31:0/3344109015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8601094a0 0x7fe8601098b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe84802a410 comp rx=0 tx=0).stop 2024-09-21T02:55:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 -- 172.21.15.31:0/3344109015 shutdown_connections 2024-09-21T02:55:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 --2- 172.21.15.31:0/3344109015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860109df0 0x7fe860110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 --2- 172.21.15.31:0/3344109015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8601094a0 0x7fe8601098b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 --2- 172.21.15.31:0/3344109015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe860103b60 0x7fe860103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 -- 172.21.15.31:0/3344109015 >> 172.21.15.31:0/3344109015 conn(0x7fe86007b850 msgr2=0x7fe86007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:11.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 -- 172.21.15.31:0/3344109015 shutdown_connections 2024-09-21T02:55:11.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.084+0000 7fe866fac640 1 -- 172.21.15.31:0/3344109015 wait complete. 2024-09-21T02:55:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.085+0000 7fe866fac640 1 Processor -- start 2024-09-21T02:55:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.085+0000 7fe866fac640 1 -- start start 2024-09-21T02:55:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.085+0000 7fe866fac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 0x7fe86019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.085+0000 7fe866fac640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 0x7fe86019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.085+0000 7fe866fac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe860109df0 0x7fe86019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe866fac640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe860113ab0 con 0x7fe860109df0 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe866fac640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe860113930 con 0x7fe8601094a0 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 0x7fe86019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe864d48640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 0x7fe86019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe866fac640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe860113c30 con 0x7fe860103b60 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe864d48640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 0x7fe86019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:46618/0 (socket says 172.21.15.31:46618) 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe864d48640 1 -- 172.21.15.31:0/3580126557 learned_addr learned my addr 172.21.15.31:0/3580126557 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 0x7fe86019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:47564/0 (socket says 172.21.15.31:47564) 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe865549640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe860109df0 0x7fe86019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 -- 172.21.15.31:0/3580126557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 msgr2=0x7fe86019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 0x7fe86019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 -- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe860109df0 msgr2=0x7fe86019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe860109df0 0x7fe86019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe85ffff640 1 -- 172.21.15.31:0/3580126557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8601a03f0 con 0x7fe8601094a0 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe864d48640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 0x7fe86019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.086+0000 7fe865549640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe860109df0 0x7fe86019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.087+0000 7fe85ffff640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 0x7fe86019aa20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe84802a930 tx=0x7fe848031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.087+0000 7fe85dffb640 1 -- 172.21.15.31:0/3580126557 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84803f070 con 0x7fe8601094a0 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.087+0000 7fe85dffb640 1 -- 172.21.15.31:0/3580126557 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe848004490 con 0x7fe8601094a0 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.087+0000 7fe85dffb640 1 -- 172.21.15.31:0/3580126557 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848035070 con 0x7fe8601094a0 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.087+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8601a0680 con 0x7fe8601094a0 2024-09-21T02:55:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.087+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8600755f0 con 0x7fe8601094a0 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.090+0000 7fe86d520740 1 Processor -- start 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.090+0000 7fe86d520740 1 -- start start 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe86d520740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e6917070 0x5635e6cbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe86d520740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cbe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe86d520740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cbecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe86d520740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5635e6c34e50 con 0x5635e6917070 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe86d520740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5635e6c24280 con 0x5635e68f6390 2024-09-21T02:55:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe86d520740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5635e6c28090 con 0x5635e68ee1b0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe865549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e6917070 0x5635e6cbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe864d48640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cbe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe865549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e6917070 0x5635e6cbe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40530/0 (socket says 172.21.15.31:40530) 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe865549640 1 -- 172.21.15.31:0/3243423487 learned_addr learned my addr 172.21.15.31:0/3243423487 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe864d48640 1 -- 172.21.15.31:0/3243423487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 msgr2=0x5635e6cbecf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe864d48640 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cbecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe864d48640 1 -- 172.21.15.31:0/3243423487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e6917070 msgr2=0x5635e6cbe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe864d48640 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e6917070 0x5635e6cbe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.091+0000 7fe864d48640 1 -- 172.21.15.31:0/3243423487 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635e6982650 con 0x5635e68ee1b0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.092+0000 7fe85dffb640 1 -- 172.21.15.31:0/3580126557 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe84803b460 con 0x7fe8601094a0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.092+0000 7fe864d48640 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cbe7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe850011870 tx=0x7fe850011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b2c60f1507160e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.092+0000 7fe856ffd640 1 -- 172.21.15.31:0/3243423487 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500112f0 con 0x5635e68ee1b0 2024-09-21T02:55:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.092+0000 7fe856ffd640 1 -- 172.21.15.31:0/3243423487 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe850003e20 con 0x5635e68ee1b0 2024-09-21T02:55:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.092+0000 7fe856ffd640 1 -- 172.21.15.31:0/3243423487 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850004450 con 0x5635e68ee1b0 2024-09-21T02:55:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.092+0000 7fe85dffb640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe84c0b6700 0x7fe84c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe865549640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe84c0b6700 0x7fe84c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe85dffb640 1 -- 172.21.15.31:0/3580126557 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe848107b90 con 0x7fe8601094a0 2024-09-21T02:55:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe865549640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe84c0b6700 0x7fe84c0b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe858009580 tx=0x7fe858005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 -- 172.21.15.31:0/3243423487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 msgr2=0x5635e6cbe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cbe7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe850011870 tx=0x7fe850011d30 comp rx=0 tx=0).stop 2024-09-21T02:55:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 -- 172.21.15.31:0/3243423487 shutdown_connections 2024-09-21T02:55:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cbe7b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cbecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 --2- 172.21.15.31:0/3243423487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e6917070 0x5635e6cbe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 -- 172.21.15.31:0/3243423487 >> 172.21.15.31:0/3243423487 conn(0x5635e690b460 msgr2=0x5635e6cbc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 -- 172.21.15.31:0/3243423487 shutdown_connections 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.093+0000 7fe86d520740 1 -- 172.21.15.31:0/3243423487 wait complete. 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 -- start start 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 Processor -- start 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cc98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cc9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cd5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5635e6a85700 con 0x5635e68f6390 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5635e6bbf110 con 0x5635e6917070 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5635e6ccca50 con 0x5635e68ee1b0 2024-09-21T02:55:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.094+0000 7fe86d520740 10 client.0 _ll_register_callbacks cb 0x5635e6bd8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cc9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cc9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40546/0 (socket says 172.21.15.31:40546) 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe864d48640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cc98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 -- 172.21.15.31:0/705304588 learned_addr learned my addr 172.21.15.31:0/705304588 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe865549640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cd5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 -- 172.21.15.31:0/705304588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 msgr2=0x5635e6cc98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cc98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 -- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 msgr2=0x5635e6cd5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cd5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 -- 172.21.15.31:0/705304588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635e6becc40 con 0x5635e68f6390 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe864d48640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cc98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe85ffff640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cc9e20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe848105900 tx=0x7fe8480c9960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a10d7d8c75d602 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe83ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe83ffff640 1 -- 172.21.15.31:0/705304588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848048040 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe83ffff640 1 -- 172.21.15.31:0/705304588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe8480ccc60 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.095+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe850003e20 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe83ffff640 1 -- 172.21.15.31:0/705304588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8480cd170 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635e6cd8720 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe83ffff640 1 -- 172.21.15.31:0/705304588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe850003e20 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe83ffff640 1 -- 172.21.15.31:0/705304588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe84804b060 con 0x5635e68f6390 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 20 client.16164 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68d59f0 0x5635e6cdbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 4 client.16164 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.096+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635e6c36da0 con 0x5635e68d59f0 2024-09-21T02:55:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.097+0000 7fe864d48640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68d59f0 0x5635e6cdbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.097+0000 7fe83ffff640 10 client.16164 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.097+0000 7fe864d48640 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68d59f0 0x5635e6cdbb30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ebd46d83e1d57a server_cookie=5844ca3e3761d2b5 in_seq=0 out_seq=0 2024-09-21T02:55:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe83ffff640 1 -- 172.21.15.31:0/705304588 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635e6c36da0 con 0x5635e68d59f0 2024-09-21T02:55:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe83ffff640 10 client.16164 handle_command_reply: tid=0 2024-09-21T02:55:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 client.16164 shutdown 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 msgr2=0x5635e6cc9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cc9e20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe848105900 tx=0x7fe8480c9960 comp rx=0 tx=0).stop 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 shutdown_connections 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68d59f0 0x5635e6cdbb30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cd5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cc9e20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 --2- 172.21.15.31:0/705304588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6cc98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 >> 172.21.15.31:0/705304588 conn(0x5635e690b460 msgr2=0x5635e6cc1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.098+0000 7fe83ffff640 0 client.16164 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.099+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 shutdown_connections 2024-09-21T02:55:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.099+0000 7fe86d520740 1 -- 172.21.15.31:0/705304588 wait complete. 2024-09-21T02:55:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.099+0000 7fe86d520740 20 client.16164 trim_cache size 0 max 16384 2024-09-21T02:55:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.117+0000 7fe86d520740 1 Processor -- start 2024-09-21T02:55:11.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.117+0000 7fe86d520740 1 -- start start 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.117+0000 7fe86d520740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68ee1b0 0x5635e6ccb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.117+0000 7fe86d520740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cd5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.117+0000 7fe86d520740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e6917070 0x5635e6cc6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe86d520740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ad4a0 con 0x5635e68ee1b0 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe86d520740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601aefe0 con 0x5635e68f6390 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe86d520740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601adb70 con 0x5635e6917070 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cd5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe865549640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e6917070 0x5635e6cc6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cd5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:47576/0 (socket says 172.21.15.31:47576) 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 -- 172.21.15.31:0/2468997651 learned_addr learned my addr 172.21.15.31:0/2468997651 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe864d48640 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68ee1b0 0x5635e6ccb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 -- 172.21.15.31:0/2468997651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e6917070 msgr2=0x5635e6cc6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e6917070 0x5635e6cc6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 -- 172.21.15.31:0/2468997651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68ee1b0 msgr2=0x5635e6ccb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68ee1b0 0x5635e6ccb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 -- 172.21.15.31:0/2468997651 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635e6cca980 con 0x5635e68f6390 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe85ffff640 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cd5c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8480cac70 tx=0x7fe8480c99d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a55d34d87efec6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.118+0000 7fe8567fc640 1 -- 172.21.15.31:0/2468997651 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848048020 con 0x5635e68f6390 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe8567fc640 1 -- 172.21.15.31:0/2468997651 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8480cd510 con 0x5635e68f6390 2024-09-21T02:55:11.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe8567fc640 1 -- 172.21.15.31:0/2468997651 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8480c9170 con 0x5635e68f6390 2024-09-21T02:55:11.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 -- 172.21.15.31:0/2468997651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 msgr2=0x5635e6cd5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cd5c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8480cac70 tx=0x7fe8480c99d0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 -- 172.21.15.31:0/2468997651 shutdown_connections 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e6917070 0x5635e6cc6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e68f6390 0x5635e6cd5c40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 --2- 172.21.15.31:0/2468997651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68ee1b0 0x5635e6ccb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 -- 172.21.15.31:0/2468997651 >> 172.21.15.31:0/2468997651 conn(0x5635e690b460 msgr2=0x5635e6ccae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 -- 172.21.15.31:0/2468997651 shutdown_connections 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.119+0000 7fe86d520740 1 -- 172.21.15.31:0/2468997651 wait complete. 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 -- start start 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 Processor -- start 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6ccdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ae910 con 0x5635e68f6390 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ae240 con 0x5635e6917070 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.120+0000 7fe86d520740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601ab480 con 0x5635e68ee1b0 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe86d520740 10 client.0 _ll_register_callbacks cb 0x7fe8481057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe85ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40564/0 (socket says 172.21.15.31:40564) 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe85ffff640 1 -- 172.21.15.31:0/2777550946 learned_addr learned my addr 172.21.15.31:0/2777550946 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6ccdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe865549640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 -- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 msgr2=0x5635e6cce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 -- 172.21.15.31:0/2777550946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 msgr2=0x5635e6cce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 -- 172.21.15.31:0/2777550946 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635e6d454e0 con 0x5635e68ee1b0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe85ffff640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe864d48640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6ccdc30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe850000f80 tx=0x7fe850011530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5077735a6562e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe83ffff640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.121+0000 7fe83ffff640 1 -- 172.21.15.31:0/2777550946 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85000fbb0 con 0x5635e68ee1b0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8480cd510 con 0x5635e68ee1b0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe83ffff640 1 -- 172.21.15.31:0/2777550946 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe85000fd50 con 0x5635e68ee1b0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe83ffff640 1 -- 172.21.15.31:0/2777550946 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8500050f0 con 0x5635e68ee1b0 2024-09-21T02:55:11.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 10 client.25846 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635e6d46730 con 0x5635e68ee1b0 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe83ffff640 1 -- 172.21.15.31:0/2777550946 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8480cd510 con 0x5635e68ee1b0 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe83ffff640 1 -- 172.21.15.31:0/2777550946 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe85001a610 con 0x5635e68ee1b0 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 10 client.25846 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 20 client.25846 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68aada0 0x5635e6d48d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 4 client.25846 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.122+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635e6c32850 con 0x5635e68aada0 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe85ffff640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68aada0 0x5635e6d48d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:11.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe83ffff640 10 client.25846 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe85ffff640 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68aada0 0x5635e6d48d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f354fd3ad44ac13 server_cookie=510087e62fb3542b in_seq=0 out_seq=0 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe83ffff640 1 -- 172.21.15.31:0/2777550946 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x5635e6c32850 con 0x5635e68aada0 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe83ffff640 10 client.25846 handle_command_reply: tid=0 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe86d520740 1 client.25846 shutdown 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 msgr2=0x5635e6ccdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe86d520740 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6ccdc30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe850000f80 tx=0x7fe850011530 comp rx=0 tx=0).stop 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 shutdown_connections 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.123+0000 7fe86d520740 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5635e68aada0 0x5635e6d48d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5635e6917070 0x5635e6cce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe83ffff640 0 client.25846 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5635e68f6390 0x5635e6cce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 1 --2- 172.21.15.31:0/2777550946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5635e68ee1b0 0x5635e6ccdc30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 >> 172.21.15.31:0/2777550946 conn(0x5635e690b460 msgr2=0x5635e6ccaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 shutdown_connections 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 1 -- 172.21.15.31:0/2777550946 wait complete. 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe86d520740 20 client.25846 trim_cache size 0 max 16384 2024-09-21T02:55:11.131 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:55:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:55:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.124+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe84c0b6700 msgr2=0x7fe84c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe84c0b6700 0x7fe84c0b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe858009580 tx=0x7fe858005f90 comp rx=0 tx=0).stop 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 msgr2=0x7fe86019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 0x7fe86019aa20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe84802a930 tx=0x7fe848031a20 comp rx=0 tx=0).stop 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 shutdown_connections 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe84c0b6700 0x7fe84c0b8c10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe860109df0 0x7fe86019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8601094a0 0x7fe86019aa20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 --2- 172.21.15.31:0/3580126557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860103b60 0x7fe86019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 >> 172.21.15.31:0/3580126557 conn(0x7fe86007b850 msgr2=0x7fe860107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 shutdown_connections 2024-09-21T02:55:11.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:11.125+0000 7fe866fac640 1 -- 172.21.15.31:0/3580126557 wait complete. 2024-09-21T02:55:12.077 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:55:12.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:55:12.373 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:12 smithi031 ceph-mon[31209]: pgmap v394: 97 pgs: 97 active+clean; 260 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.2 MiB/s wr, 513 op/s 2024-09-21T02:55:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:12.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.874+0000 7f0f9b660640 1 -- 172.21.15.31:0/3542894007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 msgr2=0x7f0f941098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.874+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3542894007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f941098e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009900 tx=0x7f0f8402a410 comp rx=0 tx=0).stop 2024-09-21T02:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.874+0000 7f0f9b660640 1 -- 172.21.15.31:0/3542894007 shutdown_connections 2024-09-21T02:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.874+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3542894007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94109e20 0x7f0f94110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.874+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3542894007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f941098e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.874+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3542894007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103b90 0x7f0f94103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.875+0000 7f0f9b660640 1 -- 172.21.15.31:0/3542894007 >> 172.21.15.31:0/3542894007 conn(0x7f0f940ff600 msgr2=0x7f0f94101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.875+0000 7f0f9b660640 1 -- 172.21.15.31:0/3542894007 shutdown_connections 2024-09-21T02:55:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.875+0000 7f0f9b660640 1 -- 172.21.15.31:0/3542894007 wait complete. 2024-09-21T02:55:12.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.875+0000 7f0f9b660640 1 Processor -- start 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.875+0000 7f0f9b660640 1 -- start start 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f9b660640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 0x7f0f9419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f9b660640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f9419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f9b660640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94109e20 0x7f0f9419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f9b660640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94113b30 con 0x7f0f941094d0 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f9b660640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941139b0 con 0x7f0f94103b90 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f9b660640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94113cb0 con 0x7f0f94109e20 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 0x7f0f9419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f98bfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f9419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 0x7f0f9419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:47608/0 (socket says 172.21.15.31:47608) 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f98bfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f9419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40582/0 (socket says 172.21.15.31:40582) 2024-09-21T02:55:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 -- 172.21.15.31:0/3424857267 learned_addr learned my addr 172.21.15.31:0/3424857267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f99bfd640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94109e20 0x7f0f9419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 -- 172.21.15.31:0/3424857267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94109e20 msgr2=0x7f0f9419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94109e20 0x7f0f9419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 -- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 msgr2=0x7f0f9419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f9419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.876+0000 7f0f993fc640 1 -- 172.21.15.31:0/3424857267 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f941a0430 con 0x7f0f94103b90 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.877+0000 7f0f993fc640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 0x7f0f9419a4c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9f0 tx=0x7f0f9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.877+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000cd80 con 0x7f0f94103b90 2024-09-21T02:55:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.877+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f941a0720 con 0x7f0f94103b90 2024-09-21T02:55:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.877+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f900108e0 con 0x7f0f94103b90 2024-09-21T02:55:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.877+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90018980 con 0x7f0f94103b90 2024-09-21T02:55:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.877+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f941a6e80 con 0x7f0f94103b90 2024-09-21T02:55:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.878+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f94104e10 con 0x7f0f94103b90 2024-09-21T02:55:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.880+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0f90010430 con 0x7f0f94103b90 2024-09-21T02:55:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.881+0000 7f0f8a7fc640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f800b6680 0x7f0f800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.881+0000 7f0f98bfb640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f800b6680 0x7f0f800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.881+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0f900e7e30 con 0x7f0f94103b90 2024-09-21T02:55:12.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.882+0000 7f0f98bfb640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f800b6680 0x7f0f800b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f8402aa00 tx=0x7f0f84005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:12.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:12.884+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f900b2780 con 0x7f0f94103b90 2024-09-21T02:55:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:12 smithi165 ceph-mon[34818]: pgmap v394: 97 pgs: 97 active+clean; 260 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.2 MiB/s wr, 513 op/s 2024-09-21T02:55:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:12.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:55:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:12 smithi178 ceph-mon[34759]: pgmap v394: 97 pgs: 97 active+clean; 260 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.2 MiB/s wr, 513 op/s 2024-09-21T02:55:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:12.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:55:13.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.145+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f94103fa0 con 0x7f0f94103b90 2024-09-21T02:55:13.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.146+0000 7f0f8a7fc640 1 -- 172.21.15.31:0/3424857267 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f0f900b76c0 con 0x7f0f94103b90 2024-09-21T02:55:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:55:13.147 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:55:13.148 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f800b6680 msgr2=0x7f0f800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f800b6680 0x7f0f800b8b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f8402aa00 tx=0x7f0f84005a20 comp rx=0 tx=0).stop 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 msgr2=0x7f0f9419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 0x7f0f9419a4c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9f0 tx=0x7f0f9000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 shutdown_connections 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0f800b6680 0x7f0f800b8b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94109e20 0x7f0f9419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f941094d0 0x7f0f9419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:13.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 --2- 172.21.15.31:0/3424857267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f94103b90 0x7f0f9419a4c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 >> 172.21.15.31:0/3424857267 conn(0x7f0f940ff600 msgr2=0x7f0f94101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 shutdown_connections 2024-09-21T02:55:13.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:13.149+0000 7f0f9b660640 1 -- 172.21.15.31:0/3424857267 wait complete. 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:setting up random write file 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:done setting up random write file 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:starting fsync run 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0078s 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0052s 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0065s 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0062s 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0062s 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0053s 2024-09-21T02:55:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0063s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0047s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0041s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0041s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0043s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0042s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0043s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0053s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0061s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0069s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0063s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0053s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0047s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0060s 2024-09-21T02:55:13.710 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0045s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0047s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0055s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0046s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0044s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0046s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0050s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0050s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0047s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0042s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0042s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0038s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0047s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0005s fsync time: 0.0050s 2024-09-21T02:55:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0060s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0046s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0042s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0044s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0049s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0043s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0048s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0047s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0052s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0045s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0052s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0042s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0166s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0052s 2024-09-21T02:55:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0039s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0040s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0043s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0164s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0043s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0063s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0053s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0040s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0040s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0036s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0003s fsync time: 0.0046s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:write time: 0.0002s fsync time: 0.0151s 2024-09-21T02:55:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:run done 61 fsyncs total, killing random writer 2024-09-21T02:55:13.714 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-09-21T02:55:13.714 INFO:tasks.workunit.client.0.smithi031.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-09-21T02:55:13.714 INFO:tasks.workunit.client.0.smithi031.stderr:+ whereis lsof 2024-09-21T02:55:13.794 INFO:tasks.workunit.client.0.smithi031.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-09-21T02:55:13.794 INFO:tasks.workunit.client.0.smithi031.stderr:+ lsof 2024-09-21T02:55:13.834 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d665c4be05660e873d2afafb062cee877df69895f2bfc654cd31e2f6253977b8/merged 2024-09-21T02:55:13.834 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.834 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/5f24af81c9653d5ae3d807384ffac44e020ed126d20531f96d087b5fcc9e56b3/merged 2024-09-21T02:55:13.834 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.836 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/061a9b95de2277fe0606fc3f41d5fe341e58081bca11e67df30dd3e75323563b/merged 2024-09-21T02:55:13.836 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.836 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/aa7495fb2314802bbea454316b43be24b336f1e3d25f9855cf96acab05203f05/merged 2024-09-21T02:55:13.836 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.836 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/04f64830c88b240da0f97d84b2798d35dc3d7b49a743f95d503bb844a3f0cfe1/merged 2024-09-21T02:55:13.836 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:13 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3424857267' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:55:13.840 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fbd4f6cc8aec410de14d2bce00d23a5fd662ed93e4ce9188116d5cee1e508fa5/merged 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/ff3a9bb1de344086287f8cafbde2971ea714fb348b25ccab97f42ab67da9fd87/merged 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/cfeaabd79fd1917db14ff5f04550ff5ff736342f3483feaf29bbb4a81819561e/merged 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f79e37582398212688665aad43e9a979ef261a54eb0f707b75b5eb302eee9bf5/merged 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/5174d33c8c3013bd5b61344796fcc5f736397d0928dbd51202a4ecbbe265281f/merged 2024-09-21T02:55:13.841 INFO:tasks.workunit.client.0.smithi031.stderr: Output information may be incomplete. 2024-09-21T02:55:13.905 INFO:tasks.workunit.client.0.smithi031.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-09-21T02:55:13.905 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-09-21T02:55:13.906 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 7 root cwd unknown /proc/7/cwd (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 7 root rtd unknown /proc/7/root (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 7 root txt unknown /proc/7/exe (readlink: Permission denied) 2024-09-21T02:55:13.907 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 7 root NOFD /proc/7/fd (opendir: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-09-21T02:55:13.908 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_tasks 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-09-21T02:55:13.909 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:pr/ttyS1 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:pr/ttyS1 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:pr/ttyS1 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:pr/ttyS1 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_preem 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_preem 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_preem 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:rcu_preem 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-09-21T02:55:13.910 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/0 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/0 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/0 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/0 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/1 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/1 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/1 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/1 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-09-21T02:55:13.911 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:migration 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:migration 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:migration 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:migration 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-09-21T02:55:13.912 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:migration 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:migration 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:migration 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:migration 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-09-21T02:55:13.913 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-09-21T02:55:13.914 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-09-21T02:55:13.915 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/4 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/4 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/4 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/4 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:migration 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:migration 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:migration 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:migration 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-09-21T02:55:13.916 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/5 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/5 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/5 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/5 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:migration 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-09-21T02:55:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:migration 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:migration 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:migration 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/6 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/6 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-09-21T02:55:13.918 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/6 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/6 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:migration 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:migration 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:migration 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:migration 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-09-21T02:55:13.919 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/7 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/7 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/7 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:cpuhp/7 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:idle_inje 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:migration 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:migration 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-09-21T02:55:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:migration 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:migration 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 62 root cwd unknown /proc/62/cwd (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 62 root rtd unknown /proc/62/root (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 62 root txt unknown /proc/62/exe (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:ksoftirqd 62 root NOFD /proc/62/fd (opendir: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-09-21T02:55:13.921 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kdevtmpfs 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kdevtmpfs 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kdevtmpfs 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kdevtmpfs 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kauditd 74 root cwd unknown /proc/74/cwd (readlink: Permission denied) 2024-09-21T02:55:13.922 INFO:tasks.workunit.client.0.smithi031.stdout:kauditd 74 root rtd unknown /proc/74/root (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:kauditd 74 root txt unknown /proc/74/exe (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:kauditd 74 root NOFD /proc/74/fd (opendir: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:khungtask 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:khungtask 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:khungtask 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:khungtask 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:oom_reape 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:oom_reape 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:oom_reape 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:oom_reape 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-09-21T02:55:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:kcompactd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:kcompactd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:kcompactd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:kcompactd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:ksmd 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:ksmd 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:ksmd 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:ksmd 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:khugepage 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:khugepage 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:khugepage 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-09-21T02:55:13.924 INFO:tasks.workunit.client.0.smithi031.stdout:khugepage 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-09-21T02:55:13.925 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-09-21T02:55:13.926 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:watchdogd 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:watchdogd 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:watchdogd 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:watchdogd 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kswapd0 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-09-21T02:55:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:kswapd0 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kswapd0 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kswapd0 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 101 root cwd unknown /proc/101/cwd (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 101 root rtd unknown /proc/101/root (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 101 root txt unknown /proc/101/exe (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 101 root NOFD /proc/101/fd (opendir: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-09-21T02:55:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-09-21T02:55:13.929 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 125 root cwd unknown /proc/125/cwd (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 125 root rtd unknown /proc/125/root (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 125 root txt unknown /proc/125/exe (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 125 root NOFD /proc/125/fd (opendir: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 239 root cwd unknown /proc/239/cwd (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 239 root rtd unknown /proc/239/root (readlink: Permission denied) 2024-09-21T02:55:13.930 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 239 root txt unknown /proc/239/exe (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 239 root NOFD /proc/239/fd (opendir: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 240 root cwd unknown /proc/240/cwd (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 240 root rtd unknown /proc/240/root (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 240 root txt unknown /proc/240/exe (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 240 root NOFD /proc/240/fd (opendir: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 241 root cwd unknown /proc/241/cwd (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 241 root rtd unknown /proc/241/root (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 241 root txt unknown /proc/241/exe (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 241 root NOFD /proc/241/fd (opendir: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 280 root cwd unknown /proc/280/cwd (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 280 root rtd unknown /proc/280/root (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 280 root txt unknown /proc/280/exe (readlink: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 280 root NOFD /proc/280/fd (opendir: Permission denied) 2024-09-21T02:55:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 285 root cwd unknown /proc/285/cwd (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 285 root rtd unknown /proc/285/root (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 285 root txt unknown /proc/285/exe (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 285 root NOFD /proc/285/fd (opendir: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 293 root cwd unknown /proc/293/cwd (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 293 root rtd unknown /proc/293/root (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 293 root txt unknown /proc/293/exe (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 293 root NOFD /proc/293/fd (opendir: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 383 root cwd unknown /proc/383/cwd (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 383 root rtd unknown /proc/383/root (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 383 root txt unknown /proc/383/exe (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 383 root NOFD /proc/383/fd (opendir: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 433 root cwd unknown /proc/433/cwd (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 433 root rtd unknown /proc/433/root (readlink: Permission denied) 2024-09-21T02:55:13.932 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 433 root txt unknown /proc/433/exe (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 433 root NOFD /proc/433/fd (opendir: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 437 root cwd unknown /proc/437/cwd (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 437 root rtd unknown /proc/437/root (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 437 root txt unknown /proc/437/exe (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 437 root NOFD /proc/437/fd (opendir: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 438 root cwd unknown /proc/438/cwd (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 438 root rtd unknown /proc/438/root (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 438 root txt unknown /proc/438/exe (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 438 root NOFD /proc/438/fd (opendir: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 439 root cwd unknown /proc/439/cwd (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 439 root rtd unknown /proc/439/root (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 439 root txt unknown /proc/439/exe (readlink: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 439 root NOFD /proc/439/fd (opendir: Permission denied) 2024-09-21T02:55:13.933 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 440 root cwd unknown /proc/440/cwd (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 440 root rtd unknown /proc/440/root (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 440 root txt unknown /proc/440/exe (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 440 root NOFD /proc/440/fd (opendir: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 442 root cwd unknown /proc/442/cwd (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 442 root rtd unknown /proc/442/root (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 442 root txt unknown /proc/442/exe (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 442 root NOFD /proc/442/fd (opendir: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_0 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_0 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_0 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_0 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-09-21T02:55:13.934 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_1 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_1 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_1 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_1 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_2 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_2 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_2 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_2 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-09-21T02:55:13.935 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_3 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_3 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_3 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_3 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_4 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_4 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-09-21T02:55:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_4 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_4 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_5 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_5 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_5 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_5 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-09-21T02:55:13.937 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_6 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_6 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_6 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_6 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_7 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_7 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_7 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_7 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-09-21T02:55:13.938 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_8 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_8 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_8 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_8 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_9 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_9 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_9 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-09-21T02:55:13.939 INFO:tasks.workunit.client.0.smithi031.stdout:scsi_eh_9 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 470 root cwd unknown /proc/470/cwd (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 470 root rtd unknown /proc/470/root (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 470 root txt unknown /proc/470/exe (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 470 root NOFD /proc/470/fd (opendir: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 499 root cwd unknown /proc/499/cwd (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 499 root rtd unknown /proc/499/root (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 499 root txt unknown /proc/499/exe (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 499 root NOFD /proc/499/fd (opendir: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:jbd2/sda1 500 root cwd unknown /proc/500/cwd (readlink: Permission denied) 2024-09-21T02:55:13.940 INFO:tasks.workunit.client.0.smithi031.stdout:jbd2/sda1 500 root rtd unknown /proc/500/root (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:jbd2/sda1 500 root txt unknown /proc/500/exe (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:jbd2/sda1 500 root NOFD /proc/500/fd (opendir: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 501 root cwd unknown /proc/501/cwd (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 501 root rtd unknown /proc/501/root (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 501 root txt unknown /proc/501/exe (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 501 root NOFD /proc/501/fd (opendir: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 562 root cwd unknown /proc/562/cwd (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 562 root rtd unknown /proc/562/root (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 562 root txt unknown /proc/562/exe (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 562 root NOFD /proc/562/fd (opendir: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-j 577 root cwd unknown /proc/577/cwd (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-j 577 root rtd unknown /proc/577/root (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-j 577 root txt unknown /proc/577/exe (readlink: Permission denied) 2024-09-21T02:55:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-j 577 root NOFD /proc/577/fd (opendir: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:ext4lazyi 583 root cwd unknown /proc/583/cwd (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:ext4lazyi 583 root rtd unknown /proc/583/root (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:ext4lazyi 583 root txt unknown /proc/583/exe (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:ext4lazyi 583 root NOFD /proc/583/fd (opendir: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-u 594 root cwd unknown /proc/594/cwd (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-u 594 root rtd unknown /proc/594/root (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-u 594 root txt unknown /proc/594/exe (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-u 594 root NOFD /proc/594/fd (opendir: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 643 root cwd unknown /proc/643/cwd (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 643 root rtd unknown /proc/643/root (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 643 root txt unknown /proc/643/exe (readlink: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 643 root NOFD /proc/643/fd (opendir: Permission denied) 2024-09-21T02:55:13.942 INFO:tasks.workunit.client.0.smithi031.stdout:kipmi0 651 root cwd unknown /proc/651/cwd (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kipmi0 651 root rtd unknown /proc/651/root (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kipmi0 651 root txt unknown /proc/651/exe (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kipmi0 651 root NOFD /proc/651/fd (opendir: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:rpcbind 671 rpc cwd unknown /proc/671/cwd (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:rpcbind 671 rpc rtd unknown /proc/671/root (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:rpcbind 671 rpc txt unknown /proc/671/exe (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:rpcbind 671 rpc NOFD /proc/671/fd (opendir: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 674 root cwd unknown /proc/674/cwd (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 674 root rtd unknown /proc/674/root (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 674 root txt unknown /proc/674/exe (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 674 root NOFD /proc/674/fd (opendir: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 675 root cwd unknown /proc/675/cwd (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 675 root rtd unknown /proc/675/root (readlink: Permission denied) 2024-09-21T02:55:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 675 root txt unknown /proc/675/exe (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 675 root NOFD /proc/675/fd (opendir: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 root cwd unknown /proc/692/cwd (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 root rtd unknown /proc/692/root (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 root txt unknown /proc/692/exe (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 root NOFD /proc/692/fd (opendir: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 693 auditd root cwd unknown /proc/692/task/693/cwd (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 693 auditd root rtd unknown /proc/692/task/693/root (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 693 auditd root txt unknown /proc/692/task/693/exe (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 693 auditd root NOFD /proc/692/task/693/fd (opendir: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 695 auditd root cwd unknown /proc/692/task/695/cwd (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 695 auditd root rtd unknown /proc/692/task/695/root (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 695 auditd root txt unknown /proc/692/task/695/exe (readlink: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:auditd 692 695 auditd root NOFD /proc/692/task/695/fd (opendir: Permission denied) 2024-09-21T02:55:13.944 INFO:tasks.workunit.client.0.smithi031.stdout:sedispatc 694 root cwd unknown /proc/694/cwd (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:sedispatc 694 root rtd unknown /proc/694/root (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:sedispatc 694 root txt unknown /proc/694/exe (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:sedispatc 694 root NOFD /proc/694/fd (opendir: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 720 dbus cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 720 dbus rtd unknown /proc/720/root (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 720 dbus txt unknown /proc/720/exe (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 720 dbus NOFD /proc/720/fd (opendir: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 721 dbus cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 721 dbus rtd unknown /proc/721/root (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 721 dbus txt unknown /proc/721/exe (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:dbus-brok 721 dbus NOFD /proc/721/fd (opendir: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-09-21T02:55:13.945 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 739 gmain root cwd unknown /proc/722/task/739/cwd (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 739 gmain root rtd unknown /proc/722/task/739/root (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 739 gmain root txt unknown /proc/722/task/739/exe (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 739 gmain root NOFD /proc/722/task/739/fd (opendir: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 740 gdbus root cwd unknown /proc/722/task/740/cwd (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 740 gdbus root rtd unknown /proc/722/task/740/root (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 740 gdbus root txt unknown /proc/722/task/740/exe (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:NetworkMa 722 740 gdbus root NOFD /proc/722/task/740/fd (opendir: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-09-21T02:55:13.946 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 732 gmain root cwd unknown /proc/725/task/732/cwd (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 732 gmain root rtd unknown /proc/725/task/732/root (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 732 gmain root txt unknown /proc/725/task/732/exe (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:irqbalanc 725 732 gmain root NOFD /proc/725/task/732/fd (opendir: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:lsmd 726 libstoragemgmt cwd unknown /proc/726/cwd (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:lsmd 726 libstoragemgmt rtd unknown /proc/726/root (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:lsmd 726 libstoragemgmt txt unknown /proc/726/exe (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:lsmd 726 libstoragemgmt NOFD /proc/726/fd (opendir: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:mcelog 727 root cwd unknown /proc/727/cwd (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:mcelog 727 root rtd unknown /proc/727/root (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:mcelog 727 root txt unknown /proc/727/exe (readlink: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:mcelog 727 root NOFD /proc/727/fd (opendir: Permission denied) 2024-09-21T02:55:13.947 INFO:tasks.workunit.client.0.smithi031.stdout:smartd 728 root cwd unknown /proc/728/cwd (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:smartd 728 root rtd unknown /proc/728/root (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:smartd 728 root txt unknown /proc/728/exe (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:smartd 728 root NOFD /proc/728/fd (opendir: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-l 731 root cwd unknown /proc/731/cwd (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-l 731 root rtd unknown /proc/731/root (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-l 731 root txt unknown /proc/731/exe (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:systemd-l 731 root NOFD /proc/731/fd (opendir: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 774 root cwd unknown /proc/774/cwd (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 774 root rtd unknown /proc/774/root (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 774 root txt unknown /proc/774/exe (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 774 root NOFD /proc/774/fd (opendir: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 root cwd unknown /proc/778/cwd (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 root rtd unknown /proc/778/root (readlink: Permission denied) 2024-09-21T02:55:13.948 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 root txt unknown /proc/778/exe (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 root NOFD /proc/778/fd (opendir: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 779 gssproxy root cwd unknown /proc/778/task/779/cwd (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 779 gssproxy root rtd unknown /proc/778/task/779/root (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 779 gssproxy root txt unknown /proc/778/task/779/exe (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 779 gssproxy root NOFD /proc/778/task/779/fd (opendir: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 780 gssproxy root cwd unknown /proc/778/task/780/cwd (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 780 gssproxy root rtd unknown /proc/778/task/780/root (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 780 gssproxy root txt unknown /proc/778/task/780/exe (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 780 gssproxy root NOFD /proc/778/task/780/fd (opendir: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 781 gssproxy root cwd unknown /proc/778/task/781/cwd (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 781 gssproxy root rtd unknown /proc/778/task/781/root (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 781 gssproxy root txt unknown /proc/778/task/781/exe (readlink: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 781 gssproxy root NOFD /proc/778/task/781/fd (opendir: Permission denied) 2024-09-21T02:55:13.949 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 782 gssproxy root cwd unknown /proc/778/task/782/cwd (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 782 gssproxy root rtd unknown /proc/778/task/782/root (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 782 gssproxy root txt unknown /proc/778/task/782/exe (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 782 gssproxy root NOFD /proc/778/task/782/fd (opendir: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 783 gssproxy root cwd unknown /proc/778/task/783/cwd (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 783 gssproxy root rtd unknown /proc/778/task/783/root (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 783 gssproxy root txt unknown /proc/778/task/783/exe (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:gssproxy 778 783 gssproxy root NOFD /proc/778/task/783/fd (opendir: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:atd 785 root cwd unknown /proc/785/cwd (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:atd 785 root rtd unknown /proc/785/root (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:atd 785 root txt unknown /proc/785/exe (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:atd 785 root NOFD /proc/785/fd (opendir: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:crond 786 root cwd unknown /proc/786/cwd (readlink: Permission denied) 2024-09-21T02:55:13.950 INFO:tasks.workunit.client.0.smithi031.stdout:crond 786 root rtd unknown /proc/786/root (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:crond 786 root txt unknown /proc/786/exe (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:crond 786 root NOFD /proc/786/fd (opendir: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:nrpe 810 nrpe cwd unknown /proc/810/cwd (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:nrpe 810 nrpe rtd unknown /proc/810/root (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:nrpe 810 nrpe txt unknown /proc/810/exe (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:nrpe 810 nrpe NOFD /proc/810/fd (opendir: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1190 root cwd unknown /proc/1190/cwd (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1190 root rtd unknown /proc/1190/root (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1190 root txt unknown /proc/1190/exe (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1190 root NOFD /proc/1190/fd (opendir: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 1191 root cwd unknown /proc/1191/cwd (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 1191 root rtd unknown /proc/1191/root (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 1191 root txt unknown /proc/1191/exe (readlink: Permission denied) 2024-09-21T02:55:13.951 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 1191 root NOFD /proc/1191/fd (opendir: Permission denied) 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1193 root cwd unknown /proc/1193/cwd (readlink: Permission denied) 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1193 root rtd unknown /proc/1193/root (readlink: Permission denied) 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1193 root txt unknown /proc/1193/exe (readlink: Permission denied) 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:agetty 1193 root NOFD /proc/1193/fd (opendir: Permission denied) 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu cwd DIR 8,1 4096 2 / 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu txt REG 8,1 98208 22394 /usr/lib/systemd/systemd 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu DEL REG 8,1 2229 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 153600 11397 /usr/lib64/libgpg-error.so.0.32.0 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 912440 11215 /usr/lib64/libm.so.6 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 1714208 11376 /usr/lib64/libp11-kit.so.0.3.1 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu DEL REG 8,1 11595 /usr/lib64/libcrypto.so.3.2.2 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu DEL REG 8,1 2232 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 882376 11287 /usr/lib64/libzstd.so.1.5.1 2024-09-21T02:55:13.952 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 636848 11327 /usr/lib64/libpcre2-8.so.0.11.0 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 102672 11274 /usr/lib64/libz.so.1.2.11 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 1304864 11471 /usr/lib64/libgcrypt.so.20.4.0 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 3826840 50305 /usr/lib64/systemd/libsystemd-shared-252.so 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 44784 11373 /usr/lib64/libffi.so.8.1.0 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 178728 11282 /usr/lib64/liblzma.so.5.2.5 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 144120 11455 /usr/lib64/liblz4.so.1.9.3 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 2162040 50302 /usr/lib64/systemd/libsystemd-core-252.so 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 28552 11392 /usr/lib64/libattr.so.1.1.2501 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 32528 11311 /usr/lib64/libcap-ng.so.0.0.0 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 40520 12544 /usr/lib64/libeconf.so.0.4.1 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 201808 11284 /usr/lib64/libcrypt.so.2.0.0 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 36296 11290 /usr/lib64/libcap.so.2.48 2024-09-21T02:55:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 229424 11714 /usr/lib64/libblkid.so.1.1.0 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 40496 11393 /usr/lib64/libacl.so.1.1.2301 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 291496 11744 /usr/lib64/libmount.so.1.1.0 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 175744 11412 /usr/lib64/libselinux.so.1 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 107528 11768 /usr/lib64/libkmod.so.2.3.6 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 163464 11315 /usr/lib64/libaudit.so.1.0.0 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 69592 12553 /usr/lib64/libpam.so.0.85.1 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 127680 11560 /usr/lib64/libseccomp.so.2.5.2 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 108136 14698 /usr/lib64/libgcc_s-11-20240719.so.1 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 1u unix 0x0000000000000000 0t0 19913 type=STREAM (CONNECTED) 2024-09-21T02:55:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 2u unix 0x0000000000000000 0t0 19913 type=STREAM (CONNECTED) 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 3u unix 0x0000000000000000 0t0 19941 type=DGRAM (CONNECTED) 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 4u a_inode 0,14 0 2081 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 5u a_inode 0,14 0 2081 [signalfd] 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 6r a_inode 0,14 0 2081 inotify 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 7r DIR 0,25 0 3417 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 8u a_inode 0,14 0 2081 [timerfd] 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 9u a_inode 0,14 0 2081 [eventpoll:10,13] 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 10r REG 0,20 0 19930 /proc/4833/mountinfo 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 11r a_inode 0,14 0 2081 inotify 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 12u unix 0x0000000000000000 0t0 19963 /run/user/1000/bus type=STREAM (LISTEN) 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 13r a_inode 0,14 0 2081 inotify 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 14r REG 0,20 0 4026532068 /proc/swaps 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 15u netlink 0t0 19945 KOBJECT_UEVENT 2024-09-21T02:55:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 16u unix 0x0000000000000000 0t0 19950 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 17u unix 0x0000000000000000 0t0 19951 type=DGRAM (CONNECTED) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 18u unix 0x0000000000000000 0t0 19952 type=DGRAM (CONNECTED) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 19u unix 0x0000000000000000 0t0 19953 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 20u unix 0x0000000000000000 0t0 19955 type=STREAM (CONNECTED) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 4833 ubuntu 21u a_inode 0,14 0 2081 [timerfd] 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 4835 ubuntu cwd unknown /proc/4835/cwd (readlink: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 4835 ubuntu rtd unknown /proc/4835/root (readlink: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 4835 ubuntu txt unknown /proc/4835/exe (readlink: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 4835 ubuntu NOFD /proc/4835/fd (opendir: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 root cwd unknown /proc/5078/cwd (readlink: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 root rtd unknown /proc/5078/root (readlink: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 root txt unknown /proc/5078/exe (readlink: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 root NOFD /proc/5078/fd (opendir: Permission denied) 2024-09-21T02:55:13.956 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5080 in:imjour root cwd unknown /proc/5078/task/5080/cwd (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5080 in:imjour root rtd unknown /proc/5078/task/5080/root (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5080 in:imjour root txt unknown /proc/5078/task/5080/exe (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5080 in:imjour root NOFD /proc/5078/task/5080/fd (opendir: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5081 rs:main root cwd unknown /proc/5078/task/5081/cwd (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5081 rs:main root rtd unknown /proc/5078/task/5081/root (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5081 rs:main root txt unknown /proc/5078/task/5081/exe (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:rsyslogd 5078 5081 rs:main root NOFD /proc/5078/task/5081/fd (opendir: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 18664 root cwd unknown /proc/18664/cwd (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 18664 root rtd unknown /proc/18664/root (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 18664 root txt unknown /proc/18664/exe (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 18664 root NOFD /proc/18664/fd (opendir: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 polkitd cwd unknown /proc/19227/cwd (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 polkitd rtd unknown /proc/19227/root (readlink: Permission denied) 2024-09-21T02:55:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 polkitd txt unknown /proc/19227/exe (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 polkitd NOFD /proc/19227/fd (opendir: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19229 gmain polkitd cwd unknown /proc/19227/task/19229/cwd (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19229 gmain polkitd rtd unknown /proc/19227/task/19229/root (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19229 gmain polkitd txt unknown /proc/19227/task/19229/exe (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19229 gmain polkitd NOFD /proc/19227/task/19229/fd (opendir: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19230 gdbus polkitd cwd unknown /proc/19227/task/19230/cwd (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19230 gdbus polkitd rtd unknown /proc/19227/task/19230/root (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19230 gdbus polkitd txt unknown /proc/19227/task/19230/exe (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19230 gdbus polkitd NOFD /proc/19227/task/19230/fd (opendir: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19231 JS\x20Hel polkitd cwd unknown /proc/19227/task/19231/cwd (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19231 JS\x20Hel polkitd rtd unknown /proc/19227/task/19231/root (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19231 JS\x20Hel polkitd txt unknown /proc/19227/task/19231/exe (readlink: Permission denied) 2024-09-21T02:55:13.958 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19231 JS\x20Hel polkitd NOFD /proc/19227/task/19231/fd (opendir: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19232 JS\x20Hel polkitd cwd unknown /proc/19227/task/19232/cwd (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19232 JS\x20Hel polkitd rtd unknown /proc/19227/task/19232/root (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19232 JS\x20Hel polkitd txt unknown /proc/19227/task/19232/exe (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19232 JS\x20Hel polkitd NOFD /proc/19227/task/19232/fd (opendir: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19233 JS\x20Hel polkitd cwd unknown /proc/19227/task/19233/cwd (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19233 JS\x20Hel polkitd rtd unknown /proc/19227/task/19233/root (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19233 JS\x20Hel polkitd txt unknown /proc/19227/task/19233/exe (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19233 JS\x20Hel polkitd NOFD /proc/19227/task/19233/fd (opendir: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19234 JS\x20Hel polkitd cwd unknown /proc/19227/task/19234/cwd (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19234 JS\x20Hel polkitd rtd unknown /proc/19227/task/19234/root (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19234 JS\x20Hel polkitd txt unknown /proc/19227/task/19234/exe (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19234 JS\x20Hel polkitd NOFD /proc/19227/task/19234/fd (opendir: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19235 JS\x20Hel polkitd cwd unknown /proc/19227/task/19235/cwd (readlink: Permission denied) 2024-09-21T02:55:13.959 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19235 JS\x20Hel polkitd rtd unknown /proc/19227/task/19235/root (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19235 JS\x20Hel polkitd txt unknown /proc/19227/task/19235/exe (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19235 JS\x20Hel polkitd NOFD /proc/19227/task/19235/fd (opendir: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19236 JS\x20Hel polkitd cwd unknown /proc/19227/task/19236/cwd (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19236 JS\x20Hel polkitd rtd unknown /proc/19227/task/19236/root (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19236 JS\x20Hel polkitd txt unknown /proc/19227/task/19236/exe (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19236 JS\x20Hel polkitd NOFD /proc/19227/task/19236/fd (opendir: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19237 JS\x20Hel polkitd cwd unknown /proc/19227/task/19237/cwd (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19237 JS\x20Hel polkitd rtd unknown /proc/19227/task/19237/root (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19237 JS\x20Hel polkitd txt unknown /proc/19227/task/19237/exe (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19237 JS\x20Hel polkitd NOFD /proc/19227/task/19237/fd (opendir: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19238 JS\x20Hel polkitd cwd unknown /proc/19227/task/19238/cwd (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19238 JS\x20Hel polkitd rtd unknown /proc/19227/task/19238/root (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19238 JS\x20Hel polkitd txt unknown /proc/19227/task/19238/exe (readlink: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19238 JS\x20Hel polkitd NOFD /proc/19227/task/19238/fd (opendir: Permission denied) 2024-09-21T02:55:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19239 polkitd polkitd cwd unknown /proc/19227/task/19239/cwd (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19239 polkitd polkitd rtd unknown /proc/19227/task/19239/root (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19239 polkitd polkitd txt unknown /proc/19227/task/19239/exe (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:polkitd 19227 19239 polkitd polkitd NOFD /proc/19227/task/19239/fd (opendir: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21134 root cwd unknown /proc/21134/cwd (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21134 root rtd unknown /proc/21134/root (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21134 root txt unknown /proc/21134/exe (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21134 root NOFD /proc/21134/fd (opendir: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21163 root cwd unknown /proc/21163/cwd (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21163 root rtd unknown /proc/21163/root (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21163 root txt unknown /proc/21163/exe (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21163 root NOFD /proc/21163/fd (opendir: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21192 root cwd unknown /proc/21192/cwd (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21192 root rtd unknown /proc/21192/root (readlink: Permission denied) 2024-09-21T02:55:13.961 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21192 root txt unknown /proc/21192/exe (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21192 root NOFD /proc/21192/fd (opendir: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21221 root cwd unknown /proc/21221/cwd (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21221 root rtd unknown /proc/21221/root (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21221 root txt unknown /proc/21221/exe (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21221 root NOFD /proc/21221/fd (opendir: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21250 root cwd unknown /proc/21250/cwd (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21250 root rtd unknown /proc/21250/root (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21250 root txt unknown /proc/21250/exe (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21250 root NOFD /proc/21250/fd (opendir: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21414 root cwd unknown /proc/21414/cwd (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21414 root rtd unknown /proc/21414/root (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21414 root txt unknown /proc/21414/exe (readlink: Permission denied) 2024-09-21T02:55:13.962 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21414 root NOFD /proc/21414/fd (opendir: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21415 root cwd unknown /proc/21415/cwd (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21415 root rtd unknown /proc/21415/root (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21415 root txt unknown /proc/21415/exe (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21415 root NOFD /proc/21415/fd (opendir: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21416 root cwd unknown /proc/21416/cwd (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21416 root rtd unknown /proc/21416/root (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21416 root txt unknown /proc/21416/exe (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21416 root NOFD /proc/21416/fd (opendir: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21417 root cwd unknown /proc/21417/cwd (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21417 root rtd unknown /proc/21417/root (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21417 root txt unknown /proc/21417/exe (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21417 root NOFD /proc/21417/fd (opendir: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21418 root cwd unknown /proc/21418/cwd (readlink: Permission denied) 2024-09-21T02:55:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21418 root rtd unknown /proc/21418/root (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21418 root txt unknown /proc/21418/exe (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21418 root NOFD /proc/21418/fd (opendir: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21419 root cwd unknown /proc/21419/cwd (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21419 root rtd unknown /proc/21419/root (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21419 root txt unknown /proc/21419/exe (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21419 root NOFD /proc/21419/fd (opendir: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21420 root cwd unknown /proc/21420/cwd (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21420 root rtd unknown /proc/21420/root (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21420 root txt unknown /proc/21420/exe (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21420 root NOFD /proc/21420/fd (opendir: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21421 root cwd unknown /proc/21421/cwd (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21421 root rtd unknown /proc/21421/root (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21421 root txt unknown /proc/21421/exe (readlink: Permission denied) 2024-09-21T02:55:13.964 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21421 root NOFD /proc/21421/fd (opendir: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21422 root cwd unknown /proc/21422/cwd (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21422 root rtd unknown /proc/21422/root (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21422 root txt unknown /proc/21422/exe (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 21422 root NOFD /proc/21422/fd (opendir: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:xfsaild/d 21423 root cwd unknown /proc/21423/cwd (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:xfsaild/d 21423 root rtd unknown /proc/21423/root (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:xfsaild/d 21423 root txt unknown /proc/21423/exe (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:xfsaild/d 21423 root NOFD /proc/21423/fd (opendir: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22343 root cwd unknown /proc/22343/cwd (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22343 root rtd unknown /proc/22343/root (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22343 root txt unknown /proc/22343/exe (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22343 root NOFD /proc/22343/fd (opendir: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22346 ubuntu cwd unknown /proc/22346/cwd (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22346 ubuntu rtd unknown /proc/22346/root (readlink: Permission denied) 2024-09-21T02:55:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22346 ubuntu txt unknown /proc/22346/exe (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 22346 ubuntu NOFD /proc/22346/fd (opendir: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:chronyd 22406 chrony cwd unknown /proc/22406/cwd (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:chronyd 22406 chrony rtd unknown /proc/22406/root (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:chronyd 22406 chrony txt unknown /proc/22406/exe (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:chronyd 22406 chrony NOFD /proc/22406/fd (opendir: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-cras 22727 ceph cwd unknown /proc/22727/cwd (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-cras 22727 ceph rtd unknown /proc/22727/root (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-cras 22727 ceph txt unknown /proc/22727/exe (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-cras 22727 ceph NOFD /proc/22727/fd (opendir: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 24432 root cwd unknown /proc/24432/cwd (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 24432 root rtd unknown /proc/24432/root (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 24432 root txt unknown /proc/24432/exe (readlink: Permission denied) 2024-09-21T02:55:13.966 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 24432 root NOFD /proc/24432/fd (opendir: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25289 root cwd unknown /proc/25289/cwd (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25289 root rtd unknown /proc/25289/root (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25289 root txt unknown /proc/25289/exe (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25289 root NOFD /proc/25289/fd (opendir: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25291 root cwd unknown /proc/25291/cwd (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25291 root rtd unknown /proc/25291/root (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25291 root txt unknown /proc/25291/exe (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 25291 root NOFD /proc/25291/fd (opendir: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 29807 root cwd unknown /proc/29807/cwd (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 29807 root rtd unknown /proc/29807/root (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 29807 root txt unknown /proc/29807/exe (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 29807 root NOFD /proc/29807/fd (opendir: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30206 root cwd unknown /proc/30206/cwd (readlink: Permission denied) 2024-09-21T02:55:13.967 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30206 root rtd unknown /proc/30206/root (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30206 root txt unknown /proc/30206/exe (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30206 root NOFD /proc/30206/fd (opendir: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30223 root cwd unknown /proc/30223/cwd (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30223 root rtd unknown /proc/30223/root (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30223 root txt unknown /proc/30223/exe (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30223 root NOFD /proc/30223/fd (opendir: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30224 root cwd unknown /proc/30224/cwd (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30224 root rtd unknown /proc/30224/root (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30224 root txt unknown /proc/30224/exe (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 30224 root NOFD /proc/30224/fd (opendir: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30241 root cwd unknown /proc/30241/cwd (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30241 root rtd unknown /proc/30241/root (readlink: Permission denied) 2024-09-21T02:55:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30241 root txt unknown /proc/30241/exe (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 30241 root NOFD /proc/30241/fd (opendir: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 30587 root cwd unknown /proc/30587/cwd (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 30587 root rtd unknown /proc/30587/root (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 30587 root txt unknown /proc/30587/exe (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 30587 root NOFD /proc/30587/fd (opendir: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 31058 root cwd unknown /proc/31058/cwd (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 31058 root rtd unknown /proc/31058/root (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 31058 root txt unknown /proc/31058/exe (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 31058 root NOFD /proc/31058/fd (opendir: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31205 root cwd unknown /proc/31205/cwd (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31205 root rtd unknown /proc/31205/root (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31205 root txt unknown /proc/31205/exe (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31205 root NOFD /proc/31205/fd (opendir: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31207 root cwd unknown /proc/31207/cwd (readlink: Permission denied) 2024-09-21T02:55:13.969 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31207 root rtd unknown /proc/31207/root (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31207 root txt unknown /proc/31207/exe (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31207 root NOFD /proc/31207/fd (opendir: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 ceph cwd unknown /proc/31209/cwd (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 ceph rtd unknown /proc/31209/root (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 ceph txt unknown /proc/31209/exe (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 ceph NOFD /proc/31209/fd (opendir: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31210 log ceph cwd unknown /proc/31209/task/31210/cwd (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31210 log ceph rtd unknown /proc/31209/task/31210/root (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31210 log ceph txt unknown /proc/31209/task/31210/exe (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31210 log ceph NOFD /proc/31209/task/31210/fd (opendir: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31211 service ceph cwd unknown /proc/31209/task/31211/cwd (readlink: Permission denied) 2024-09-21T02:55:13.970 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31211 service ceph rtd unknown /proc/31209/task/31211/root (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31211 service ceph txt unknown /proc/31209/task/31211/exe (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31211 service ceph NOFD /proc/31209/task/31211/fd (opendir: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31212 admin_soc ceph cwd unknown /proc/31209/task/31212/cwd (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31212 admin_soc ceph rtd unknown /proc/31209/task/31212/root (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31212 admin_soc ceph txt unknown /proc/31209/task/31212/exe (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31212 admin_soc ceph NOFD /proc/31209/task/31212/fd (opendir: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31213 signal_ha ceph cwd unknown /proc/31209/task/31213/cwd (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31213 signal_ha ceph rtd unknown /proc/31209/task/31213/root (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31213 signal_ha ceph txt unknown /proc/31209/task/31213/exe (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31213 signal_ha ceph NOFD /proc/31209/task/31213/fd (opendir: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31214 rocksdb:l ceph cwd unknown /proc/31209/task/31214/cwd (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31214 rocksdb:l ceph rtd unknown /proc/31209/task/31214/root (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31214 rocksdb:l ceph txt unknown /proc/31209/task/31214/exe (readlink: Permission denied) 2024-09-21T02:55:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31214 rocksdb:l ceph NOFD /proc/31209/task/31214/fd (opendir: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31215 rocksdb:h ceph cwd unknown /proc/31209/task/31215/cwd (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31215 rocksdb:h ceph rtd unknown /proc/31209/task/31215/root (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31215 rocksdb:h ceph txt unknown /proc/31209/task/31215/exe (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31215 rocksdb:h ceph NOFD /proc/31209/task/31215/fd (opendir: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31231 ceph-mon ceph cwd unknown /proc/31209/task/31231/cwd (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31231 ceph-mon ceph rtd unknown /proc/31209/task/31231/root (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31231 ceph-mon ceph txt unknown /proc/31209/task/31231/exe (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31231 ceph-mon ceph NOFD /proc/31209/task/31231/fd (opendir: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31232 fn_monsto ceph cwd unknown /proc/31209/task/31232/cwd (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31232 fn_monsto ceph rtd unknown /proc/31209/task/31232/root (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31232 fn_monsto ceph txt unknown /proc/31209/task/31232/exe (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31232 fn_monsto ceph NOFD /proc/31209/task/31232/fd (opendir: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31233 msgr-work ceph cwd unknown /proc/31209/task/31233/cwd (readlink: Permission denied) 2024-09-21T02:55:13.972 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31233 msgr-work ceph rtd unknown /proc/31209/task/31233/root (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31233 msgr-work ceph txt unknown /proc/31209/task/31233/exe (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31233 msgr-work ceph NOFD /proc/31209/task/31233/fd (opendir: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31234 msgr-work ceph cwd unknown /proc/31209/task/31234/cwd (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31234 msgr-work ceph rtd unknown /proc/31209/task/31234/root (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31234 msgr-work ceph txt unknown /proc/31209/task/31234/exe (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31234 msgr-work ceph NOFD /proc/31209/task/31234/fd (opendir: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31235 msgr-work ceph cwd unknown /proc/31209/task/31235/cwd (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31235 msgr-work ceph rtd unknown /proc/31209/task/31235/root (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31235 msgr-work ceph txt unknown /proc/31209/task/31235/exe (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31235 msgr-work ceph NOFD /proc/31209/task/31235/fd (opendir: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31236 OpHistory ceph cwd unknown /proc/31209/task/31236/cwd (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31236 OpHistory ceph rtd unknown /proc/31209/task/31236/root (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31236 OpHistory ceph txt unknown /proc/31209/task/31236/exe (readlink: Permission denied) 2024-09-21T02:55:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31236 OpHistory ceph NOFD /proc/31209/task/31236/fd (opendir: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31237 fin ceph cwd unknown /proc/31209/task/31237/cwd (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31237 fin ceph rtd unknown /proc/31209/task/31237/root (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31237 fin ceph txt unknown /proc/31209/task/31237/exe (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31237 fin ceph NOFD /proc/31209/task/31237/fd (opendir: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31238 safe_time ceph cwd unknown /proc/31209/task/31238/cwd (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31238 safe_time ceph rtd unknown /proc/31209/task/31238/root (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31238 safe_time ceph txt unknown /proc/31209/task/31238/exe (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31238 safe_time ceph NOFD /proc/31209/task/31238/fd (opendir: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31239 cpu_tp ceph cwd unknown /proc/31209/task/31239/cwd (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31239 cpu_tp ceph rtd unknown /proc/31209/task/31239/root (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31239 cpu_tp ceph txt unknown /proc/31209/task/31239/exe (readlink: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31239 cpu_tp ceph NOFD /proc/31209/task/31239/fd (opendir: Permission denied) 2024-09-21T02:55:13.974 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31240 cpu_tp ceph cwd unknown /proc/31209/task/31240/cwd (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31240 cpu_tp ceph rtd unknown /proc/31209/task/31240/root (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31240 cpu_tp ceph txt unknown /proc/31209/task/31240/exe (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31240 cpu_tp ceph NOFD /proc/31209/task/31240/fd (opendir: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31241 cpu_tp ceph cwd unknown /proc/31209/task/31241/cwd (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31241 cpu_tp ceph rtd unknown /proc/31209/task/31241/root (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31241 cpu_tp ceph txt unknown /proc/31209/task/31241/exe (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31241 cpu_tp ceph NOFD /proc/31209/task/31241/fd (opendir: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31242 cpu_tp ceph cwd unknown /proc/31209/task/31242/cwd (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31242 cpu_tp ceph rtd unknown /proc/31209/task/31242/root (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31242 cpu_tp ceph txt unknown /proc/31209/task/31242/exe (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31242 cpu_tp ceph NOFD /proc/31209/task/31242/fd (opendir: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31243 ms_dispat ceph cwd unknown /proc/31209/task/31243/cwd (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31243 ms_dispat ceph rtd unknown /proc/31209/task/31243/root (readlink: Permission denied) 2024-09-21T02:55:13.975 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31243 ms_dispat ceph txt unknown /proc/31209/task/31243/exe (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31243 ms_dispat ceph NOFD /proc/31209/task/31243/fd (opendir: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31244 ms_local ceph cwd unknown /proc/31209/task/31244/cwd (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31244 ms_local ceph rtd unknown /proc/31209/task/31244/root (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31244 ms_local ceph txt unknown /proc/31209/task/31244/exe (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31244 ms_local ceph NOFD /proc/31209/task/31244/fd (opendir: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31245 safe_time ceph cwd unknown /proc/31209/task/31245/cwd (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31245 safe_time ceph rtd unknown /proc/31209/task/31245/root (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31245 safe_time ceph txt unknown /proc/31209/task/31245/exe (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31245 safe_time ceph NOFD /proc/31209/task/31245/fd (opendir: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31246 ms_dispat ceph cwd unknown /proc/31209/task/31246/cwd (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31246 ms_dispat ceph rtd unknown /proc/31209/task/31246/root (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31246 ms_dispat ceph txt unknown /proc/31209/task/31246/exe (readlink: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31246 ms_dispat ceph NOFD /proc/31209/task/31246/fd (opendir: Permission denied) 2024-09-21T02:55:13.976 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31247 ms_local ceph cwd unknown /proc/31209/task/31247/cwd (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31247 ms_local ceph rtd unknown /proc/31209/task/31247/root (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31247 ms_local ceph txt unknown /proc/31209/task/31247/exe (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 31247 ms_local ceph NOFD /proc/31209/task/31247/fd (opendir: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 67458 rstore_co ceph cwd unknown /proc/31209/task/67458/cwd (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 67458 rstore_co ceph rtd unknown /proc/31209/task/67458/root (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 67458 rstore_co ceph txt unknown /proc/31209/task/67458/exe (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mon 31209 67458 rstore_co ceph NOFD /proc/31209/task/67458/fd (opendir: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31434 root cwd unknown /proc/31434/cwd (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31434 root rtd unknown /proc/31434/root (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31434 root txt unknown /proc/31434/exe (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 31434 root NOFD /proc/31434/fd (opendir: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31436 root cwd unknown /proc/31436/cwd (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31436 root rtd unknown /proc/31436/root (readlink: Permission denied) 2024-09-21T02:55:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31436 root txt unknown /proc/31436/exe (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 31436 root NOFD /proc/31436/fd (opendir: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 ceph cwd unknown /proc/31438/cwd (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 ceph rtd unknown /proc/31438/root (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 ceph txt unknown /proc/31438/exe (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 ceph NOFD /proc/31438/fd (opendir: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40940 log ceph cwd unknown /proc/31438/task/40940/cwd (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40940 log ceph rtd unknown /proc/31438/task/40940/root (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40940 log ceph txt unknown /proc/31438/task/40940/exe (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40940 log ceph NOFD /proc/31438/task/40940/fd (opendir: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40942 msgr-work ceph cwd unknown /proc/31438/task/40942/cwd (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40942 msgr-work ceph rtd unknown /proc/31438/task/40942/root (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40942 msgr-work ceph txt unknown /proc/31438/task/40942/exe (readlink: Permission denied) 2024-09-21T02:55:13.978 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40942 msgr-work ceph NOFD /proc/31438/task/40942/fd (opendir: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40943 msgr-work ceph cwd unknown /proc/31438/task/40943/cwd (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40943 msgr-work ceph rtd unknown /proc/31438/task/40943/root (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40943 msgr-work ceph txt unknown /proc/31438/task/40943/exe (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40943 msgr-work ceph NOFD /proc/31438/task/40943/fd (opendir: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40944 msgr-work ceph cwd unknown /proc/31438/task/40944/cwd (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40944 msgr-work ceph rtd unknown /proc/31438/task/40944/root (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40944 msgr-work ceph txt unknown /proc/31438/task/40944/exe (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40944 msgr-work ceph NOFD /proc/31438/task/40944/fd (opendir: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40948 service ceph cwd unknown /proc/31438/task/40948/cwd (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40948 service ceph rtd unknown /proc/31438/task/40948/root (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40948 service ceph txt unknown /proc/31438/task/40948/exe (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40948 service ceph NOFD /proc/31438/task/40948/fd (opendir: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40949 admin_soc ceph cwd unknown /proc/31438/task/40949/cwd (readlink: Permission denied) 2024-09-21T02:55:13.979 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40949 admin_soc ceph rtd unknown /proc/31438/task/40949/root (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40949 admin_soc ceph txt unknown /proc/31438/task/40949/exe (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40949 admin_soc ceph NOFD /proc/31438/task/40949/fd (opendir: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40950 ceph_time ceph cwd unknown /proc/31438/task/40950/cwd (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40950 ceph_time ceph rtd unknown /proc/31438/task/40950/root (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40950 ceph_time ceph txt unknown /proc/31438/task/40950/exe (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40950 ceph_time ceph NOFD /proc/31438/task/40950/fd (opendir: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40951 fn_anonym ceph cwd unknown /proc/31438/task/40951/cwd (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40951 fn_anonym ceph rtd unknown /proc/31438/task/40951/root (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40951 fn_anonym ceph txt unknown /proc/31438/task/40951/exe (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40951 fn_anonym ceph NOFD /proc/31438/task/40951/fd (opendir: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40952 signal_ha ceph cwd unknown /proc/31438/task/40952/cwd (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40952 signal_ha ceph rtd unknown /proc/31438/task/40952/root (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40952 signal_ha ceph txt unknown /proc/31438/task/40952/exe (readlink: Permission denied) 2024-09-21T02:55:13.980 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40952 signal_ha ceph NOFD /proc/31438/task/40952/fd (opendir: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40953 mgrsb-fin ceph cwd unknown /proc/31438/task/40953/cwd (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40953 mgrsb-fin ceph rtd unknown /proc/31438/task/40953/root (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40953 mgrsb-fin ceph txt unknown /proc/31438/task/40953/exe (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40953 mgrsb-fin ceph NOFD /proc/31438/task/40953/fd (opendir: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40954 ms_dispat ceph cwd unknown /proc/31438/task/40954/cwd (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40954 ms_dispat ceph rtd unknown /proc/31438/task/40954/root (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40954 ms_dispat ceph txt unknown /proc/31438/task/40954/exe (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40954 ms_dispat ceph NOFD /proc/31438/task/40954/fd (opendir: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40955 ms_local ceph cwd unknown /proc/31438/task/40955/cwd (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40955 ms_local ceph rtd unknown /proc/31438/task/40955/root (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40955 ms_local ceph txt unknown /proc/31438/task/40955/exe (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40955 ms_local ceph NOFD /proc/31438/task/40955/fd (opendir: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40956 io_contex ceph cwd unknown /proc/31438/task/40956/cwd (readlink: Permission denied) 2024-09-21T02:55:13.981 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40956 io_contex ceph rtd unknown /proc/31438/task/40956/root (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40956 io_contex ceph txt unknown /proc/31438/task/40956/exe (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40956 io_contex ceph NOFD /proc/31438/task/40956/fd (opendir: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40957 io_contex ceph cwd unknown /proc/31438/task/40957/cwd (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40957 io_contex ceph rtd unknown /proc/31438/task/40957/root (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40957 io_contex ceph txt unknown /proc/31438/task/40957/exe (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40957 io_contex ceph NOFD /proc/31438/task/40957/fd (opendir: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40958 safe_time ceph cwd unknown /proc/31438/task/40958/cwd (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40958 safe_time ceph rtd unknown /proc/31438/task/40958/root (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40958 safe_time ceph txt unknown /proc/31438/task/40958/exe (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40958 safe_time ceph NOFD /proc/31438/task/40958/fd (opendir: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40959 safe_time ceph cwd unknown /proc/31438/task/40959/cwd (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40959 safe_time ceph rtd unknown /proc/31438/task/40959/root (readlink: Permission denied) 2024-09-21T02:55:13.982 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40959 safe_time ceph txt unknown /proc/31438/task/40959/exe (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40959 safe_time ceph NOFD /proc/31438/task/40959/fd (opendir: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40960 safe_time ceph cwd unknown /proc/31438/task/40960/cwd (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40960 safe_time ceph rtd unknown /proc/31438/task/40960/root (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40960 safe_time ceph txt unknown /proc/31438/task/40960/exe (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40960 safe_time ceph NOFD /proc/31438/task/40960/fd (opendir: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40961 fn_anonym ceph cwd unknown /proc/31438/task/40961/cwd (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40961 fn_anonym ceph rtd unknown /proc/31438/task/40961/root (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40961 fn_anonym ceph txt unknown /proc/31438/task/40961/exe (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40961 fn_anonym ceph NOFD /proc/31438/task/40961/fd (opendir: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40962 flusher ceph cwd unknown /proc/31438/task/40962/cwd (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40962 flusher ceph rtd unknown /proc/31438/task/40962/root (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40962 flusher ceph txt unknown /proc/31438/task/40962/exe (readlink: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40962 flusher ceph NOFD /proc/31438/task/40962/fd (opendir: Permission denied) 2024-09-21T02:55:13.983 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40963 safe_time ceph cwd unknown /proc/31438/task/40963/cwd (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40963 safe_time ceph rtd unknown /proc/31438/task/40963/root (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40963 safe_time ceph txt unknown /proc/31438/task/40963/exe (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40963 safe_time ceph NOFD /proc/31438/task/40963/fd (opendir: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40979 OpHistory ceph cwd unknown /proc/31438/task/40979/cwd (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40979 OpHistory ceph rtd unknown /proc/31438/task/40979/root (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40979 OpHistory ceph txt unknown /proc/31438/task/40979/exe (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40979 OpHistory ceph NOFD /proc/31438/task/40979/fd (opendir: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40980 mgr-fin ceph cwd unknown /proc/31438/task/40980/cwd (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40980 mgr-fin ceph rtd unknown /proc/31438/task/40980/root (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40980 mgr-fin ceph txt unknown /proc/31438/task/40980/exe (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40980 mgr-fin ceph NOFD /proc/31438/task/40980/fd (opendir: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40981 ms_dispat ceph cwd unknown /proc/31438/task/40981/cwd (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40981 ms_dispat ceph rtd unknown /proc/31438/task/40981/root (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40981 ms_dispat ceph txt unknown /proc/31438/task/40981/exe (readlink: Permission denied) 2024-09-21T02:55:13.984 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40981 ms_dispat ceph NOFD /proc/31438/task/40981/fd (opendir: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40982 ms_local ceph cwd unknown /proc/31438/task/40982/cwd (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40982 ms_local ceph rtd unknown /proc/31438/task/40982/root (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40982 ms_local ceph txt unknown /proc/31438/task/40982/exe (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40982 ms_local ceph NOFD /proc/31438/task/40982/fd (opendir: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40983 safe_time ceph cwd unknown /proc/31438/task/40983/cwd (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40983 safe_time ceph rtd unknown /proc/31438/task/40983/root (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40983 safe_time ceph txt unknown /proc/31438/task/40983/exe (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40983 safe_time ceph NOFD /proc/31438/task/40983/fd (opendir: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40984 cmdfin ceph cwd unknown /proc/31438/task/40984/cwd (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40984 cmdfin ceph rtd unknown /proc/31438/task/40984/root (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40984 cmdfin ceph txt unknown /proc/31438/task/40984/exe (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40984 cmdfin ceph NOFD /proc/31438/task/40984/fd (opendir: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40988 io_contex ceph cwd unknown /proc/31438/task/40988/cwd (readlink: Permission denied) 2024-09-21T02:55:13.985 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40988 io_contex ceph rtd unknown /proc/31438/task/40988/root (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40988 io_contex ceph txt unknown /proc/31438/task/40988/exe (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40988 io_contex ceph NOFD /proc/31438/task/40988/fd (opendir: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40989 io_contex ceph cwd unknown /proc/31438/task/40989/cwd (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40989 io_contex ceph rtd unknown /proc/31438/task/40989/root (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40989 io_contex ceph txt unknown /proc/31438/task/40989/exe (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40989 io_contex ceph NOFD /proc/31438/task/40989/fd (opendir: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40990 ceph_time ceph cwd unknown /proc/31438/task/40990/cwd (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40990 ceph_time ceph rtd unknown /proc/31438/task/40990/root (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40990 ceph_time ceph txt unknown /proc/31438/task/40990/exe (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40990 ceph_time ceph NOFD /proc/31438/task/40990/fd (opendir: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40991 ms_dispat ceph cwd unknown /proc/31438/task/40991/cwd (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40991 ms_dispat ceph rtd unknown /proc/31438/task/40991/root (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40991 ms_dispat ceph txt unknown /proc/31438/task/40991/exe (readlink: Permission denied) 2024-09-21T02:55:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40991 ms_dispat ceph NOFD /proc/31438/task/40991/fd (opendir: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40992 ms_local ceph cwd unknown /proc/31438/task/40992/cwd (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40992 ms_local ceph rtd unknown /proc/31438/task/40992/root (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40992 ms_local ceph txt unknown /proc/31438/task/40992/exe (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40992 ms_local ceph NOFD /proc/31438/task/40992/fd (opendir: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40993 safe_time ceph cwd unknown /proc/31438/task/40993/cwd (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40993 safe_time ceph rtd unknown /proc/31438/task/40993/root (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40993 safe_time ceph txt unknown /proc/31438/task/40993/exe (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40993 safe_time ceph NOFD /proc/31438/task/40993/fd (opendir: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40994 safe_time ceph cwd unknown /proc/31438/task/40994/cwd (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40994 safe_time ceph rtd unknown /proc/31438/task/40994/root (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40994 safe_time ceph txt unknown /proc/31438/task/40994/exe (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40994 safe_time ceph NOFD /proc/31438/task/40994/fd (opendir: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40995 balancer ceph cwd unknown /proc/31438/task/40995/cwd (readlink: Permission denied) 2024-09-21T02:55:13.987 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40995 balancer ceph rtd unknown /proc/31438/task/40995/root (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40995 balancer ceph txt unknown /proc/31438/task/40995/exe (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40995 balancer ceph NOFD /proc/31438/task/40995/fd (opendir: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40996 m-fin-bal ceph cwd unknown /proc/31438/task/40996/cwd (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40996 m-fin-bal ceph rtd unknown /proc/31438/task/40996/root (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40996 m-fin-bal ceph txt unknown /proc/31438/task/40996/exe (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40996 m-fin-bal ceph NOFD /proc/31438/task/40996/fd (opendir: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40997 mgr-fin ceph cwd unknown /proc/31438/task/40997/cwd (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40997 mgr-fin ceph rtd unknown /proc/31438/task/40997/root (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40997 mgr-fin ceph txt unknown /proc/31438/task/40997/exe (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40997 mgr-fin ceph NOFD /proc/31438/task/40997/fd (opendir: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40998 mgr-fin ceph cwd unknown /proc/31438/task/40998/cwd (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40998 mgr-fin ceph rtd unknown /proc/31438/task/40998/root (readlink: Permission denied) 2024-09-21T02:55:13.988 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40998 mgr-fin ceph txt unknown /proc/31438/task/40998/exe (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40998 mgr-fin ceph NOFD /proc/31438/task/40998/fd (opendir: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40999 mgr-fin ceph cwd unknown /proc/31438/task/40999/cwd (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40999 mgr-fin ceph rtd unknown /proc/31438/task/40999/root (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40999 mgr-fin ceph txt unknown /proc/31438/task/40999/exe (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 40999 mgr-fin ceph NOFD /proc/31438/task/40999/fd (opendir: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41000 mgr-fin ceph cwd unknown /proc/31438/task/41000/cwd (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41000 mgr-fin ceph rtd unknown /proc/31438/task/41000/root (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41000 mgr-fin ceph txt unknown /proc/31438/task/41000/exe (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41000 mgr-fin ceph NOFD /proc/31438/task/41000/fd (opendir: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41001 mgr-fin ceph cwd unknown /proc/31438/task/41001/cwd (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41001 mgr-fin ceph rtd unknown /proc/31438/task/41001/root (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41001 mgr-fin ceph txt unknown /proc/31438/task/41001/exe (readlink: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41001 mgr-fin ceph NOFD /proc/31438/task/41001/fd (opendir: Permission denied) 2024-09-21T02:55:13.989 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41002 mgr-fin ceph cwd unknown /proc/31438/task/41002/cwd (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41002 mgr-fin ceph rtd unknown /proc/31438/task/41002/root (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41002 mgr-fin ceph txt unknown /proc/31438/task/41002/exe (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41002 mgr-fin ceph NOFD /proc/31438/task/41002/fd (opendir: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41003 mgr-fin ceph cwd unknown /proc/31438/task/41003/cwd (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41003 mgr-fin ceph rtd unknown /proc/31438/task/41003/root (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41003 mgr-fin ceph txt unknown /proc/31438/task/41003/exe (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41003 mgr-fin ceph NOFD /proc/31438/task/41003/fd (opendir: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41004 mgr-fin ceph cwd unknown /proc/31438/task/41004/cwd (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41004 mgr-fin ceph rtd unknown /proc/31438/task/41004/root (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41004 mgr-fin ceph txt unknown /proc/31438/task/41004/exe (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41004 mgr-fin ceph NOFD /proc/31438/task/41004/fd (opendir: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41005 mgr-fin ceph cwd unknown /proc/31438/task/41005/cwd (readlink: Permission denied) 2024-09-21T02:55:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41005 mgr-fin ceph rtd unknown /proc/31438/task/41005/root (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41005 mgr-fin ceph txt unknown /proc/31438/task/41005/exe (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41005 mgr-fin ceph NOFD /proc/31438/task/41005/fd (opendir: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41006 mgr-fin ceph cwd unknown /proc/31438/task/41006/cwd (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41006 mgr-fin ceph rtd unknown /proc/31438/task/41006/root (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41006 mgr-fin ceph txt unknown /proc/31438/task/41006/exe (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41006 mgr-fin ceph NOFD /proc/31438/task/41006/fd (opendir: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41007 mgr-fin ceph cwd unknown /proc/31438/task/41007/cwd (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41007 mgr-fin ceph rtd unknown /proc/31438/task/41007/root (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41007 mgr-fin ceph txt unknown /proc/31438/task/41007/exe (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41007 mgr-fin ceph NOFD /proc/31438/task/41007/fd (opendir: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41008 mgr-fin ceph cwd unknown /proc/31438/task/41008/cwd (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41008 mgr-fin ceph rtd unknown /proc/31438/task/41008/root (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41008 mgr-fin ceph txt unknown /proc/31438/task/41008/exe (readlink: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41008 mgr-fin ceph NOFD /proc/31438/task/41008/fd (opendir: Permission denied) 2024-09-21T02:55:13.991 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41009 mgr-fin ceph cwd unknown /proc/31438/task/41009/cwd (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41009 mgr-fin ceph rtd unknown /proc/31438/task/41009/root (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41009 mgr-fin ceph txt unknown /proc/31438/task/41009/exe (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41009 mgr-fin ceph NOFD /proc/31438/task/41009/fd (opendir: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41010 mgr-fin ceph cwd unknown /proc/31438/task/41010/cwd (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41010 mgr-fin ceph rtd unknown /proc/31438/task/41010/root (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41010 mgr-fin ceph txt unknown /proc/31438/task/41010/exe (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41010 mgr-fin ceph NOFD /proc/31438/task/41010/fd (opendir: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41011 mgr-fin ceph cwd unknown /proc/31438/task/41011/cwd (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41011 mgr-fin ceph rtd unknown /proc/31438/task/41011/root (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41011 mgr-fin ceph txt unknown /proc/31438/task/41011/exe (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41011 mgr-fin ceph NOFD /proc/31438/task/41011/fd (opendir: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41012 mgr-fin ceph cwd unknown /proc/31438/task/41012/cwd (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41012 mgr-fin ceph rtd unknown /proc/31438/task/41012/root (readlink: Permission denied) 2024-09-21T02:55:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41012 mgr-fin ceph txt unknown /proc/31438/task/41012/exe (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41012 mgr-fin ceph NOFD /proc/31438/task/41012/fd (opendir: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41013 cephadm ceph cwd unknown /proc/31438/task/41013/cwd (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41013 cephadm ceph rtd unknown /proc/31438/task/41013/root (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41013 cephadm ceph txt unknown /proc/31438/task/41013/exe (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41013 cephadm ceph NOFD /proc/31438/task/41013/fd (opendir: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41014 m-fin-cep ceph cwd unknown /proc/31438/task/41014/cwd (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41014 m-fin-cep ceph rtd unknown /proc/31438/task/41014/root (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41014 m-fin-cep ceph txt unknown /proc/31438/task/41014/exe (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41014 m-fin-cep ceph NOFD /proc/31438/task/41014/fd (opendir: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41015 mgr-fin ceph cwd unknown /proc/31438/task/41015/cwd (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41015 mgr-fin ceph rtd unknown /proc/31438/task/41015/root (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41015 mgr-fin ceph txt unknown /proc/31438/task/41015/exe (readlink: Permission denied) 2024-09-21T02:55:13.993 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41015 mgr-fin ceph NOFD /proc/31438/task/41015/fd (opendir: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41016 mgr-fin ceph cwd unknown /proc/31438/task/41016/cwd (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41016 mgr-fin ceph rtd unknown /proc/31438/task/41016/root (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41016 mgr-fin ceph txt unknown /proc/31438/task/41016/exe (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41016 mgr-fin ceph NOFD /proc/31438/task/41016/fd (opendir: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41017 cephadm ceph cwd unknown /proc/31438/task/41017/cwd (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41017 cephadm ceph rtd unknown /proc/31438/task/41017/root (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41017 cephadm ceph txt unknown /proc/31438/task/41017/exe (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41017 cephadm ceph NOFD /proc/31438/task/41017/fd (opendir: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41018 mgr-fin ceph cwd unknown /proc/31438/task/41018/cwd (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41018 mgr-fin ceph rtd unknown /proc/31438/task/41018/root (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41018 mgr-fin ceph txt unknown /proc/31438/task/41018/exe (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41018 mgr-fin ceph NOFD /proc/31438/task/41018/fd (opendir: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41019 crash ceph cwd unknown /proc/31438/task/41019/cwd (readlink: Permission denied) 2024-09-21T02:55:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41019 crash ceph rtd unknown /proc/31438/task/41019/root (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41019 crash ceph txt unknown /proc/31438/task/41019/exe (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41019 crash ceph NOFD /proc/31438/task/41019/fd (opendir: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41020 m-fin-cra ceph cwd unknown /proc/31438/task/41020/cwd (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41020 m-fin-cra ceph rtd unknown /proc/31438/task/41020/root (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41020 m-fin-cra ceph txt unknown /proc/31438/task/41020/exe (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41020 m-fin-cra ceph NOFD /proc/31438/task/41020/fd (opendir: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41021 mgr-fin ceph cwd unknown /proc/31438/task/41021/cwd (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41021 mgr-fin ceph rtd unknown /proc/31438/task/41021/root (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41021 mgr-fin ceph txt unknown /proc/31438/task/41021/exe (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41021 mgr-fin ceph NOFD /proc/31438/task/41021/fd (opendir: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41022 mgr-fin ceph cwd unknown /proc/31438/task/41022/cwd (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41022 mgr-fin ceph rtd unknown /proc/31438/task/41022/root (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41022 mgr-fin ceph txt unknown /proc/31438/task/41022/exe (readlink: Permission denied) 2024-09-21T02:55:13.995 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41022 mgr-fin ceph NOFD /proc/31438/task/41022/fd (opendir: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41023 dashboard ceph cwd unknown /proc/31438/task/41023/cwd (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41023 dashboard ceph rtd unknown /proc/31438/task/41023/root (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41023 dashboard ceph txt unknown /proc/31438/task/41023/exe (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41023 dashboard ceph NOFD /proc/31438/task/41023/fd (opendir: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41024 mgr-fin ceph cwd unknown /proc/31438/task/41024/cwd (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41024 mgr-fin ceph rtd unknown /proc/31438/task/41024/root (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41024 mgr-fin ceph txt unknown /proc/31438/task/41024/exe (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41024 mgr-fin ceph NOFD /proc/31438/task/41024/fd (opendir: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41025 m-fin-das ceph cwd unknown /proc/31438/task/41025/cwd (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41025 m-fin-das ceph rtd unknown /proc/31438/task/41025/root (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41025 m-fin-das ceph txt unknown /proc/31438/task/41025/exe (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41025 m-fin-das ceph NOFD /proc/31438/task/41025/fd (opendir: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41026 mgr-fin ceph cwd unknown /proc/31438/task/41026/cwd (readlink: Permission denied) 2024-09-21T02:55:13.996 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41026 mgr-fin ceph rtd unknown /proc/31438/task/41026/root (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41026 mgr-fin ceph txt unknown /proc/31438/task/41026/exe (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41026 mgr-fin ceph NOFD /proc/31438/task/41026/fd (opendir: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41027 mgr-fin ceph cwd unknown /proc/31438/task/41027/cwd (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41027 mgr-fin ceph rtd unknown /proc/31438/task/41027/root (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41027 mgr-fin ceph txt unknown /proc/31438/task/41027/exe (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41027 mgr-fin ceph NOFD /proc/31438/task/41027/fd (opendir: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41028 devicehea ceph cwd unknown /proc/31438/task/41028/cwd (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41028 devicehea ceph rtd unknown /proc/31438/task/41028/root (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41028 devicehea ceph txt unknown /proc/31438/task/41028/exe (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41028 devicehea ceph NOFD /proc/31438/task/41028/fd (opendir: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41029 mgr-fin ceph cwd unknown /proc/31438/task/41029/cwd (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41029 mgr-fin ceph rtd unknown /proc/31438/task/41029/root (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41029 mgr-fin ceph txt unknown /proc/31438/task/41029/exe (readlink: Permission denied) 2024-09-21T02:55:13.997 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41029 mgr-fin ceph NOFD /proc/31438/task/41029/fd (opendir: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41030 m-fin-dev ceph cwd unknown /proc/31438/task/41030/cwd (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41030 m-fin-dev ceph rtd unknown /proc/31438/task/41030/root (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41030 m-fin-dev ceph txt unknown /proc/31438/task/41030/exe (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41030 m-fin-dev ceph NOFD /proc/31438/task/41030/fd (opendir: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41031 mgr-fin ceph cwd unknown /proc/31438/task/41031/cwd (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41031 mgr-fin ceph rtd unknown /proc/31438/task/41031/root (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41031 mgr-fin ceph txt unknown /proc/31438/task/41031/exe (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41031 mgr-fin ceph NOFD /proc/31438/task/41031/fd (opendir: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41032 mgr-fin ceph cwd unknown /proc/31438/task/41032/cwd (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41032 mgr-fin ceph rtd unknown /proc/31438/task/41032/root (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41032 mgr-fin ceph txt unknown /proc/31438/task/41032/exe (readlink: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41032 mgr-fin ceph NOFD /proc/31438/task/41032/fd (opendir: Permission denied) 2024-09-21T02:55:13.998 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41033 mgr-fin ceph cwd unknown /proc/31438/task/41033/cwd (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41033 mgr-fin ceph rtd unknown /proc/31438/task/41033/root (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41033 mgr-fin ceph txt unknown /proc/31438/task/41033/exe (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41033 mgr-fin ceph NOFD /proc/31438/task/41033/fd (opendir: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41035 m-fin-ios ceph cwd unknown /proc/31438/task/41035/cwd (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41035 m-fin-ios ceph rtd unknown /proc/31438/task/41035/root (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41035 m-fin-ios ceph txt unknown /proc/31438/task/41035/exe (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41035 m-fin-ios ceph NOFD /proc/31438/task/41035/fd (opendir: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41037 m-fin-nfs ceph cwd unknown /proc/31438/task/41037/cwd (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41037 m-fin-nfs ceph rtd unknown /proc/31438/task/41037/root (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41037 m-fin-nfs ceph txt unknown /proc/31438/task/41037/exe (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41037 m-fin-nfs ceph NOFD /proc/31438/task/41037/fd (opendir: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41038 cephadm ceph cwd unknown /proc/31438/task/41038/cwd (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41038 cephadm ceph rtd unknown /proc/31438/task/41038/root (readlink: Permission denied) 2024-09-21T02:55:13.999 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41038 cephadm ceph txt unknown /proc/31438/task/41038/exe (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41038 cephadm ceph NOFD /proc/31438/task/41038/fd (opendir: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41040 m-fin-orc ceph cwd unknown /proc/31438/task/41040/cwd (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41040 m-fin-orc ceph rtd unknown /proc/31438/task/41040/root (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41040 m-fin-orc ceph txt unknown /proc/31438/task/41040/exe (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41040 m-fin-orc ceph NOFD /proc/31438/task/41040/fd (opendir: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41041 pg_autosc ceph cwd unknown /proc/31438/task/41041/cwd (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41041 pg_autosc ceph rtd unknown /proc/31438/task/41041/root (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41041 pg_autosc ceph txt unknown /proc/31438/task/41041/exe (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41041 pg_autosc ceph NOFD /proc/31438/task/41041/fd (opendir: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41042 m-fin-pg_ ceph cwd unknown /proc/31438/task/41042/cwd (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41042 m-fin-pg_ ceph rtd unknown /proc/31438/task/41042/root (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41042 m-fin-pg_ ceph txt unknown /proc/31438/task/41042/exe (readlink: Permission denied) 2024-09-21T02:55:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41042 m-fin-pg_ ceph NOFD /proc/31438/task/41042/fd (opendir: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41043 cephadm ceph cwd unknown /proc/31438/task/41043/cwd (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41043 cephadm ceph rtd unknown /proc/31438/task/41043/root (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41043 cephadm ceph txt unknown /proc/31438/task/41043/exe (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41043 cephadm ceph NOFD /proc/31438/task/41043/fd (opendir: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41044 cephadm ceph cwd unknown /proc/31438/task/41044/cwd (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41044 cephadm ceph rtd unknown /proc/31438/task/41044/root (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41044 cephadm ceph txt unknown /proc/31438/task/41044/exe (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41044 cephadm ceph NOFD /proc/31438/task/41044/fd (opendir: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41045 progress ceph cwd unknown /proc/31438/task/41045/cwd (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41045 progress ceph rtd unknown /proc/31438/task/41045/root (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41045 progress ceph txt unknown /proc/31438/task/41045/exe (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41045 progress ceph NOFD /proc/31438/task/41045/fd (opendir: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41046 m-fin-pro ceph cwd unknown /proc/31438/task/41046/cwd (readlink: Permission denied) 2024-09-21T02:55:14.001 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41046 m-fin-pro ceph rtd unknown /proc/31438/task/41046/root (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41046 m-fin-pro ceph txt unknown /proc/31438/task/41046/exe (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41046 m-fin-pro ceph NOFD /proc/31438/task/41046/fd (opendir: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41050 io_contex ceph cwd unknown /proc/31438/task/41050/cwd (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41050 io_contex ceph rtd unknown /proc/31438/task/41050/root (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41050 io_contex ceph txt unknown /proc/31438/task/41050/exe (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41050 io_contex ceph NOFD /proc/31438/task/41050/fd (opendir: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41051 io_contex ceph cwd unknown /proc/31438/task/41051/cwd (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41051 io_contex ceph rtd unknown /proc/31438/task/41051/root (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41051 io_contex ceph txt unknown /proc/31438/task/41051/exe (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41051 io_contex ceph NOFD /proc/31438/task/41051/fd (opendir: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41052 ceph_time ceph cwd unknown /proc/31438/task/41052/cwd (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41052 ceph_time ceph rtd unknown /proc/31438/task/41052/root (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41052 ceph_time ceph txt unknown /proc/31438/task/41052/exe (readlink: Permission denied) 2024-09-21T02:55:14.002 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41052 ceph_time ceph NOFD /proc/31438/task/41052/fd (opendir: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41053 ms_dispat ceph cwd unknown /proc/31438/task/41053/cwd (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41053 ms_dispat ceph rtd unknown /proc/31438/task/41053/root (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41053 ms_dispat ceph txt unknown /proc/31438/task/41053/exe (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41053 ms_dispat ceph NOFD /proc/31438/task/41053/fd (opendir: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41054 ms_local ceph cwd unknown /proc/31438/task/41054/cwd (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41054 ms_local ceph rtd unknown /proc/31438/task/41054/root (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41054 ms_local ceph txt unknown /proc/31438/task/41054/exe (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41054 ms_local ceph NOFD /proc/31438/task/41054/fd (opendir: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41055 safe_time ceph cwd unknown /proc/31438/task/41055/cwd (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41055 safe_time ceph rtd unknown /proc/31438/task/41055/root (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41055 safe_time ceph txt unknown /proc/31438/task/41055/exe (readlink: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41055 safe_time ceph NOFD /proc/31438/task/41055/fd (opendir: Permission denied) 2024-09-21T02:55:14.003 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41056 safe_time ceph cwd unknown /proc/31438/task/41056/cwd (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41056 safe_time ceph rtd unknown /proc/31438/task/41056/root (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41056 safe_time ceph txt unknown /proc/31438/task/41056/exe (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41056 safe_time ceph NOFD /proc/31438/task/41056/fd (opendir: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41057 mgr-fin ceph cwd unknown /proc/31438/task/41057/cwd (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41057 mgr-fin ceph rtd unknown /proc/31438/task/41057/root (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41057 mgr-fin ceph txt unknown /proc/31438/task/41057/exe (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41057 mgr-fin ceph NOFD /proc/31438/task/41057/fd (opendir: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41059 m-fin-rbd ceph cwd unknown /proc/31438/task/41059/cwd (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41059 m-fin-rbd ceph rtd unknown /proc/31438/task/41059/root (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41059 m-fin-rbd ceph txt unknown /proc/31438/task/41059/exe (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41059 m-fin-rbd ceph NOFD /proc/31438/task/41059/fd (opendir: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41060 restful ceph cwd unknown /proc/31438/task/41060/cwd (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41060 restful ceph rtd unknown /proc/31438/task/41060/root (readlink: Permission denied) 2024-09-21T02:55:14.004 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41060 restful ceph txt unknown /proc/31438/task/41060/exe (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41060 restful ceph NOFD /proc/31438/task/41060/fd (opendir: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41061 m-fin-res ceph cwd unknown /proc/31438/task/41061/cwd (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41061 m-fin-res ceph rtd unknown /proc/31438/task/41061/root (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41061 m-fin-res ceph txt unknown /proc/31438/task/41061/exe (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41061 m-fin-res ceph NOFD /proc/31438/task/41061/fd (opendir: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41062 mgr-fin ceph cwd unknown /proc/31438/task/41062/cwd (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41062 mgr-fin ceph rtd unknown /proc/31438/task/41062/root (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41062 mgr-fin ceph txt unknown /proc/31438/task/41062/exe (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41062 mgr-fin ceph NOFD /proc/31438/task/41062/fd (opendir: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41063 mgr-fin ceph cwd unknown /proc/31438/task/41063/cwd (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41063 mgr-fin ceph rtd unknown /proc/31438/task/41063/root (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41063 mgr-fin ceph txt unknown /proc/31438/task/41063/exe (readlink: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41063 mgr-fin ceph NOFD /proc/31438/task/41063/fd (opendir: Permission denied) 2024-09-21T02:55:14.005 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41065 m-fin-sna ceph cwd unknown /proc/31438/task/41065/cwd (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41065 m-fin-sna ceph rtd unknown /proc/31438/task/41065/root (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41065 m-fin-sna ceph txt unknown /proc/31438/task/41065/exe (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41065 m-fin-sna ceph NOFD /proc/31438/task/41065/fd (opendir: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41066 mgr-fin ceph cwd unknown /proc/31438/task/41066/cwd (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41066 mgr-fin ceph rtd unknown /proc/31438/task/41066/root (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41066 mgr-fin ceph txt unknown /proc/31438/task/41066/exe (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41066 mgr-fin ceph NOFD /proc/31438/task/41066/fd (opendir: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41067 mgr-fin ceph cwd unknown /proc/31438/task/41067/cwd (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41067 mgr-fin ceph rtd unknown /proc/31438/task/41067/root (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41067 mgr-fin ceph txt unknown /proc/31438/task/41067/exe (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41067 mgr-fin ceph NOFD /proc/31438/task/41067/fd (opendir: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41069 m-fin-sta ceph cwd unknown /proc/31438/task/41069/cwd (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41069 m-fin-sta ceph rtd unknown /proc/31438/task/41069/root (readlink: Permission denied) 2024-09-21T02:55:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41069 m-fin-sta ceph txt unknown /proc/31438/task/41069/exe (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41069 m-fin-sta ceph NOFD /proc/31438/task/41069/fd (opendir: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41070 mgr-fin ceph cwd unknown /proc/31438/task/41070/cwd (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41070 mgr-fin ceph rtd unknown /proc/31438/task/41070/root (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41070 mgr-fin ceph txt unknown /proc/31438/task/41070/exe (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41070 mgr-fin ceph NOFD /proc/31438/task/41070/fd (opendir: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41071 telemetry ceph cwd unknown /proc/31438/task/41071/cwd (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41071 telemetry ceph rtd unknown /proc/31438/task/41071/root (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41071 telemetry ceph txt unknown /proc/31438/task/41071/exe (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41071 telemetry ceph NOFD /proc/31438/task/41071/fd (opendir: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41072 m-fin-tel ceph cwd unknown /proc/31438/task/41072/cwd (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41072 m-fin-tel ceph rtd unknown /proc/31438/task/41072/root (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41072 m-fin-tel ceph txt unknown /proc/31438/task/41072/exe (readlink: Permission denied) 2024-09-21T02:55:14.007 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41072 m-fin-tel ceph NOFD /proc/31438/task/41072/fd (opendir: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41073 mgr-fin ceph cwd unknown /proc/31438/task/41073/cwd (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41073 mgr-fin ceph rtd unknown /proc/31438/task/41073/root (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41073 mgr-fin ceph txt unknown /proc/31438/task/41073/exe (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41073 mgr-fin ceph NOFD /proc/31438/task/41073/fd (opendir: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41077 io_contex ceph cwd unknown /proc/31438/task/41077/cwd (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41077 io_contex ceph rtd unknown /proc/31438/task/41077/root (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41077 io_contex ceph txt unknown /proc/31438/task/41077/exe (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41077 io_contex ceph NOFD /proc/31438/task/41077/fd (opendir: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41078 io_contex ceph cwd unknown /proc/31438/task/41078/cwd (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41078 io_contex ceph rtd unknown /proc/31438/task/41078/root (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41078 io_contex ceph txt unknown /proc/31438/task/41078/exe (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41078 io_contex ceph NOFD /proc/31438/task/41078/fd (opendir: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41079 ceph_time ceph cwd unknown /proc/31438/task/41079/cwd (readlink: Permission denied) 2024-09-21T02:55:14.008 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41079 ceph_time ceph rtd unknown /proc/31438/task/41079/root (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41079 ceph_time ceph txt unknown /proc/31438/task/41079/exe (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41079 ceph_time ceph NOFD /proc/31438/task/41079/fd (opendir: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41080 ms_dispat ceph cwd unknown /proc/31438/task/41080/cwd (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41080 ms_dispat ceph rtd unknown /proc/31438/task/41080/root (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41080 ms_dispat ceph txt unknown /proc/31438/task/41080/exe (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41080 ms_dispat ceph NOFD /proc/31438/task/41080/fd (opendir: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41081 ms_local ceph cwd unknown /proc/31438/task/41081/cwd (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41081 ms_local ceph rtd unknown /proc/31438/task/41081/root (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41081 ms_local ceph txt unknown /proc/31438/task/41081/exe (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41081 ms_local ceph NOFD /proc/31438/task/41081/fd (opendir: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41082 safe_time ceph cwd unknown /proc/31438/task/41082/cwd (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41082 safe_time ceph rtd unknown /proc/31438/task/41082/root (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41082 safe_time ceph txt unknown /proc/31438/task/41082/exe (readlink: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41082 safe_time ceph NOFD /proc/31438/task/41082/fd (opendir: Permission denied) 2024-09-21T02:55:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41083 safe_time ceph cwd unknown /proc/31438/task/41083/cwd (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41083 safe_time ceph rtd unknown /proc/31438/task/41083/root (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41083 safe_time ceph txt unknown /proc/31438/task/41083/exe (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41083 safe_time ceph NOFD /proc/31438/task/41083/fd (opendir: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41084 mgr-fin ceph cwd unknown /proc/31438/task/41084/cwd (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41084 mgr-fin ceph rtd unknown /proc/31438/task/41084/root (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41084 mgr-fin ceph txt unknown /proc/31438/task/41084/exe (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41084 mgr-fin ceph NOFD /proc/31438/task/41084/fd (opendir: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41085 mgr-fin ceph cwd unknown /proc/31438/task/41085/cwd (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41085 mgr-fin ceph rtd unknown /proc/31438/task/41085/root (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41085 mgr-fin ceph txt unknown /proc/31438/task/41085/exe (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41085 mgr-fin ceph NOFD /proc/31438/task/41085/fd (opendir: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41086 mgr-fin ceph cwd unknown /proc/31438/task/41086/cwd (readlink: Permission denied) 2024-09-21T02:55:14.010 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41086 mgr-fin ceph rtd unknown /proc/31438/task/41086/root (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41086 mgr-fin ceph txt unknown /proc/31438/task/41086/exe (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41086 mgr-fin ceph NOFD /proc/31438/task/41086/fd (opendir: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41087 mgr-fin ceph cwd unknown /proc/31438/task/41087/cwd (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41087 mgr-fin ceph rtd unknown /proc/31438/task/41087/root (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41087 mgr-fin ceph txt unknown /proc/31438/task/41087/exe (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41087 mgr-fin ceph NOFD /proc/31438/task/41087/fd (opendir: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41088 mgr-fin ceph cwd unknown /proc/31438/task/41088/cwd (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41088 mgr-fin ceph rtd unknown /proc/31438/task/41088/root (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41088 mgr-fin ceph txt unknown /proc/31438/task/41088/exe (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41088 mgr-fin ceph NOFD /proc/31438/task/41088/fd (opendir: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41089 mgr-fin ceph cwd unknown /proc/31438/task/41089/cwd (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41089 mgr-fin ceph rtd unknown /proc/31438/task/41089/root (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41089 mgr-fin ceph txt unknown /proc/31438/task/41089/exe (readlink: Permission denied) 2024-09-21T02:55:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41089 mgr-fin ceph NOFD /proc/31438/task/41089/fd (opendir: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41090 mgr-fin ceph cwd unknown /proc/31438/task/41090/cwd (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41090 mgr-fin ceph rtd unknown /proc/31438/task/41090/root (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41090 mgr-fin ceph txt unknown /proc/31438/task/41090/exe (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41090 mgr-fin ceph NOFD /proc/31438/task/41090/fd (opendir: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41091 mgr-fin ceph cwd unknown /proc/31438/task/41091/cwd (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41091 mgr-fin ceph rtd unknown /proc/31438/task/41091/root (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41091 mgr-fin ceph txt unknown /proc/31438/task/41091/exe (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41091 mgr-fin ceph NOFD /proc/31438/task/41091/fd (opendir: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41092 mgr-fin ceph cwd unknown /proc/31438/task/41092/cwd (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41092 mgr-fin ceph rtd unknown /proc/31438/task/41092/root (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41092 mgr-fin ceph txt unknown /proc/31438/task/41092/exe (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41092 mgr-fin ceph NOFD /proc/31438/task/41092/fd (opendir: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41093 mgr-fin ceph cwd unknown /proc/31438/task/41093/cwd (readlink: Permission denied) 2024-09-21T02:55:14.012 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41093 mgr-fin ceph rtd unknown /proc/31438/task/41093/root (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41093 mgr-fin ceph txt unknown /proc/31438/task/41093/exe (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41093 mgr-fin ceph NOFD /proc/31438/task/41093/fd (opendir: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41094 mgr-fin ceph cwd unknown /proc/31438/task/41094/cwd (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41094 mgr-fin ceph rtd unknown /proc/31438/task/41094/root (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41094 mgr-fin ceph txt unknown /proc/31438/task/41094/exe (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41094 mgr-fin ceph NOFD /proc/31438/task/41094/fd (opendir: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41095 mgr-fin ceph cwd unknown /proc/31438/task/41095/cwd (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41095 mgr-fin ceph rtd unknown /proc/31438/task/41095/root (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41095 mgr-fin ceph txt unknown /proc/31438/task/41095/exe (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41095 mgr-fin ceph NOFD /proc/31438/task/41095/fd (opendir: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41097 m-fin-vol ceph cwd unknown /proc/31438/task/41097/cwd (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41097 m-fin-vol ceph rtd unknown /proc/31438/task/41097/root (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41097 m-fin-vol ceph txt unknown /proc/31438/task/41097/exe (readlink: Permission denied) 2024-09-21T02:55:14.013 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41097 m-fin-vol ceph NOFD /proc/31438/task/41097/fd (opendir: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41217 dashboard ceph cwd unknown /proc/31438/task/41217/cwd (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41217 dashboard ceph rtd unknown /proc/31438/task/41217/root (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41217 dashboard ceph txt unknown /proc/31438/task/41217/exe (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41217 dashboard ceph NOFD /proc/31438/task/41217/fd (opendir: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41218 dashboard ceph cwd unknown /proc/31438/task/41218/cwd (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41218 dashboard ceph rtd unknown /proc/31438/task/41218/root (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41218 dashboard ceph txt unknown /proc/31438/task/41218/exe (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41218 dashboard ceph NOFD /proc/31438/task/41218/fd (opendir: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41219 dashboard ceph cwd unknown /proc/31438/task/41219/cwd (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41219 dashboard ceph rtd unknown /proc/31438/task/41219/root (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41219 dashboard ceph txt unknown /proc/31438/task/41219/exe (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41219 dashboard ceph NOFD /proc/31438/task/41219/fd (opendir: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41220 dashboard ceph cwd unknown /proc/31438/task/41220/cwd (readlink: Permission denied) 2024-09-21T02:55:14.014 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41220 dashboard ceph rtd unknown /proc/31438/task/41220/root (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41220 dashboard ceph txt unknown /proc/31438/task/41220/exe (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41220 dashboard ceph NOFD /proc/31438/task/41220/fd (opendir: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41221 dashboard ceph cwd unknown /proc/31438/task/41221/cwd (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41221 dashboard ceph rtd unknown /proc/31438/task/41221/root (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41221 dashboard ceph txt unknown /proc/31438/task/41221/exe (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41221 dashboard ceph NOFD /proc/31438/task/41221/fd (opendir: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41222 dashboard ceph cwd unknown /proc/31438/task/41222/cwd (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41222 dashboard ceph rtd unknown /proc/31438/task/41222/root (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41222 dashboard ceph txt unknown /proc/31438/task/41222/exe (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41222 dashboard ceph NOFD /proc/31438/task/41222/fd (opendir: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41223 dashboard ceph cwd unknown /proc/31438/task/41223/cwd (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41223 dashboard ceph rtd unknown /proc/31438/task/41223/root (readlink: Permission denied) 2024-09-21T02:55:14.015 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41223 dashboard ceph txt unknown /proc/31438/task/41223/exe (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41223 dashboard ceph NOFD /proc/31438/task/41223/fd (opendir: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41224 dashboard ceph cwd unknown /proc/31438/task/41224/cwd (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41224 dashboard ceph rtd unknown /proc/31438/task/41224/root (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41224 dashboard ceph txt unknown /proc/31438/task/41224/exe (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41224 dashboard ceph NOFD /proc/31438/task/41224/fd (opendir: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41225 dashboard ceph cwd unknown /proc/31438/task/41225/cwd (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41225 dashboard ceph rtd unknown /proc/31438/task/41225/root (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41225 dashboard ceph txt unknown /proc/31438/task/41225/exe (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41225 dashboard ceph NOFD /proc/31438/task/41225/fd (opendir: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41226 dashboard ceph cwd unknown /proc/31438/task/41226/cwd (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41226 dashboard ceph rtd unknown /proc/31438/task/41226/root (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41226 dashboard ceph txt unknown /proc/31438/task/41226/exe (readlink: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41226 dashboard ceph NOFD /proc/31438/task/41226/fd (opendir: Permission denied) 2024-09-21T02:55:14.016 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41227 dashboard ceph cwd unknown /proc/31438/task/41227/cwd (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41227 dashboard ceph rtd unknown /proc/31438/task/41227/root (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41227 dashboard ceph txt unknown /proc/31438/task/41227/exe (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41227 dashboard ceph NOFD /proc/31438/task/41227/fd (opendir: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41228 dashboard ceph cwd unknown /proc/31438/task/41228/cwd (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41228 dashboard ceph rtd unknown /proc/31438/task/41228/root (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41228 dashboard ceph txt unknown /proc/31438/task/41228/exe (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41228 dashboard ceph NOFD /proc/31438/task/41228/fd (opendir: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41633 mgr-fin ceph cwd unknown /proc/31438/task/41633/cwd (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41633 mgr-fin ceph rtd unknown /proc/31438/task/41633/root (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41633 mgr-fin ceph txt unknown /proc/31438/task/41633/exe (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41633 mgr-fin ceph NOFD /proc/31438/task/41633/fd (opendir: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41634 mgr-fin ceph cwd unknown /proc/31438/task/41634/cwd (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41634 mgr-fin ceph rtd unknown /proc/31438/task/41634/root (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41634 mgr-fin ceph txt unknown /proc/31438/task/41634/exe (readlink: Permission denied) 2024-09-21T02:55:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41634 mgr-fin ceph NOFD /proc/31438/task/41634/fd (opendir: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41635 mgr-fin ceph cwd unknown /proc/31438/task/41635/cwd (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41635 mgr-fin ceph rtd unknown /proc/31438/task/41635/root (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41635 mgr-fin ceph txt unknown /proc/31438/task/41635/exe (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41635 mgr-fin ceph NOFD /proc/31438/task/41635/fd (opendir: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41636 mgr-fin ceph cwd unknown /proc/31438/task/41636/cwd (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41636 mgr-fin ceph rtd unknown /proc/31438/task/41636/root (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41636 mgr-fin ceph txt unknown /proc/31438/task/41636/exe (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41636 mgr-fin ceph NOFD /proc/31438/task/41636/fd (opendir: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41637 mgr-fin ceph cwd unknown /proc/31438/task/41637/cwd (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41637 mgr-fin ceph rtd unknown /proc/31438/task/41637/root (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41637 mgr-fin ceph txt unknown /proc/31438/task/41637/exe (readlink: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41637 mgr-fin ceph NOFD /proc/31438/task/41637/fd (opendir: Permission denied) 2024-09-21T02:55:14.018 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41639 mgr-fin ceph cwd unknown /proc/31438/task/41639/cwd (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41639 mgr-fin ceph rtd unknown /proc/31438/task/41639/root (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41639 mgr-fin ceph txt unknown /proc/31438/task/41639/exe (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41639 mgr-fin ceph NOFD /proc/31438/task/41639/fd (opendir: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41644 mgr-fin ceph cwd unknown /proc/31438/task/41644/cwd (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41644 mgr-fin ceph rtd unknown /proc/31438/task/41644/root (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41644 mgr-fin ceph txt unknown /proc/31438/task/41644/exe (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41644 mgr-fin ceph NOFD /proc/31438/task/41644/fd (opendir: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41645 mgr-fin ceph cwd unknown /proc/31438/task/41645/cwd (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41645 mgr-fin ceph rtd unknown /proc/31438/task/41645/root (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41645 mgr-fin ceph txt unknown /proc/31438/task/41645/exe (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41645 mgr-fin ceph NOFD /proc/31438/task/41645/fd (opendir: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41648 mgr-fin ceph cwd unknown /proc/31438/task/41648/cwd (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41648 mgr-fin ceph rtd unknown /proc/31438/task/41648/root (readlink: Permission denied) 2024-09-21T02:55:14.019 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41648 mgr-fin ceph txt unknown /proc/31438/task/41648/exe (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41648 mgr-fin ceph NOFD /proc/31438/task/41648/fd (opendir: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41650 mgr-fin ceph cwd unknown /proc/31438/task/41650/cwd (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41650 mgr-fin ceph rtd unknown /proc/31438/task/41650/root (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41650 mgr-fin ceph txt unknown /proc/31438/task/41650/exe (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41650 mgr-fin ceph NOFD /proc/31438/task/41650/fd (opendir: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41652 mgr-fin ceph cwd unknown /proc/31438/task/41652/cwd (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41652 mgr-fin ceph rtd unknown /proc/31438/task/41652/root (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41652 mgr-fin ceph txt unknown /proc/31438/task/41652/exe (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41652 mgr-fin ceph NOFD /proc/31438/task/41652/fd (opendir: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41704 mgr-fin ceph cwd unknown /proc/31438/task/41704/cwd (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41704 mgr-fin ceph rtd unknown /proc/31438/task/41704/root (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41704 mgr-fin ceph txt unknown /proc/31438/task/41704/exe (readlink: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41704 mgr-fin ceph NOFD /proc/31438/task/41704/fd (opendir: Permission denied) 2024-09-21T02:55:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41705 mgr-fin ceph cwd unknown /proc/31438/task/41705/cwd (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41705 mgr-fin ceph rtd unknown /proc/31438/task/41705/root (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41705 mgr-fin ceph txt unknown /proc/31438/task/41705/exe (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41705 mgr-fin ceph NOFD /proc/31438/task/41705/fd (opendir: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41706 mgr-fin ceph cwd unknown /proc/31438/task/41706/cwd (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41706 mgr-fin ceph rtd unknown /proc/31438/task/41706/root (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41706 mgr-fin ceph txt unknown /proc/31438/task/41706/exe (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41706 mgr-fin ceph NOFD /proc/31438/task/41706/fd (opendir: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41711 mgr-fin ceph cwd unknown /proc/31438/task/41711/cwd (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41711 mgr-fin ceph rtd unknown /proc/31438/task/41711/root (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41711 mgr-fin ceph txt unknown /proc/31438/task/41711/exe (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41711 mgr-fin ceph NOFD /proc/31438/task/41711/fd (opendir: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41712 mgr-fin ceph cwd unknown /proc/31438/task/41712/cwd (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41712 mgr-fin ceph rtd unknown /proc/31438/task/41712/root (readlink: Permission denied) 2024-09-21T02:55:14.021 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41712 mgr-fin ceph txt unknown /proc/31438/task/41712/exe (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41712 mgr-fin ceph NOFD /proc/31438/task/41712/fd (opendir: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41713 mgr-fin ceph cwd unknown /proc/31438/task/41713/cwd (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41713 mgr-fin ceph rtd unknown /proc/31438/task/41713/root (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41713 mgr-fin ceph txt unknown /proc/31438/task/41713/exe (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41713 mgr-fin ceph NOFD /proc/31438/task/41713/fd (opendir: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41714 mgr-fin ceph cwd unknown /proc/31438/task/41714/cwd (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41714 mgr-fin ceph rtd unknown /proc/31438/task/41714/root (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41714 mgr-fin ceph txt unknown /proc/31438/task/41714/exe (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41714 mgr-fin ceph NOFD /proc/31438/task/41714/fd (opendir: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41715 mgr-fin ceph cwd unknown /proc/31438/task/41715/cwd (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41715 mgr-fin ceph rtd unknown /proc/31438/task/41715/root (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41715 mgr-fin ceph txt unknown /proc/31438/task/41715/exe (readlink: Permission denied) 2024-09-21T02:55:14.022 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41715 mgr-fin ceph NOFD /proc/31438/task/41715/fd (opendir: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41723 mgr-fin ceph cwd unknown /proc/31438/task/41723/cwd (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41723 mgr-fin ceph rtd unknown /proc/31438/task/41723/root (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41723 mgr-fin ceph txt unknown /proc/31438/task/41723/exe (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41723 mgr-fin ceph NOFD /proc/31438/task/41723/fd (opendir: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41725 mgr-fin ceph cwd unknown /proc/31438/task/41725/cwd (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41725 mgr-fin ceph rtd unknown /proc/31438/task/41725/root (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41725 mgr-fin ceph txt unknown /proc/31438/task/41725/exe (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41725 mgr-fin ceph NOFD /proc/31438/task/41725/fd (opendir: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41726 mgr-fin ceph cwd unknown /proc/31438/task/41726/cwd (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41726 mgr-fin ceph rtd unknown /proc/31438/task/41726/root (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41726 mgr-fin ceph txt unknown /proc/31438/task/41726/exe (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 41726 mgr-fin ceph NOFD /proc/31438/task/41726/fd (opendir: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52375 devicehea ceph cwd unknown /proc/31438/task/52375/cwd (readlink: Permission denied) 2024-09-21T02:55:14.023 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52375 devicehea ceph rtd unknown /proc/31438/task/52375/root (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52375 devicehea ceph txt unknown /proc/31438/task/52375/exe (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52375 devicehea ceph NOFD /proc/31438/task/52375/fd (opendir: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52381 io_contex ceph cwd unknown /proc/31438/task/52381/cwd (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52381 io_contex ceph rtd unknown /proc/31438/task/52381/root (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52381 io_contex ceph txt unknown /proc/31438/task/52381/exe (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52381 io_contex ceph NOFD /proc/31438/task/52381/fd (opendir: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52382 io_contex ceph cwd unknown /proc/31438/task/52382/cwd (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52382 io_contex ceph rtd unknown /proc/31438/task/52382/root (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52382 io_contex ceph txt unknown /proc/31438/task/52382/exe (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52382 io_contex ceph NOFD /proc/31438/task/52382/fd (opendir: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52383 ceph_time ceph cwd unknown /proc/31438/task/52383/cwd (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52383 ceph_time ceph rtd unknown /proc/31438/task/52383/root (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52383 ceph_time ceph txt unknown /proc/31438/task/52383/exe (readlink: Permission denied) 2024-09-21T02:55:14.024 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52383 ceph_time ceph NOFD /proc/31438/task/52383/fd (opendir: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52384 ms_dispat ceph cwd unknown /proc/31438/task/52384/cwd (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52384 ms_dispat ceph rtd unknown /proc/31438/task/52384/root (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52384 ms_dispat ceph txt unknown /proc/31438/task/52384/exe (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52384 ms_dispat ceph NOFD /proc/31438/task/52384/fd (opendir: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52385 ms_local ceph cwd unknown /proc/31438/task/52385/cwd (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52385 ms_local ceph rtd unknown /proc/31438/task/52385/root (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52385 ms_local ceph txt unknown /proc/31438/task/52385/exe (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52385 ms_local ceph NOFD /proc/31438/task/52385/fd (opendir: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52386 safe_time ceph cwd unknown /proc/31438/task/52386/cwd (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52386 safe_time ceph rtd unknown /proc/31438/task/52386/root (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52386 safe_time ceph txt unknown /proc/31438/task/52386/exe (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52386 safe_time ceph NOFD /proc/31438/task/52386/fd (opendir: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52387 safe_time ceph cwd unknown /proc/31438/task/52387/cwd (readlink: Permission denied) 2024-09-21T02:55:14.025 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52387 safe_time ceph rtd unknown /proc/31438/task/52387/root (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52387 safe_time ceph txt unknown /proc/31438/task/52387/exe (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 52387 safe_time ceph NOFD /proc/31438/task/52387/fd (opendir: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85494 io_contex ceph cwd unknown /proc/31438/task/85494/cwd (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85494 io_contex ceph rtd unknown /proc/31438/task/85494/root (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85494 io_contex ceph txt unknown /proc/31438/task/85494/exe (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85494 io_contex ceph NOFD /proc/31438/task/85494/fd (opendir: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85495 io_contex ceph cwd unknown /proc/31438/task/85495/cwd (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85495 io_contex ceph rtd unknown /proc/31438/task/85495/root (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85495 io_contex ceph txt unknown /proc/31438/task/85495/exe (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 85495 io_contex ceph NOFD /proc/31438/task/85495/fd (opendir: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86252 m-fin-sna ceph cwd unknown /proc/31438/task/86252/cwd (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86252 m-fin-sna ceph rtd unknown /proc/31438/task/86252/root (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86252 m-fin-sna ceph txt unknown /proc/31438/task/86252/exe (readlink: Permission denied) 2024-09-21T02:55:14.026 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86252 m-fin-sna ceph NOFD /proc/31438/task/86252/fd (opendir: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86256 io_contex ceph cwd unknown /proc/31438/task/86256/cwd (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86256 io_contex ceph rtd unknown /proc/31438/task/86256/root (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86256 io_contex ceph txt unknown /proc/31438/task/86256/exe (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86256 io_contex ceph NOFD /proc/31438/task/86256/fd (opendir: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86257 io_contex ceph cwd unknown /proc/31438/task/86257/cwd (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86257 io_contex ceph rtd unknown /proc/31438/task/86257/root (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86257 io_contex ceph txt unknown /proc/31438/task/86257/exe (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86257 io_contex ceph NOFD /proc/31438/task/86257/fd (opendir: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86258 ceph_time ceph cwd unknown /proc/31438/task/86258/cwd (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86258 ceph_time ceph rtd unknown /proc/31438/task/86258/root (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86258 ceph_time ceph txt unknown /proc/31438/task/86258/exe (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86258 ceph_time ceph NOFD /proc/31438/task/86258/fd (opendir: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86259 ms_dispat ceph cwd unknown /proc/31438/task/86259/cwd (readlink: Permission denied) 2024-09-21T02:55:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86259 ms_dispat ceph rtd unknown /proc/31438/task/86259/root (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86259 ms_dispat ceph txt unknown /proc/31438/task/86259/exe (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86259 ms_dispat ceph NOFD /proc/31438/task/86259/fd (opendir: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86260 ms_local ceph cwd unknown /proc/31438/task/86260/cwd (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86260 ms_local ceph rtd unknown /proc/31438/task/86260/root (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86260 ms_local ceph txt unknown /proc/31438/task/86260/exe (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86260 ms_local ceph NOFD /proc/31438/task/86260/fd (opendir: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86261 safe_time ceph cwd unknown /proc/31438/task/86261/cwd (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86261 safe_time ceph rtd unknown /proc/31438/task/86261/root (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86261 safe_time ceph txt unknown /proc/31438/task/86261/exe (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86261 safe_time ceph NOFD /proc/31438/task/86261/fd (opendir: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86262 safe_time ceph cwd unknown /proc/31438/task/86262/cwd (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86262 safe_time ceph rtd unknown /proc/31438/task/86262/root (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86262 safe_time ceph txt unknown /proc/31438/task/86262/exe (readlink: Permission denied) 2024-09-21T02:55:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 86262 safe_time ceph NOFD /proc/31438/task/86262/fd (opendir: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88274 ceph_time ceph cwd unknown /proc/31438/task/88274/cwd (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88274 ceph_time ceph rtd unknown /proc/31438/task/88274/root (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88274 ceph_time ceph txt unknown /proc/31438/task/88274/exe (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88274 ceph_time ceph NOFD /proc/31438/task/88274/fd (opendir: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88275 fn_anonym ceph cwd unknown /proc/31438/task/88275/cwd (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88275 fn_anonym ceph rtd unknown /proc/31438/task/88275/root (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88275 fn_anonym ceph txt unknown /proc/31438/task/88275/exe (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88275 fn_anonym ceph NOFD /proc/31438/task/88275/fd (opendir: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88276 safe_time ceph cwd unknown /proc/31438/task/88276/cwd (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88276 safe_time ceph rtd unknown /proc/31438/task/88276/root (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88276 safe_time ceph txt unknown /proc/31438/task/88276/exe (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88276 safe_time ceph NOFD /proc/31438/task/88276/fd (opendir: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88277 fn_anonym ceph cwd unknown /proc/31438/task/88277/cwd (readlink: Permission denied) 2024-09-21T02:55:14.029 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88277 fn_anonym ceph rtd unknown /proc/31438/task/88277/root (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88277 fn_anonym ceph txt unknown /proc/31438/task/88277/exe (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88277 fn_anonym ceph NOFD /proc/31438/task/88277/fd (opendir: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88278 flusher ceph cwd unknown /proc/31438/task/88278/cwd (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88278 flusher ceph rtd unknown /proc/31438/task/88278/root (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88278 flusher ceph txt unknown /proc/31438/task/88278/exe (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88278 flusher ceph NOFD /proc/31438/task/88278/fd (opendir: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88279 ms_dispat ceph cwd unknown /proc/31438/task/88279/cwd (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88279 ms_dispat ceph rtd unknown /proc/31438/task/88279/root (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88279 ms_dispat ceph txt unknown /proc/31438/task/88279/exe (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88279 ms_dispat ceph NOFD /proc/31438/task/88279/fd (opendir: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88280 ms_local ceph cwd unknown /proc/31438/task/88280/cwd (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88280 ms_local ceph rtd unknown /proc/31438/task/88280/root (readlink: Permission denied) 2024-09-21T02:55:14.030 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88280 ms_local ceph txt unknown /proc/31438/task/88280/exe (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88280 ms_local ceph NOFD /proc/31438/task/88280/fd (opendir: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88281 safe_time ceph cwd unknown /proc/31438/task/88281/cwd (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88281 safe_time ceph rtd unknown /proc/31438/task/88281/root (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88281 safe_time ceph txt unknown /proc/31438/task/88281/exe (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88281 safe_time ceph NOFD /proc/31438/task/88281/fd (opendir: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88282 m-fin-sna ceph cwd unknown /proc/31438/task/88282/cwd (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88282 m-fin-sna ceph rtd unknown /proc/31438/task/88282/root (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88282 m-fin-sna ceph txt unknown /proc/31438/task/88282/exe (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 88282 m-fin-sna ceph NOFD /proc/31438/task/88282/fd (opendir: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 91122 m-fin-sna ceph cwd unknown /proc/31438/task/91122/cwd (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 91122 m-fin-sna ceph rtd unknown /proc/31438/task/91122/root (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 91122 m-fin-sna ceph txt unknown /proc/31438/task/91122/exe (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mgr 31438 91122 m-fin-sna ceph NOFD /proc/31438/task/91122/fd (opendir: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 31987 root cwd unknown /proc/31987/cwd (readlink: Permission denied) 2024-09-21T02:55:14.031 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 31987 root rtd unknown /proc/31987/root (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 31987 root txt unknown /proc/31987/exe (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 31987 root NOFD /proc/31987/fd (opendir: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 32468 root cwd unknown /proc/32468/cwd (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 32468 root rtd unknown /proc/32468/root (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 32468 root txt unknown /proc/32468/exe (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:systemd 32468 root NOFD /proc/32468/fd (opendir: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 32470 root cwd unknown /proc/32470/cwd (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 32470 root rtd unknown /proc/32470/root (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 32470 root txt unknown /proc/32470/exe (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:(sd-pam) 32470 root NOFD /proc/32470/fd (opendir: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 33530 root cwd unknown /proc/33530/cwd (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 33530 root rtd unknown /proc/33530/root (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 33530 root txt unknown /proc/33530/exe (readlink: Permission denied) 2024-09-21T02:55:14.032 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 33530 root NOFD /proc/33530/fd (opendir: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 33666 root cwd unknown /proc/33666/cwd (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 33666 root rtd unknown /proc/33666/root (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 33666 root txt unknown /proc/33666/exe (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 33666 root NOFD /proc/33666/fd (opendir: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 36998 root cwd unknown /proc/36998/cwd (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 36998 root rtd unknown /proc/36998/root (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 36998 root txt unknown /proc/36998/exe (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 36998 root NOFD /proc/36998/fd (opendir: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41098 root cwd unknown /proc/41098/cwd (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41098 root rtd unknown /proc/41098/root (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41098 root txt unknown /proc/41098/exe (readlink: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41098 root NOFD /proc/41098/fd (opendir: Permission denied) 2024-09-21T02:55:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41101 root cwd unknown /proc/41101/cwd (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41101 root rtd unknown /proc/41101/root (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41101 root txt unknown /proc/41101/exe (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:sshd 41101 root NOFD /proc/41101/fd (opendir: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 47986 root cwd unknown /proc/47986/cwd (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 47986 root rtd unknown /proc/47986/root (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 47986 root txt unknown /proc/47986/exe (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 47986 root NOFD /proc/47986/fd (opendir: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 47988 root cwd unknown /proc/47988/cwd (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 47988 root rtd unknown /proc/47988/root (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 47988 root txt unknown /proc/47988/exe (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 47988 root NOFD /proc/47988/fd (opendir: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 ceph cwd unknown /proc/47990/cwd (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 ceph rtd unknown /proc/47990/root (readlink: Permission denied) 2024-09-21T02:55:14.034 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 ceph txt unknown /proc/47990/exe (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 ceph NOFD /proc/47990/fd (opendir: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47991 log ceph cwd unknown /proc/47990/task/47991/cwd (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47991 log ceph rtd unknown /proc/47990/task/47991/root (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47991 log ceph txt unknown /proc/47990/task/47991/exe (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47991 log ceph NOFD /proc/47990/task/47991/fd (opendir: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47993 msgr-work ceph cwd unknown /proc/47990/task/47993/cwd (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47993 msgr-work ceph rtd unknown /proc/47990/task/47993/root (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47993 msgr-work ceph txt unknown /proc/47990/task/47993/exe (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47993 msgr-work ceph NOFD /proc/47990/task/47993/fd (opendir: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47994 msgr-work ceph cwd unknown /proc/47990/task/47994/cwd (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47994 msgr-work ceph rtd unknown /proc/47990/task/47994/root (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47994 msgr-work ceph txt unknown /proc/47990/task/47994/exe (readlink: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47994 msgr-work ceph NOFD /proc/47990/task/47994/fd (opendir: Permission denied) 2024-09-21T02:55:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47995 msgr-work ceph cwd unknown /proc/47990/task/47995/cwd (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47995 msgr-work ceph rtd unknown /proc/47990/task/47995/root (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47995 msgr-work ceph txt unknown /proc/47990/task/47995/exe (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47995 msgr-work ceph NOFD /proc/47990/task/47995/fd (opendir: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47999 service ceph cwd unknown /proc/47990/task/47999/cwd (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47999 service ceph rtd unknown /proc/47990/task/47999/root (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47999 service ceph txt unknown /proc/47990/task/47999/exe (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 47999 service ceph NOFD /proc/47990/task/47999/fd (opendir: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48000 admin_soc ceph cwd unknown /proc/47990/task/48000/cwd (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48000 admin_soc ceph rtd unknown /proc/47990/task/48000/root (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48000 admin_soc ceph txt unknown /proc/47990/task/48000/exe (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48000 admin_soc ceph NOFD /proc/47990/task/48000/fd (opendir: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48264 signal_ha ceph cwd unknown /proc/47990/task/48264/cwd (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48264 signal_ha ceph rtd unknown /proc/47990/task/48264/root (readlink: Permission denied) 2024-09-21T02:55:14.036 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48264 signal_ha ceph txt unknown /proc/47990/task/48264/exe (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48264 signal_ha ceph NOFD /proc/47990/task/48264/fd (opendir: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48265 io_contex ceph cwd unknown /proc/47990/task/48265/cwd (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48265 io_contex ceph rtd unknown /proc/47990/task/48265/root (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48265 io_contex ceph txt unknown /proc/47990/task/48265/exe (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48265 io_contex ceph NOFD /proc/47990/task/48265/fd (opendir: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48266 io_contex ceph cwd unknown /proc/47990/task/48266/cwd (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48266 io_contex ceph rtd unknown /proc/47990/task/48266/root (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48266 io_contex ceph txt unknown /proc/47990/task/48266/exe (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48266 io_contex ceph NOFD /proc/47990/task/48266/fd (opendir: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48286 OpHistory ceph cwd unknown /proc/47990/task/48286/cwd (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48286 OpHistory ceph rtd unknown /proc/47990/task/48286/root (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48286 OpHistory ceph txt unknown /proc/47990/task/48286/exe (readlink: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48286 OpHistory ceph NOFD /proc/47990/task/48286/fd (opendir: Permission denied) 2024-09-21T02:55:14.037 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48287 ceph_time ceph cwd unknown /proc/47990/task/48287/cwd (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48287 ceph_time ceph rtd unknown /proc/47990/task/48287/root (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48287 ceph_time ceph txt unknown /proc/47990/task/48287/exe (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48287 ceph_time ceph NOFD /proc/47990/task/48287/fd (opendir: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48418 ceph-osd ceph cwd unknown /proc/47990/task/48418/cwd (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48418 ceph-osd ceph rtd unknown /proc/47990/task/48418/root (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48418 ceph-osd ceph txt unknown /proc/47990/task/48418/exe (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48418 ceph-osd ceph NOFD /proc/47990/task/48418/fd (opendir: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48419 safe_time ceph cwd unknown /proc/47990/task/48419/cwd (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48419 safe_time ceph rtd unknown /proc/47990/task/48419/root (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48419 safe_time ceph txt unknown /proc/47990/task/48419/exe (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48419 safe_time ceph NOFD /proc/47990/task/48419/fd (opendir: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48420 safe_time ceph cwd unknown /proc/47990/task/48420/cwd (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48420 safe_time ceph rtd unknown /proc/47990/task/48420/root (readlink: Permission denied) 2024-09-21T02:55:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48420 safe_time ceph txt unknown /proc/47990/task/48420/exe (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48420 safe_time ceph NOFD /proc/47990/task/48420/fd (opendir: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48421 safe_time ceph cwd unknown /proc/47990/task/48421/cwd (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48421 safe_time ceph rtd unknown /proc/47990/task/48421/root (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48421 safe_time ceph txt unknown /proc/47990/task/48421/exe (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48421 safe_time ceph NOFD /proc/47990/task/48421/fd (opendir: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48422 safe_time ceph cwd unknown /proc/47990/task/48422/cwd (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48422 safe_time ceph rtd unknown /proc/47990/task/48422/root (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48422 safe_time ceph txt unknown /proc/47990/task/48422/exe (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48422 safe_time ceph NOFD /proc/47990/task/48422/fd (opendir: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48423 fn_anonym ceph cwd unknown /proc/47990/task/48423/cwd (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48423 fn_anonym ceph rtd unknown /proc/47990/task/48423/root (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48423 fn_anonym ceph txt unknown /proc/47990/task/48423/exe (readlink: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48423 fn_anonym ceph NOFD /proc/47990/task/48423/fd (opendir: Permission denied) 2024-09-21T02:55:14.039 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48483 rocksdb:l ceph cwd unknown /proc/47990/task/48483/cwd (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48483 rocksdb:l ceph rtd unknown /proc/47990/task/48483/root (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48483 rocksdb:l ceph txt unknown /proc/47990/task/48483/exe (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48483 rocksdb:l ceph NOFD /proc/47990/task/48483/fd (opendir: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48484 rocksdb:l ceph cwd unknown /proc/47990/task/48484/cwd (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48484 rocksdb:l ceph rtd unknown /proc/47990/task/48484/root (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48484 rocksdb:l ceph txt unknown /proc/47990/task/48484/exe (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48484 rocksdb:l ceph NOFD /proc/47990/task/48484/fd (opendir: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48485 rocksdb:l ceph cwd unknown /proc/47990/task/48485/cwd (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48485 rocksdb:l ceph rtd unknown /proc/47990/task/48485/root (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48485 rocksdb:l ceph txt unknown /proc/47990/task/48485/exe (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48485 rocksdb:l ceph NOFD /proc/47990/task/48485/fd (opendir: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48486 rocksdb:h ceph cwd unknown /proc/47990/task/48486/cwd (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48486 rocksdb:h ceph rtd unknown /proc/47990/task/48486/root (readlink: Permission denied) 2024-09-21T02:55:14.040 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48486 rocksdb:h ceph txt unknown /proc/47990/task/48486/exe (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 48486 rocksdb:h ceph NOFD /proc/47990/task/48486/fd (opendir: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49006 bstore_ai ceph cwd unknown /proc/47990/task/49006/cwd (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49006 bstore_ai ceph rtd unknown /proc/47990/task/49006/root (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49006 bstore_ai ceph txt unknown /proc/47990/task/49006/exe (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49006 bstore_ai ceph NOFD /proc/47990/task/49006/fd (opendir: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49190 bstore_ai ceph cwd unknown /proc/47990/task/49190/cwd (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49190 bstore_ai ceph rtd unknown /proc/47990/task/49190/root (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49190 bstore_ai ceph txt unknown /proc/47990/task/49190/exe (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49190 bstore_ai ceph NOFD /proc/47990/task/49190/fd (opendir: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49372 ceph-osd ceph cwd unknown /proc/47990/task/49372/cwd (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49372 ceph-osd ceph rtd unknown /proc/47990/task/49372/root (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49372 ceph-osd ceph txt unknown /proc/47990/task/49372/exe (readlink: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49372 ceph-osd ceph NOFD /proc/47990/task/49372/fd (opendir: Permission denied) 2024-09-21T02:55:14.041 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49373 cfin ceph cwd unknown /proc/47990/task/49373/cwd (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49373 cfin ceph rtd unknown /proc/47990/task/49373/root (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49373 cfin ceph txt unknown /proc/47990/task/49373/exe (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49373 cfin ceph NOFD /proc/47990/task/49373/fd (opendir: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49374 bstore_kv ceph cwd unknown /proc/47990/task/49374/cwd (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49374 bstore_kv ceph rtd unknown /proc/47990/task/49374/root (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49374 bstore_kv ceph txt unknown /proc/47990/task/49374/exe (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49374 bstore_kv ceph NOFD /proc/47990/task/49374/fd (opendir: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49375 bstore_kv ceph cwd unknown /proc/47990/task/49375/cwd (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49375 bstore_kv ceph rtd unknown /proc/47990/task/49375/root (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49375 bstore_kv ceph txt unknown /proc/47990/task/49375/exe (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49375 bstore_kv ceph NOFD /proc/47990/task/49375/fd (opendir: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49376 bstore_me ceph cwd unknown /proc/47990/task/49376/cwd (readlink: Permission denied) 2024-09-21T02:55:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49376 bstore_me ceph rtd unknown /proc/47990/task/49376/root (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49376 bstore_me ceph txt unknown /proc/47990/task/49376/exe (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49376 bstore_me ceph NOFD /proc/47990/task/49376/fd (opendir: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49377 ms_dispat ceph cwd unknown /proc/47990/task/49377/cwd (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49377 ms_dispat ceph rtd unknown /proc/47990/task/49377/root (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49377 ms_dispat ceph txt unknown /proc/47990/task/49377/exe (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49377 ms_dispat ceph NOFD /proc/47990/task/49377/fd (opendir: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49378 ms_local ceph cwd unknown /proc/47990/task/49378/cwd (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49378 ms_local ceph rtd unknown /proc/47990/task/49378/root (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49378 ms_local ceph txt unknown /proc/47990/task/49378/exe (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49378 ms_local ceph NOFD /proc/47990/task/49378/fd (opendir: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49379 safe_time ceph cwd unknown /proc/47990/task/49379/cwd (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49379 safe_time ceph rtd unknown /proc/47990/task/49379/root (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49379 safe_time ceph txt unknown /proc/47990/task/49379/exe (readlink: Permission denied) 2024-09-21T02:55:14.043 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49379 safe_time ceph NOFD /proc/47990/task/49379/fd (opendir: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49380 safe_time ceph cwd unknown /proc/47990/task/49380/cwd (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49380 safe_time ceph rtd unknown /proc/47990/task/49380/root (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49380 safe_time ceph txt unknown /proc/47990/task/49380/exe (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49380 safe_time ceph NOFD /proc/47990/task/49380/fd (opendir: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49381 ms_dispat ceph cwd unknown /proc/47990/task/49381/cwd (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49381 ms_dispat ceph rtd unknown /proc/47990/task/49381/root (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49381 ms_dispat ceph txt unknown /proc/47990/task/49381/exe (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49381 ms_dispat ceph NOFD /proc/47990/task/49381/fd (opendir: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49382 ms_local ceph cwd unknown /proc/47990/task/49382/cwd (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49382 ms_local ceph rtd unknown /proc/47990/task/49382/root (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49382 ms_local ceph txt unknown /proc/47990/task/49382/exe (readlink: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49382 ms_local ceph NOFD /proc/47990/task/49382/fd (opendir: Permission denied) 2024-09-21T02:55:14.044 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49383 ms_dispat ceph cwd unknown /proc/47990/task/49383/cwd (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49383 ms_dispat ceph rtd unknown /proc/47990/task/49383/root (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49383 ms_dispat ceph txt unknown /proc/47990/task/49383/exe (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49383 ms_dispat ceph NOFD /proc/47990/task/49383/fd (opendir: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49384 ms_local ceph cwd unknown /proc/47990/task/49384/cwd (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49384 ms_local ceph rtd unknown /proc/47990/task/49384/root (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49384 ms_local ceph txt unknown /proc/47990/task/49384/exe (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49384 ms_local ceph NOFD /proc/47990/task/49384/fd (opendir: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49385 ms_dispat ceph cwd unknown /proc/47990/task/49385/cwd (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49385 ms_dispat ceph rtd unknown /proc/47990/task/49385/root (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49385 ms_dispat ceph txt unknown /proc/47990/task/49385/exe (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49385 ms_dispat ceph NOFD /proc/47990/task/49385/fd (opendir: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49386 ms_local ceph cwd unknown /proc/47990/task/49386/cwd (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49386 ms_local ceph rtd unknown /proc/47990/task/49386/root (readlink: Permission denied) 2024-09-21T02:55:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49386 ms_local ceph txt unknown /proc/47990/task/49386/exe (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49386 ms_local ceph NOFD /proc/47990/task/49386/fd (opendir: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49387 ms_dispat ceph cwd unknown /proc/47990/task/49387/cwd (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49387 ms_dispat ceph rtd unknown /proc/47990/task/49387/root (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49387 ms_dispat ceph txt unknown /proc/47990/task/49387/exe (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49387 ms_dispat ceph NOFD /proc/47990/task/49387/fd (opendir: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49388 ms_local ceph cwd unknown /proc/47990/task/49388/cwd (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49388 ms_local ceph rtd unknown /proc/47990/task/49388/root (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49388 ms_local ceph txt unknown /proc/47990/task/49388/exe (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49388 ms_local ceph NOFD /proc/47990/task/49388/fd (opendir: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49389 ms_dispat ceph cwd unknown /proc/47990/task/49389/cwd (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49389 ms_dispat ceph rtd unknown /proc/47990/task/49389/root (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49389 ms_dispat ceph txt unknown /proc/47990/task/49389/exe (readlink: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49389 ms_dispat ceph NOFD /proc/47990/task/49389/fd (opendir: Permission denied) 2024-09-21T02:55:14.046 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49390 ms_local ceph cwd unknown /proc/47990/task/49390/cwd (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49390 ms_local ceph rtd unknown /proc/47990/task/49390/root (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49390 ms_local ceph txt unknown /proc/47990/task/49390/exe (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49390 ms_local ceph NOFD /proc/47990/task/49390/fd (opendir: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49391 ms_dispat ceph cwd unknown /proc/47990/task/49391/cwd (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49391 ms_dispat ceph rtd unknown /proc/47990/task/49391/root (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49391 ms_dispat ceph txt unknown /proc/47990/task/49391/exe (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49391 ms_dispat ceph NOFD /proc/47990/task/49391/fd (opendir: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49392 ms_local ceph cwd unknown /proc/47990/task/49392/cwd (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49392 ms_local ceph rtd unknown /proc/47990/task/49392/root (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49392 ms_local ceph txt unknown /proc/47990/task/49392/exe (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49392 ms_local ceph NOFD /proc/47990/task/49392/fd (opendir: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49393 fn_anonym ceph cwd unknown /proc/47990/task/49393/cwd (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49393 fn_anonym ceph rtd unknown /proc/47990/task/49393/root (readlink: Permission denied) 2024-09-21T02:55:14.047 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49393 fn_anonym ceph txt unknown /proc/47990/task/49393/exe (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49393 fn_anonym ceph NOFD /proc/47990/task/49393/fd (opendir: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49394 finisher ceph cwd unknown /proc/47990/task/49394/cwd (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49394 finisher ceph rtd unknown /proc/47990/task/49394/root (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49394 finisher ceph txt unknown /proc/47990/task/49394/exe (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49394 finisher ceph NOFD /proc/47990/task/49394/fd (opendir: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49395 safe_time ceph cwd unknown /proc/47990/task/49395/cwd (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49395 safe_time ceph rtd unknown /proc/47990/task/49395/root (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49395 safe_time ceph txt unknown /proc/47990/task/49395/exe (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49395 safe_time ceph NOFD /proc/47990/task/49395/fd (opendir: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49396 safe_time ceph cwd unknown /proc/47990/task/49396/cwd (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49396 safe_time ceph rtd unknown /proc/47990/task/49396/root (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49396 safe_time ceph txt unknown /proc/47990/task/49396/exe (readlink: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49396 safe_time ceph NOFD /proc/47990/task/49396/fd (opendir: Permission denied) 2024-09-21T02:55:14.048 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49397 ceph-osd ceph cwd unknown /proc/47990/task/49397/cwd (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49397 ceph-osd ceph rtd unknown /proc/47990/task/49397/root (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49397 ceph-osd ceph txt unknown /proc/47990/task/49397/exe (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49397 ceph-osd ceph NOFD /proc/47990/task/49397/fd (opendir: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49398 osd_srv_a ceph cwd unknown /proc/47990/task/49398/cwd (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49398 osd_srv_a ceph rtd unknown /proc/47990/task/49398/root (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49398 osd_srv_a ceph txt unknown /proc/47990/task/49398/exe (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49398 osd_srv_a ceph NOFD /proc/47990/task/49398/fd (opendir: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49399 tp_osd_tp ceph cwd unknown /proc/47990/task/49399/cwd (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49399 tp_osd_tp ceph rtd unknown /proc/47990/task/49399/root (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49399 tp_osd_tp ceph txt unknown /proc/47990/task/49399/exe (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49399 tp_osd_tp ceph NOFD /proc/47990/task/49399/fd (opendir: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49400 tp_osd_tp ceph cwd unknown /proc/47990/task/49400/cwd (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49400 tp_osd_tp ceph rtd unknown /proc/47990/task/49400/root (readlink: Permission denied) 2024-09-21T02:55:14.049 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49400 tp_osd_tp ceph txt unknown /proc/47990/task/49400/exe (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49400 tp_osd_tp ceph NOFD /proc/47990/task/49400/fd (opendir: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49401 tp_osd_tp ceph cwd unknown /proc/47990/task/49401/cwd (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49401 tp_osd_tp ceph rtd unknown /proc/47990/task/49401/root (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49401 tp_osd_tp ceph txt unknown /proc/47990/task/49401/exe (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49401 tp_osd_tp ceph NOFD /proc/47990/task/49401/fd (opendir: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49402 tp_osd_tp ceph cwd unknown /proc/47990/task/49402/cwd (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49402 tp_osd_tp ceph rtd unknown /proc/47990/task/49402/root (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49402 tp_osd_tp ceph txt unknown /proc/47990/task/49402/exe (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49402 tp_osd_tp ceph NOFD /proc/47990/task/49402/fd (opendir: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49403 tp_osd_tp ceph cwd unknown /proc/47990/task/49403/cwd (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49403 tp_osd_tp ceph rtd unknown /proc/47990/task/49403/root (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49403 tp_osd_tp ceph txt unknown /proc/47990/task/49403/exe (readlink: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49403 tp_osd_tp ceph NOFD /proc/47990/task/49403/fd (opendir: Permission denied) 2024-09-21T02:55:14.050 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49404 tp_osd_tp ceph cwd unknown /proc/47990/task/49404/cwd (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49404 tp_osd_tp ceph rtd unknown /proc/47990/task/49404/root (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49404 tp_osd_tp ceph txt unknown /proc/47990/task/49404/exe (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49404 tp_osd_tp ceph NOFD /proc/47990/task/49404/fd (opendir: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49405 tp_osd_tp ceph cwd unknown /proc/47990/task/49405/cwd (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49405 tp_osd_tp ceph rtd unknown /proc/47990/task/49405/root (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49405 tp_osd_tp ceph txt unknown /proc/47990/task/49405/exe (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49405 tp_osd_tp ceph NOFD /proc/47990/task/49405/fd (opendir: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49406 tp_osd_tp ceph cwd unknown /proc/47990/task/49406/cwd (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49406 tp_osd_tp ceph rtd unknown /proc/47990/task/49406/root (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49406 tp_osd_tp ceph txt unknown /proc/47990/task/49406/exe (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49406 tp_osd_tp ceph NOFD /proc/47990/task/49406/fd (opendir: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49407 tp_osd_tp ceph cwd unknown /proc/47990/task/49407/cwd (readlink: Permission denied) 2024-09-21T02:55:14.051 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49407 tp_osd_tp ceph rtd unknown /proc/47990/task/49407/root (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49407 tp_osd_tp ceph txt unknown /proc/47990/task/49407/exe (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49407 tp_osd_tp ceph NOFD /proc/47990/task/49407/fd (opendir: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49408 tp_osd_tp ceph cwd unknown /proc/47990/task/49408/cwd (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49408 tp_osd_tp ceph rtd unknown /proc/47990/task/49408/root (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49408 tp_osd_tp ceph txt unknown /proc/47990/task/49408/exe (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49408 tp_osd_tp ceph NOFD /proc/47990/task/49408/fd (opendir: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49409 tp_osd_tp ceph cwd unknown /proc/47990/task/49409/cwd (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49409 tp_osd_tp ceph rtd unknown /proc/47990/task/49409/root (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49409 tp_osd_tp ceph txt unknown /proc/47990/task/49409/exe (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49409 tp_osd_tp ceph NOFD /proc/47990/task/49409/fd (opendir: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49410 tp_osd_tp ceph cwd unknown /proc/47990/task/49410/cwd (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49410 tp_osd_tp ceph rtd unknown /proc/47990/task/49410/root (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49410 tp_osd_tp ceph txt unknown /proc/47990/task/49410/exe (readlink: Permission denied) 2024-09-21T02:55:14.052 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49410 tp_osd_tp ceph NOFD /proc/47990/task/49410/fd (opendir: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49411 tp_osd_tp ceph cwd unknown /proc/47990/task/49411/cwd (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49411 tp_osd_tp ceph rtd unknown /proc/47990/task/49411/root (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49411 tp_osd_tp ceph txt unknown /proc/47990/task/49411/exe (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49411 tp_osd_tp ceph NOFD /proc/47990/task/49411/fd (opendir: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49412 tp_osd_tp ceph cwd unknown /proc/47990/task/49412/cwd (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49412 tp_osd_tp ceph rtd unknown /proc/47990/task/49412/root (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49412 tp_osd_tp ceph txt unknown /proc/47990/task/49412/exe (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49412 tp_osd_tp ceph NOFD /proc/47990/task/49412/fd (opendir: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49413 tp_osd_tp ceph cwd unknown /proc/47990/task/49413/cwd (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49413 tp_osd_tp ceph rtd unknown /proc/47990/task/49413/root (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49413 tp_osd_tp ceph txt unknown /proc/47990/task/49413/exe (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49413 tp_osd_tp ceph NOFD /proc/47990/task/49413/fd (opendir: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49414 tp_osd_tp ceph cwd unknown /proc/47990/task/49414/cwd (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49414 tp_osd_tp ceph rtd unknown /proc/47990/task/49414/root (readlink: Permission denied) 2024-09-21T02:55:14.053 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49414 tp_osd_tp ceph txt unknown /proc/47990/task/49414/exe (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49414 tp_osd_tp ceph NOFD /proc/47990/task/49414/fd (opendir: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49415 osd_srv_h ceph cwd unknown /proc/47990/task/49415/cwd (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49415 osd_srv_h ceph rtd unknown /proc/47990/task/49415/root (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49415 osd_srv_h ceph txt unknown /proc/47990/task/49415/exe (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 47990 49415 osd_srv_h ceph NOFD /proc/47990/task/49415/fd (opendir: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 48942 root cwd unknown /proc/48942/cwd (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 48942 root rtd unknown /proc/48942/root (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 48942 root txt unknown /proc/48942/exe (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 48942 root NOFD /proc/48942/fd (opendir: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 48971 root cwd unknown /proc/48971/cwd (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 48971 root rtd unknown /proc/48971/root (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 48971 root txt unknown /proc/48971/exe (readlink: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 48971 root NOFD /proc/48971/fd (opendir: Permission denied) 2024-09-21T02:55:14.054 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 54384 root cwd unknown /proc/54384/cwd (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 54384 root rtd unknown /proc/54384/root (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 54384 root txt unknown /proc/54384/exe (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 54384 root NOFD /proc/54384/fd (opendir: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 54386 root cwd unknown /proc/54386/cwd (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 54386 root rtd unknown /proc/54386/root (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 54386 root txt unknown /proc/54386/exe (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 54386 root NOFD /proc/54386/fd (opendir: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 ceph cwd unknown /proc/54388/cwd (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 ceph rtd unknown /proc/54388/root (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 ceph txt unknown /proc/54388/exe (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 ceph NOFD /proc/54388/fd (opendir: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54389 log ceph cwd unknown /proc/54388/task/54389/cwd (readlink: Permission denied) 2024-09-21T02:55:14.055 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54389 log ceph rtd unknown /proc/54388/task/54389/root (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54389 log ceph txt unknown /proc/54388/task/54389/exe (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54389 log ceph NOFD /proc/54388/task/54389/fd (opendir: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54391 msgr-work ceph cwd unknown /proc/54388/task/54391/cwd (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54391 msgr-work ceph rtd unknown /proc/54388/task/54391/root (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54391 msgr-work ceph txt unknown /proc/54388/task/54391/exe (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54391 msgr-work ceph NOFD /proc/54388/task/54391/fd (opendir: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54392 msgr-work ceph cwd unknown /proc/54388/task/54392/cwd (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54392 msgr-work ceph rtd unknown /proc/54388/task/54392/root (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54392 msgr-work ceph txt unknown /proc/54388/task/54392/exe (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54392 msgr-work ceph NOFD /proc/54388/task/54392/fd (opendir: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54393 msgr-work ceph cwd unknown /proc/54388/task/54393/cwd (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54393 msgr-work ceph rtd unknown /proc/54388/task/54393/root (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54393 msgr-work ceph txt unknown /proc/54388/task/54393/exe (readlink: Permission denied) 2024-09-21T02:55:14.056 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54393 msgr-work ceph NOFD /proc/54388/task/54393/fd (opendir: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54397 service ceph cwd unknown /proc/54388/task/54397/cwd (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54397 service ceph rtd unknown /proc/54388/task/54397/root (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54397 service ceph txt unknown /proc/54388/task/54397/exe (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54397 service ceph NOFD /proc/54388/task/54397/fd (opendir: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54398 admin_soc ceph cwd unknown /proc/54388/task/54398/cwd (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54398 admin_soc ceph rtd unknown /proc/54388/task/54398/root (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54398 admin_soc ceph txt unknown /proc/54388/task/54398/exe (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54398 admin_soc ceph NOFD /proc/54388/task/54398/fd (opendir: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54657 signal_ha ceph cwd unknown /proc/54388/task/54657/cwd (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54657 signal_ha ceph rtd unknown /proc/54388/task/54657/root (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54657 signal_ha ceph txt unknown /proc/54388/task/54657/exe (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54657 signal_ha ceph NOFD /proc/54388/task/54657/fd (opendir: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54658 io_contex ceph cwd unknown /proc/54388/task/54658/cwd (readlink: Permission denied) 2024-09-21T02:55:14.057 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54658 io_contex ceph rtd unknown /proc/54388/task/54658/root (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54658 io_contex ceph txt unknown /proc/54388/task/54658/exe (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54658 io_contex ceph NOFD /proc/54388/task/54658/fd (opendir: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54659 io_contex ceph cwd unknown /proc/54388/task/54659/cwd (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54659 io_contex ceph rtd unknown /proc/54388/task/54659/root (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54659 io_contex ceph txt unknown /proc/54388/task/54659/exe (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54659 io_contex ceph NOFD /proc/54388/task/54659/fd (opendir: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54682 OpHistory ceph cwd unknown /proc/54388/task/54682/cwd (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54682 OpHistory ceph rtd unknown /proc/54388/task/54682/root (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54682 OpHistory ceph txt unknown /proc/54388/task/54682/exe (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54682 OpHistory ceph NOFD /proc/54388/task/54682/fd (opendir: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54683 ceph_time ceph cwd unknown /proc/54388/task/54683/cwd (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54683 ceph_time ceph rtd unknown /proc/54388/task/54683/root (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54683 ceph_time ceph txt unknown /proc/54388/task/54683/exe (readlink: Permission denied) 2024-09-21T02:55:14.058 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54683 ceph_time ceph NOFD /proc/54388/task/54683/fd (opendir: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54686 ceph-osd ceph cwd unknown /proc/54388/task/54686/cwd (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54686 ceph-osd ceph rtd unknown /proc/54388/task/54686/root (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54686 ceph-osd ceph txt unknown /proc/54388/task/54686/exe (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54686 ceph-osd ceph NOFD /proc/54388/task/54686/fd (opendir: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54744 ceph-osd ceph cwd unknown /proc/54388/task/54744/cwd (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54744 ceph-osd ceph rtd unknown /proc/54388/task/54744/root (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54744 ceph-osd ceph txt unknown /proc/54388/task/54744/exe (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54744 ceph-osd ceph NOFD /proc/54388/task/54744/fd (opendir: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54789 ceph-osd ceph cwd unknown /proc/54388/task/54789/cwd (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54789 ceph-osd ceph rtd unknown /proc/54388/task/54789/root (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54789 ceph-osd ceph txt unknown /proc/54388/task/54789/exe (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54789 ceph-osd ceph NOFD /proc/54388/task/54789/fd (opendir: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54852 ceph-osd ceph cwd unknown /proc/54388/task/54852/cwd (readlink: Permission denied) 2024-09-21T02:55:14.059 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54852 ceph-osd ceph rtd unknown /proc/54388/task/54852/root (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54852 ceph-osd ceph txt unknown /proc/54388/task/54852/exe (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54852 ceph-osd ceph NOFD /proc/54388/task/54852/fd (opendir: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54860 ceph-osd ceph cwd unknown /proc/54388/task/54860/cwd (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54860 ceph-osd ceph rtd unknown /proc/54388/task/54860/root (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54860 ceph-osd ceph txt unknown /proc/54388/task/54860/exe (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54860 ceph-osd ceph NOFD /proc/54388/task/54860/fd (opendir: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54870 ceph-osd ceph cwd unknown /proc/54388/task/54870/cwd (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54870 ceph-osd ceph rtd unknown /proc/54388/task/54870/root (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54870 ceph-osd ceph txt unknown /proc/54388/task/54870/exe (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54870 ceph-osd ceph NOFD /proc/54388/task/54870/fd (opendir: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54890 ceph-osd ceph cwd unknown /proc/54388/task/54890/cwd (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54890 ceph-osd ceph rtd unknown /proc/54388/task/54890/root (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54890 ceph-osd ceph txt unknown /proc/54388/task/54890/exe (readlink: Permission denied) 2024-09-21T02:55:14.060 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54890 ceph-osd ceph NOFD /proc/54388/task/54890/fd (opendir: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54893 ceph-osd ceph cwd unknown /proc/54388/task/54893/cwd (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54893 ceph-osd ceph rtd unknown /proc/54388/task/54893/root (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54893 ceph-osd ceph txt unknown /proc/54388/task/54893/exe (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54893 ceph-osd ceph NOFD /proc/54388/task/54893/fd (opendir: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54894 ceph-osd ceph cwd unknown /proc/54388/task/54894/cwd (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54894 ceph-osd ceph rtd unknown /proc/54388/task/54894/root (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54894 ceph-osd ceph txt unknown /proc/54388/task/54894/exe (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54894 ceph-osd ceph NOFD /proc/54388/task/54894/fd (opendir: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54895 safe_time ceph cwd unknown /proc/54388/task/54895/cwd (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54895 safe_time ceph rtd unknown /proc/54388/task/54895/root (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54895 safe_time ceph txt unknown /proc/54388/task/54895/exe (readlink: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54895 safe_time ceph NOFD /proc/54388/task/54895/fd (opendir: Permission denied) 2024-09-21T02:55:14.061 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54896 safe_time ceph cwd unknown /proc/54388/task/54896/cwd (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54896 safe_time ceph rtd unknown /proc/54388/task/54896/root (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54896 safe_time ceph txt unknown /proc/54388/task/54896/exe (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54896 safe_time ceph NOFD /proc/54388/task/54896/fd (opendir: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54897 safe_time ceph cwd unknown /proc/54388/task/54897/cwd (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54897 safe_time ceph rtd unknown /proc/54388/task/54897/root (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54897 safe_time ceph txt unknown /proc/54388/task/54897/exe (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54897 safe_time ceph NOFD /proc/54388/task/54897/fd (opendir: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54898 safe_time ceph cwd unknown /proc/54388/task/54898/cwd (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54898 safe_time ceph rtd unknown /proc/54388/task/54898/root (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54898 safe_time ceph txt unknown /proc/54388/task/54898/exe (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54898 safe_time ceph NOFD /proc/54388/task/54898/fd (opendir: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54899 fn_anonym ceph cwd unknown /proc/54388/task/54899/cwd (readlink: Permission denied) 2024-09-21T02:55:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54899 fn_anonym ceph rtd unknown /proc/54388/task/54899/root (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54899 fn_anonym ceph txt unknown /proc/54388/task/54899/exe (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54899 fn_anonym ceph NOFD /proc/54388/task/54899/fd (opendir: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54983 rocksdb:l ceph cwd unknown /proc/54388/task/54983/cwd (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54983 rocksdb:l ceph rtd unknown /proc/54388/task/54983/root (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54983 rocksdb:l ceph txt unknown /proc/54388/task/54983/exe (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54983 rocksdb:l ceph NOFD /proc/54388/task/54983/fd (opendir: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54984 rocksdb:l ceph cwd unknown /proc/54388/task/54984/cwd (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54984 rocksdb:l ceph rtd unknown /proc/54388/task/54984/root (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54984 rocksdb:l ceph txt unknown /proc/54388/task/54984/exe (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54984 rocksdb:l ceph NOFD /proc/54388/task/54984/fd (opendir: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54985 rocksdb:l ceph cwd unknown /proc/54388/task/54985/cwd (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54985 rocksdb:l ceph rtd unknown /proc/54388/task/54985/root (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54985 rocksdb:l ceph txt unknown /proc/54388/task/54985/exe (readlink: Permission denied) 2024-09-21T02:55:14.063 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54985 rocksdb:l ceph NOFD /proc/54388/task/54985/fd (opendir: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54986 rocksdb:h ceph cwd unknown /proc/54388/task/54986/cwd (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54986 rocksdb:h ceph rtd unknown /proc/54388/task/54986/root (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54986 rocksdb:h ceph txt unknown /proc/54388/task/54986/exe (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 54986 rocksdb:h ceph NOFD /proc/54388/task/54986/fd (opendir: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55422 bstore_ai ceph cwd unknown /proc/54388/task/55422/cwd (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55422 bstore_ai ceph rtd unknown /proc/54388/task/55422/root (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55422 bstore_ai ceph txt unknown /proc/54388/task/55422/exe (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55422 bstore_ai ceph NOFD /proc/54388/task/55422/fd (opendir: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55606 bstore_ai ceph cwd unknown /proc/54388/task/55606/cwd (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55606 bstore_ai ceph rtd unknown /proc/54388/task/55606/root (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55606 bstore_ai ceph txt unknown /proc/54388/task/55606/exe (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55606 bstore_ai ceph NOFD /proc/54388/task/55606/fd (opendir: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55789 ceph-osd ceph cwd unknown /proc/54388/task/55789/cwd (readlink: Permission denied) 2024-09-21T02:55:14.064 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55789 ceph-osd ceph rtd unknown /proc/54388/task/55789/root (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55789 ceph-osd ceph txt unknown /proc/54388/task/55789/exe (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55789 ceph-osd ceph NOFD /proc/54388/task/55789/fd (opendir: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55790 cfin ceph cwd unknown /proc/54388/task/55790/cwd (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55790 cfin ceph rtd unknown /proc/54388/task/55790/root (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55790 cfin ceph txt unknown /proc/54388/task/55790/exe (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55790 cfin ceph NOFD /proc/54388/task/55790/fd (opendir: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55791 bstore_kv ceph cwd unknown /proc/54388/task/55791/cwd (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55791 bstore_kv ceph rtd unknown /proc/54388/task/55791/root (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55791 bstore_kv ceph txt unknown /proc/54388/task/55791/exe (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55791 bstore_kv ceph NOFD /proc/54388/task/55791/fd (opendir: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55792 bstore_kv ceph cwd unknown /proc/54388/task/55792/cwd (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55792 bstore_kv ceph rtd unknown /proc/54388/task/55792/root (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55792 bstore_kv ceph txt unknown /proc/54388/task/55792/exe (readlink: Permission denied) 2024-09-21T02:55:14.065 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55792 bstore_kv ceph NOFD /proc/54388/task/55792/fd (opendir: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55793 bstore_me ceph cwd unknown /proc/54388/task/55793/cwd (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55793 bstore_me ceph rtd unknown /proc/54388/task/55793/root (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55793 bstore_me ceph txt unknown /proc/54388/task/55793/exe (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55793 bstore_me ceph NOFD /proc/54388/task/55793/fd (opendir: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55794 ms_dispat ceph cwd unknown /proc/54388/task/55794/cwd (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55794 ms_dispat ceph rtd unknown /proc/54388/task/55794/root (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55794 ms_dispat ceph txt unknown /proc/54388/task/55794/exe (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55794 ms_dispat ceph NOFD /proc/54388/task/55794/fd (opendir: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55795 ms_local ceph cwd unknown /proc/54388/task/55795/cwd (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55795 ms_local ceph rtd unknown /proc/54388/task/55795/root (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55795 ms_local ceph txt unknown /proc/54388/task/55795/exe (readlink: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55795 ms_local ceph NOFD /proc/54388/task/55795/fd (opendir: Permission denied) 2024-09-21T02:55:14.066 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55796 safe_time ceph cwd unknown /proc/54388/task/55796/cwd (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55796 safe_time ceph rtd unknown /proc/54388/task/55796/root (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55796 safe_time ceph txt unknown /proc/54388/task/55796/exe (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55796 safe_time ceph NOFD /proc/54388/task/55796/fd (opendir: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55797 safe_time ceph cwd unknown /proc/54388/task/55797/cwd (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55797 safe_time ceph rtd unknown /proc/54388/task/55797/root (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55797 safe_time ceph txt unknown /proc/54388/task/55797/exe (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55797 safe_time ceph NOFD /proc/54388/task/55797/fd (opendir: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55798 ms_dispat ceph cwd unknown /proc/54388/task/55798/cwd (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55798 ms_dispat ceph rtd unknown /proc/54388/task/55798/root (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55798 ms_dispat ceph txt unknown /proc/54388/task/55798/exe (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55798 ms_dispat ceph NOFD /proc/54388/task/55798/fd (opendir: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55799 ms_local ceph cwd unknown /proc/54388/task/55799/cwd (readlink: Permission denied) 2024-09-21T02:55:14.067 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55799 ms_local ceph rtd unknown /proc/54388/task/55799/root (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55799 ms_local ceph txt unknown /proc/54388/task/55799/exe (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55799 ms_local ceph NOFD /proc/54388/task/55799/fd (opendir: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55800 ms_dispat ceph cwd unknown /proc/54388/task/55800/cwd (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55800 ms_dispat ceph rtd unknown /proc/54388/task/55800/root (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55800 ms_dispat ceph txt unknown /proc/54388/task/55800/exe (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55800 ms_dispat ceph NOFD /proc/54388/task/55800/fd (opendir: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55801 ms_local ceph cwd unknown /proc/54388/task/55801/cwd (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55801 ms_local ceph rtd unknown /proc/54388/task/55801/root (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55801 ms_local ceph txt unknown /proc/54388/task/55801/exe (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55801 ms_local ceph NOFD /proc/54388/task/55801/fd (opendir: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55802 ms_dispat ceph cwd unknown /proc/54388/task/55802/cwd (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55802 ms_dispat ceph rtd unknown /proc/54388/task/55802/root (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55802 ms_dispat ceph txt unknown /proc/54388/task/55802/exe (readlink: Permission denied) 2024-09-21T02:55:14.068 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55802 ms_dispat ceph NOFD /proc/54388/task/55802/fd (opendir: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55803 ms_local ceph cwd unknown /proc/54388/task/55803/cwd (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55803 ms_local ceph rtd unknown /proc/54388/task/55803/root (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55803 ms_local ceph txt unknown /proc/54388/task/55803/exe (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55803 ms_local ceph NOFD /proc/54388/task/55803/fd (opendir: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55804 ms_dispat ceph cwd unknown /proc/54388/task/55804/cwd (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55804 ms_dispat ceph rtd unknown /proc/54388/task/55804/root (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55804 ms_dispat ceph txt unknown /proc/54388/task/55804/exe (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55804 ms_dispat ceph NOFD /proc/54388/task/55804/fd (opendir: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55805 ms_local ceph cwd unknown /proc/54388/task/55805/cwd (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55805 ms_local ceph rtd unknown /proc/54388/task/55805/root (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55805 ms_local ceph txt unknown /proc/54388/task/55805/exe (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55805 ms_local ceph NOFD /proc/54388/task/55805/fd (opendir: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55806 ms_dispat ceph cwd unknown /proc/54388/task/55806/cwd (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55806 ms_dispat ceph rtd unknown /proc/54388/task/55806/root (readlink: Permission denied) 2024-09-21T02:55:14.069 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55806 ms_dispat ceph txt unknown /proc/54388/task/55806/exe (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55806 ms_dispat ceph NOFD /proc/54388/task/55806/fd (opendir: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55807 ms_local ceph cwd unknown /proc/54388/task/55807/cwd (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55807 ms_local ceph rtd unknown /proc/54388/task/55807/root (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55807 ms_local ceph txt unknown /proc/54388/task/55807/exe (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55807 ms_local ceph NOFD /proc/54388/task/55807/fd (opendir: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55808 ms_dispat ceph cwd unknown /proc/54388/task/55808/cwd (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55808 ms_dispat ceph rtd unknown /proc/54388/task/55808/root (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55808 ms_dispat ceph txt unknown /proc/54388/task/55808/exe (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55808 ms_dispat ceph NOFD /proc/54388/task/55808/fd (opendir: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55809 ms_local ceph cwd unknown /proc/54388/task/55809/cwd (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55809 ms_local ceph rtd unknown /proc/54388/task/55809/root (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55809 ms_local ceph txt unknown /proc/54388/task/55809/exe (readlink: Permission denied) 2024-09-21T02:55:14.070 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55809 ms_local ceph NOFD /proc/54388/task/55809/fd (opendir: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55810 fn_anonym ceph cwd unknown /proc/54388/task/55810/cwd (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55810 fn_anonym ceph rtd unknown /proc/54388/task/55810/root (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55810 fn_anonym ceph txt unknown /proc/54388/task/55810/exe (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55810 fn_anonym ceph NOFD /proc/54388/task/55810/fd (opendir: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55811 finisher ceph cwd unknown /proc/54388/task/55811/cwd (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55811 finisher ceph rtd unknown /proc/54388/task/55811/root (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55811 finisher ceph txt unknown /proc/54388/task/55811/exe (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55811 finisher ceph NOFD /proc/54388/task/55811/fd (opendir: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55812 safe_time ceph cwd unknown /proc/54388/task/55812/cwd (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55812 safe_time ceph rtd unknown /proc/54388/task/55812/root (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55812 safe_time ceph txt unknown /proc/54388/task/55812/exe (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55812 safe_time ceph NOFD /proc/54388/task/55812/fd (opendir: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55813 safe_time ceph cwd unknown /proc/54388/task/55813/cwd (readlink: Permission denied) 2024-09-21T02:55:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55813 safe_time ceph rtd unknown /proc/54388/task/55813/root (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55813 safe_time ceph txt unknown /proc/54388/task/55813/exe (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55813 safe_time ceph NOFD /proc/54388/task/55813/fd (opendir: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55814 ceph-osd ceph cwd unknown /proc/54388/task/55814/cwd (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55814 ceph-osd ceph rtd unknown /proc/54388/task/55814/root (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55814 ceph-osd ceph txt unknown /proc/54388/task/55814/exe (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55814 ceph-osd ceph NOFD /proc/54388/task/55814/fd (opendir: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55815 osd_srv_a ceph cwd unknown /proc/54388/task/55815/cwd (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55815 osd_srv_a ceph rtd unknown /proc/54388/task/55815/root (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55815 osd_srv_a ceph txt unknown /proc/54388/task/55815/exe (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55815 osd_srv_a ceph NOFD /proc/54388/task/55815/fd (opendir: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55816 tp_osd_tp ceph cwd unknown /proc/54388/task/55816/cwd (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55816 tp_osd_tp ceph rtd unknown /proc/54388/task/55816/root (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55816 tp_osd_tp ceph txt unknown /proc/54388/task/55816/exe (readlink: Permission denied) 2024-09-21T02:55:14.072 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55816 tp_osd_tp ceph NOFD /proc/54388/task/55816/fd (opendir: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55817 tp_osd_tp ceph cwd unknown /proc/54388/task/55817/cwd (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55817 tp_osd_tp ceph rtd unknown /proc/54388/task/55817/root (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55817 tp_osd_tp ceph txt unknown /proc/54388/task/55817/exe (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55817 tp_osd_tp ceph NOFD /proc/54388/task/55817/fd (opendir: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55818 tp_osd_tp ceph cwd unknown /proc/54388/task/55818/cwd (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55818 tp_osd_tp ceph rtd unknown /proc/54388/task/55818/root (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55818 tp_osd_tp ceph txt unknown /proc/54388/task/55818/exe (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55818 tp_osd_tp ceph NOFD /proc/54388/task/55818/fd (opendir: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55819 tp_osd_tp ceph cwd unknown /proc/54388/task/55819/cwd (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55819 tp_osd_tp ceph rtd unknown /proc/54388/task/55819/root (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55819 tp_osd_tp ceph txt unknown /proc/54388/task/55819/exe (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55819 tp_osd_tp ceph NOFD /proc/54388/task/55819/fd (opendir: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55820 tp_osd_tp ceph cwd unknown /proc/54388/task/55820/cwd (readlink: Permission denied) 2024-09-21T02:55:14.073 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55820 tp_osd_tp ceph rtd unknown /proc/54388/task/55820/root (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55820 tp_osd_tp ceph txt unknown /proc/54388/task/55820/exe (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55820 tp_osd_tp ceph NOFD /proc/54388/task/55820/fd (opendir: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55821 tp_osd_tp ceph cwd unknown /proc/54388/task/55821/cwd (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55821 tp_osd_tp ceph rtd unknown /proc/54388/task/55821/root (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55821 tp_osd_tp ceph txt unknown /proc/54388/task/55821/exe (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55821 tp_osd_tp ceph NOFD /proc/54388/task/55821/fd (opendir: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55822 tp_osd_tp ceph cwd unknown /proc/54388/task/55822/cwd (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55822 tp_osd_tp ceph rtd unknown /proc/54388/task/55822/root (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55822 tp_osd_tp ceph txt unknown /proc/54388/task/55822/exe (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55822 tp_osd_tp ceph NOFD /proc/54388/task/55822/fd (opendir: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55823 tp_osd_tp ceph cwd unknown /proc/54388/task/55823/cwd (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55823 tp_osd_tp ceph rtd unknown /proc/54388/task/55823/root (readlink: Permission denied) 2024-09-21T02:55:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55823 tp_osd_tp ceph txt unknown /proc/54388/task/55823/exe (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55823 tp_osd_tp ceph NOFD /proc/54388/task/55823/fd (opendir: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55824 tp_osd_tp ceph cwd unknown /proc/54388/task/55824/cwd (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55824 tp_osd_tp ceph rtd unknown /proc/54388/task/55824/root (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55824 tp_osd_tp ceph txt unknown /proc/54388/task/55824/exe (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55824 tp_osd_tp ceph NOFD /proc/54388/task/55824/fd (opendir: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55825 tp_osd_tp ceph cwd unknown /proc/54388/task/55825/cwd (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55825 tp_osd_tp ceph rtd unknown /proc/54388/task/55825/root (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55825 tp_osd_tp ceph txt unknown /proc/54388/task/55825/exe (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55825 tp_osd_tp ceph NOFD /proc/54388/task/55825/fd (opendir: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55826 tp_osd_tp ceph cwd unknown /proc/54388/task/55826/cwd (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55826 tp_osd_tp ceph rtd unknown /proc/54388/task/55826/root (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55826 tp_osd_tp ceph txt unknown /proc/54388/task/55826/exe (readlink: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55826 tp_osd_tp ceph NOFD /proc/54388/task/55826/fd (opendir: Permission denied) 2024-09-21T02:55:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55827 tp_osd_tp ceph cwd unknown /proc/54388/task/55827/cwd (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55827 tp_osd_tp ceph rtd unknown /proc/54388/task/55827/root (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55827 tp_osd_tp ceph txt unknown /proc/54388/task/55827/exe (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55827 tp_osd_tp ceph NOFD /proc/54388/task/55827/fd (opendir: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55828 tp_osd_tp ceph cwd unknown /proc/54388/task/55828/cwd (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55828 tp_osd_tp ceph rtd unknown /proc/54388/task/55828/root (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55828 tp_osd_tp ceph txt unknown /proc/54388/task/55828/exe (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55828 tp_osd_tp ceph NOFD /proc/54388/task/55828/fd (opendir: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55829 tp_osd_tp ceph cwd unknown /proc/54388/task/55829/cwd (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55829 tp_osd_tp ceph rtd unknown /proc/54388/task/55829/root (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55829 tp_osd_tp ceph txt unknown /proc/54388/task/55829/exe (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55829 tp_osd_tp ceph NOFD /proc/54388/task/55829/fd (opendir: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55830 tp_osd_tp ceph cwd unknown /proc/54388/task/55830/cwd (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55830 tp_osd_tp ceph rtd unknown /proc/54388/task/55830/root (readlink: Permission denied) 2024-09-21T02:55:14.076 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55830 tp_osd_tp ceph txt unknown /proc/54388/task/55830/exe (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55830 tp_osd_tp ceph NOFD /proc/54388/task/55830/fd (opendir: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55831 tp_osd_tp ceph cwd unknown /proc/54388/task/55831/cwd (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55831 tp_osd_tp ceph rtd unknown /proc/54388/task/55831/root (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55831 tp_osd_tp ceph txt unknown /proc/54388/task/55831/exe (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55831 tp_osd_tp ceph NOFD /proc/54388/task/55831/fd (opendir: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55832 osd_srv_h ceph cwd unknown /proc/54388/task/55832/cwd (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55832 osd_srv_h ceph rtd unknown /proc/54388/task/55832/root (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55832 osd_srv_h ceph txt unknown /proc/54388/task/55832/exe (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 54388 55832 osd_srv_h ceph NOFD /proc/54388/task/55832/fd (opendir: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 55184 root cwd unknown /proc/55184/cwd (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 55184 root rtd unknown /proc/55184/root (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 55184 root txt unknown /proc/55184/exe (readlink: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 55184 root NOFD /proc/55184/fd (opendir: Permission denied) 2024-09-21T02:55:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 55214 root cwd unknown /proc/55214/cwd (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 55214 root rtd unknown /proc/55214/root (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 55214 root txt unknown /proc/55214/exe (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 55214 root NOFD /proc/55214/fd (opendir: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 60823 root cwd unknown /proc/60823/cwd (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 60823 root rtd unknown /proc/60823/root (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 60823 root txt unknown /proc/60823/exe (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 60823 root NOFD /proc/60823/fd (opendir: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 60825 root cwd unknown /proc/60825/cwd (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 60825 root rtd unknown /proc/60825/root (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 60825 root txt unknown /proc/60825/exe (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 60825 root NOFD /proc/60825/fd (opendir: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 ceph cwd unknown /proc/60827/cwd (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 ceph rtd unknown /proc/60827/root (readlink: Permission denied) 2024-09-21T02:55:14.078 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 ceph txt unknown /proc/60827/exe (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 ceph NOFD /proc/60827/fd (opendir: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60828 log ceph cwd unknown /proc/60827/task/60828/cwd (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60828 log ceph rtd unknown /proc/60827/task/60828/root (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60828 log ceph txt unknown /proc/60827/task/60828/exe (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60828 log ceph NOFD /proc/60827/task/60828/fd (opendir: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60830 msgr-work ceph cwd unknown /proc/60827/task/60830/cwd (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60830 msgr-work ceph rtd unknown /proc/60827/task/60830/root (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60830 msgr-work ceph txt unknown /proc/60827/task/60830/exe (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60830 msgr-work ceph NOFD /proc/60827/task/60830/fd (opendir: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60831 msgr-work ceph cwd unknown /proc/60827/task/60831/cwd (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60831 msgr-work ceph rtd unknown /proc/60827/task/60831/root (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60831 msgr-work ceph txt unknown /proc/60827/task/60831/exe (readlink: Permission denied) 2024-09-21T02:55:14.079 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60831 msgr-work ceph NOFD /proc/60827/task/60831/fd (opendir: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60832 msgr-work ceph cwd unknown /proc/60827/task/60832/cwd (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60832 msgr-work ceph rtd unknown /proc/60827/task/60832/root (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60832 msgr-work ceph txt unknown /proc/60827/task/60832/exe (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60832 msgr-work ceph NOFD /proc/60827/task/60832/fd (opendir: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60836 service ceph cwd unknown /proc/60827/task/60836/cwd (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60836 service ceph rtd unknown /proc/60827/task/60836/root (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60836 service ceph txt unknown /proc/60827/task/60836/exe (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60836 service ceph NOFD /proc/60827/task/60836/fd (opendir: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60837 admin_soc ceph cwd unknown /proc/60827/task/60837/cwd (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60837 admin_soc ceph rtd unknown /proc/60827/task/60837/root (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60837 admin_soc ceph txt unknown /proc/60827/task/60837/exe (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 60837 admin_soc ceph NOFD /proc/60827/task/60837/fd (opendir: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61119 signal_ha ceph cwd unknown /proc/60827/task/61119/cwd (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61119 signal_ha ceph rtd unknown /proc/60827/task/61119/root (readlink: Permission denied) 2024-09-21T02:55:14.080 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61119 signal_ha ceph txt unknown /proc/60827/task/61119/exe (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61119 signal_ha ceph NOFD /proc/60827/task/61119/fd (opendir: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61120 io_contex ceph cwd unknown /proc/60827/task/61120/cwd (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61120 io_contex ceph rtd unknown /proc/60827/task/61120/root (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61120 io_contex ceph txt unknown /proc/60827/task/61120/exe (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61120 io_contex ceph NOFD /proc/60827/task/61120/fd (opendir: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61121 io_contex ceph cwd unknown /proc/60827/task/61121/cwd (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61121 io_contex ceph rtd unknown /proc/60827/task/61121/root (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61121 io_contex ceph txt unknown /proc/60827/task/61121/exe (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61121 io_contex ceph NOFD /proc/60827/task/61121/fd (opendir: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61127 OpHistory ceph cwd unknown /proc/60827/task/61127/cwd (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61127 OpHistory ceph rtd unknown /proc/60827/task/61127/root (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61127 OpHistory ceph txt unknown /proc/60827/task/61127/exe (readlink: Permission denied) 2024-09-21T02:55:14.081 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61127 OpHistory ceph NOFD /proc/60827/task/61127/fd (opendir: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61128 ceph_time ceph cwd unknown /proc/60827/task/61128/cwd (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61128 ceph_time ceph rtd unknown /proc/60827/task/61128/root (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61128 ceph_time ceph txt unknown /proc/60827/task/61128/exe (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61128 ceph_time ceph NOFD /proc/60827/task/61128/fd (opendir: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61168 ceph-osd ceph cwd unknown /proc/60827/task/61168/cwd (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61168 ceph-osd ceph rtd unknown /proc/60827/task/61168/root (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61168 ceph-osd ceph txt unknown /proc/60827/task/61168/exe (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61168 ceph-osd ceph NOFD /proc/60827/task/61168/fd (opendir: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61198 ceph-osd ceph cwd unknown /proc/60827/task/61198/cwd (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61198 ceph-osd ceph rtd unknown /proc/60827/task/61198/root (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61198 ceph-osd ceph txt unknown /proc/60827/task/61198/exe (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61198 ceph-osd ceph NOFD /proc/60827/task/61198/fd (opendir: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61268 ceph-osd ceph cwd unknown /proc/60827/task/61268/cwd (readlink: Permission denied) 2024-09-21T02:55:14.082 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61268 ceph-osd ceph rtd unknown /proc/60827/task/61268/root (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61268 ceph-osd ceph txt unknown /proc/60827/task/61268/exe (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61268 ceph-osd ceph NOFD /proc/60827/task/61268/fd (opendir: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61329 ceph-osd ceph cwd unknown /proc/60827/task/61329/cwd (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61329 ceph-osd ceph rtd unknown /proc/60827/task/61329/root (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61329 ceph-osd ceph txt unknown /proc/60827/task/61329/exe (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61329 ceph-osd ceph NOFD /proc/60827/task/61329/fd (opendir: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61345 ceph-osd ceph cwd unknown /proc/60827/task/61345/cwd (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61345 ceph-osd ceph rtd unknown /proc/60827/task/61345/root (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61345 ceph-osd ceph txt unknown /proc/60827/task/61345/exe (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61345 ceph-osd ceph NOFD /proc/60827/task/61345/fd (opendir: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61351 ceph-osd ceph cwd unknown /proc/60827/task/61351/cwd (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61351 ceph-osd ceph rtd unknown /proc/60827/task/61351/root (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61351 ceph-osd ceph txt unknown /proc/60827/task/61351/exe (readlink: Permission denied) 2024-09-21T02:55:14.083 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61351 ceph-osd ceph NOFD /proc/60827/task/61351/fd (opendir: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61359 ceph-osd ceph cwd unknown /proc/60827/task/61359/cwd (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61359 ceph-osd ceph rtd unknown /proc/60827/task/61359/root (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61359 ceph-osd ceph txt unknown /proc/60827/task/61359/exe (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61359 ceph-osd ceph NOFD /proc/60827/task/61359/fd (opendir: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61426 ceph-osd ceph cwd unknown /proc/60827/task/61426/cwd (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61426 ceph-osd ceph rtd unknown /proc/60827/task/61426/root (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61426 ceph-osd ceph txt unknown /proc/60827/task/61426/exe (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61426 ceph-osd ceph NOFD /proc/60827/task/61426/fd (opendir: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61429 ceph-osd ceph cwd unknown /proc/60827/task/61429/cwd (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61429 ceph-osd ceph rtd unknown /proc/60827/task/61429/root (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61429 ceph-osd ceph txt unknown /proc/60827/task/61429/exe (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61429 ceph-osd ceph NOFD /proc/60827/task/61429/fd (opendir: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61430 safe_time ceph cwd unknown /proc/60827/task/61430/cwd (readlink: Permission denied) 2024-09-21T02:55:14.084 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61430 safe_time ceph rtd unknown /proc/60827/task/61430/root (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61430 safe_time ceph txt unknown /proc/60827/task/61430/exe (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61430 safe_time ceph NOFD /proc/60827/task/61430/fd (opendir: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61431 safe_time ceph cwd unknown /proc/60827/task/61431/cwd (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61431 safe_time ceph rtd unknown /proc/60827/task/61431/root (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61431 safe_time ceph txt unknown /proc/60827/task/61431/exe (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61431 safe_time ceph NOFD /proc/60827/task/61431/fd (opendir: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61432 safe_time ceph cwd unknown /proc/60827/task/61432/cwd (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61432 safe_time ceph rtd unknown /proc/60827/task/61432/root (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61432 safe_time ceph txt unknown /proc/60827/task/61432/exe (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61432 safe_time ceph NOFD /proc/60827/task/61432/fd (opendir: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61433 safe_time ceph cwd unknown /proc/60827/task/61433/cwd (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61433 safe_time ceph rtd unknown /proc/60827/task/61433/root (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61433 safe_time ceph txt unknown /proc/60827/task/61433/exe (readlink: Permission denied) 2024-09-21T02:55:14.085 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61433 safe_time ceph NOFD /proc/60827/task/61433/fd (opendir: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61434 fn_anonym ceph cwd unknown /proc/60827/task/61434/cwd (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61434 fn_anonym ceph rtd unknown /proc/60827/task/61434/root (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61434 fn_anonym ceph txt unknown /proc/60827/task/61434/exe (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61434 fn_anonym ceph NOFD /proc/60827/task/61434/fd (opendir: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61480 rocksdb:l ceph cwd unknown /proc/60827/task/61480/cwd (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61480 rocksdb:l ceph rtd unknown /proc/60827/task/61480/root (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61480 rocksdb:l ceph txt unknown /proc/60827/task/61480/exe (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61480 rocksdb:l ceph NOFD /proc/60827/task/61480/fd (opendir: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61481 rocksdb:l ceph cwd unknown /proc/60827/task/61481/cwd (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61481 rocksdb:l ceph rtd unknown /proc/60827/task/61481/root (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61481 rocksdb:l ceph txt unknown /proc/60827/task/61481/exe (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61481 rocksdb:l ceph NOFD /proc/60827/task/61481/fd (opendir: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61482 rocksdb:l ceph cwd unknown /proc/60827/task/61482/cwd (readlink: Permission denied) 2024-09-21T02:55:14.086 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61482 rocksdb:l ceph rtd unknown /proc/60827/task/61482/root (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61482 rocksdb:l ceph txt unknown /proc/60827/task/61482/exe (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61482 rocksdb:l ceph NOFD /proc/60827/task/61482/fd (opendir: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61483 rocksdb:h ceph cwd unknown /proc/60827/task/61483/cwd (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61483 rocksdb:h ceph rtd unknown /proc/60827/task/61483/root (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61483 rocksdb:h ceph txt unknown /proc/60827/task/61483/exe (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61483 rocksdb:h ceph NOFD /proc/60827/task/61483/fd (opendir: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61895 bstore_ai ceph cwd unknown /proc/60827/task/61895/cwd (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61895 bstore_ai ceph rtd unknown /proc/60827/task/61895/root (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61895 bstore_ai ceph txt unknown /proc/60827/task/61895/exe (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 61895 bstore_ai ceph NOFD /proc/60827/task/61895/fd (opendir: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62099 bstore_ai ceph cwd unknown /proc/60827/task/62099/cwd (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62099 bstore_ai ceph rtd unknown /proc/60827/task/62099/root (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62099 bstore_ai ceph txt unknown /proc/60827/task/62099/exe (readlink: Permission denied) 2024-09-21T02:55:14.087 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62099 bstore_ai ceph NOFD /proc/60827/task/62099/fd (opendir: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62281 ceph-osd ceph cwd unknown /proc/60827/task/62281/cwd (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62281 ceph-osd ceph rtd unknown /proc/60827/task/62281/root (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62281 ceph-osd ceph txt unknown /proc/60827/task/62281/exe (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62281 ceph-osd ceph NOFD /proc/60827/task/62281/fd (opendir: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62282 cfin ceph cwd unknown /proc/60827/task/62282/cwd (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62282 cfin ceph rtd unknown /proc/60827/task/62282/root (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62282 cfin ceph txt unknown /proc/60827/task/62282/exe (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62282 cfin ceph NOFD /proc/60827/task/62282/fd (opendir: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62283 bstore_kv ceph cwd unknown /proc/60827/task/62283/cwd (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62283 bstore_kv ceph rtd unknown /proc/60827/task/62283/root (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62283 bstore_kv ceph txt unknown /proc/60827/task/62283/exe (readlink: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62283 bstore_kv ceph NOFD /proc/60827/task/62283/fd (opendir: Permission denied) 2024-09-21T02:55:14.088 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62284 bstore_kv ceph cwd unknown /proc/60827/task/62284/cwd (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62284 bstore_kv ceph rtd unknown /proc/60827/task/62284/root (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62284 bstore_kv ceph txt unknown /proc/60827/task/62284/exe (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62284 bstore_kv ceph NOFD /proc/60827/task/62284/fd (opendir: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62285 bstore_me ceph cwd unknown /proc/60827/task/62285/cwd (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62285 bstore_me ceph rtd unknown /proc/60827/task/62285/root (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62285 bstore_me ceph txt unknown /proc/60827/task/62285/exe (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62285 bstore_me ceph NOFD /proc/60827/task/62285/fd (opendir: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62286 ms_dispat ceph cwd unknown /proc/60827/task/62286/cwd (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62286 ms_dispat ceph rtd unknown /proc/60827/task/62286/root (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62286 ms_dispat ceph txt unknown /proc/60827/task/62286/exe (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62286 ms_dispat ceph NOFD /proc/60827/task/62286/fd (opendir: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62287 ms_local ceph cwd unknown /proc/60827/task/62287/cwd (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62287 ms_local ceph rtd unknown /proc/60827/task/62287/root (readlink: Permission denied) 2024-09-21T02:55:14.089 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62287 ms_local ceph txt unknown /proc/60827/task/62287/exe (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62287 ms_local ceph NOFD /proc/60827/task/62287/fd (opendir: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62288 safe_time ceph cwd unknown /proc/60827/task/62288/cwd (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62288 safe_time ceph rtd unknown /proc/60827/task/62288/root (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62288 safe_time ceph txt unknown /proc/60827/task/62288/exe (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62288 safe_time ceph NOFD /proc/60827/task/62288/fd (opendir: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62289 safe_time ceph cwd unknown /proc/60827/task/62289/cwd (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62289 safe_time ceph rtd unknown /proc/60827/task/62289/root (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62289 safe_time ceph txt unknown /proc/60827/task/62289/exe (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62289 safe_time ceph NOFD /proc/60827/task/62289/fd (opendir: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62290 ms_dispat ceph cwd unknown /proc/60827/task/62290/cwd (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62290 ms_dispat ceph rtd unknown /proc/60827/task/62290/root (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62290 ms_dispat ceph txt unknown /proc/60827/task/62290/exe (readlink: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62290 ms_dispat ceph NOFD /proc/60827/task/62290/fd (opendir: Permission denied) 2024-09-21T02:55:14.090 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62291 ms_local ceph cwd unknown /proc/60827/task/62291/cwd (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62291 ms_local ceph rtd unknown /proc/60827/task/62291/root (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62291 ms_local ceph txt unknown /proc/60827/task/62291/exe (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62291 ms_local ceph NOFD /proc/60827/task/62291/fd (opendir: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62292 ms_dispat ceph cwd unknown /proc/60827/task/62292/cwd (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62292 ms_dispat ceph rtd unknown /proc/60827/task/62292/root (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62292 ms_dispat ceph txt unknown /proc/60827/task/62292/exe (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62292 ms_dispat ceph NOFD /proc/60827/task/62292/fd (opendir: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62293 ms_local ceph cwd unknown /proc/60827/task/62293/cwd (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62293 ms_local ceph rtd unknown /proc/60827/task/62293/root (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62293 ms_local ceph txt unknown /proc/60827/task/62293/exe (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62293 ms_local ceph NOFD /proc/60827/task/62293/fd (opendir: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62294 ms_dispat ceph cwd unknown /proc/60827/task/62294/cwd (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62294 ms_dispat ceph rtd unknown /proc/60827/task/62294/root (readlink: Permission denied) 2024-09-21T02:55:14.091 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62294 ms_dispat ceph txt unknown /proc/60827/task/62294/exe (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62294 ms_dispat ceph NOFD /proc/60827/task/62294/fd (opendir: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62295 ms_local ceph cwd unknown /proc/60827/task/62295/cwd (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62295 ms_local ceph rtd unknown /proc/60827/task/62295/root (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62295 ms_local ceph txt unknown /proc/60827/task/62295/exe (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62295 ms_local ceph NOFD /proc/60827/task/62295/fd (opendir: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62296 ms_dispat ceph cwd unknown /proc/60827/task/62296/cwd (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62296 ms_dispat ceph rtd unknown /proc/60827/task/62296/root (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62296 ms_dispat ceph txt unknown /proc/60827/task/62296/exe (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62296 ms_dispat ceph NOFD /proc/60827/task/62296/fd (opendir: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62297 ms_local ceph cwd unknown /proc/60827/task/62297/cwd (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62297 ms_local ceph rtd unknown /proc/60827/task/62297/root (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62297 ms_local ceph txt unknown /proc/60827/task/62297/exe (readlink: Permission denied) 2024-09-21T02:55:14.092 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62297 ms_local ceph NOFD /proc/60827/task/62297/fd (opendir: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62298 ms_dispat ceph cwd unknown /proc/60827/task/62298/cwd (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62298 ms_dispat ceph rtd unknown /proc/60827/task/62298/root (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62298 ms_dispat ceph txt unknown /proc/60827/task/62298/exe (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62298 ms_dispat ceph NOFD /proc/60827/task/62298/fd (opendir: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62299 ms_local ceph cwd unknown /proc/60827/task/62299/cwd (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62299 ms_local ceph rtd unknown /proc/60827/task/62299/root (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62299 ms_local ceph txt unknown /proc/60827/task/62299/exe (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62299 ms_local ceph NOFD /proc/60827/task/62299/fd (opendir: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62300 ms_dispat ceph cwd unknown /proc/60827/task/62300/cwd (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62300 ms_dispat ceph rtd unknown /proc/60827/task/62300/root (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62300 ms_dispat ceph txt unknown /proc/60827/task/62300/exe (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62300 ms_dispat ceph NOFD /proc/60827/task/62300/fd (opendir: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62301 ms_local ceph cwd unknown /proc/60827/task/62301/cwd (readlink: Permission denied) 2024-09-21T02:55:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62301 ms_local ceph rtd unknown /proc/60827/task/62301/root (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62301 ms_local ceph txt unknown /proc/60827/task/62301/exe (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62301 ms_local ceph NOFD /proc/60827/task/62301/fd (opendir: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62302 fn_anonym ceph cwd unknown /proc/60827/task/62302/cwd (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62302 fn_anonym ceph rtd unknown /proc/60827/task/62302/root (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62302 fn_anonym ceph txt unknown /proc/60827/task/62302/exe (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62302 fn_anonym ceph NOFD /proc/60827/task/62302/fd (opendir: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62303 finisher ceph cwd unknown /proc/60827/task/62303/cwd (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62303 finisher ceph rtd unknown /proc/60827/task/62303/root (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62303 finisher ceph txt unknown /proc/60827/task/62303/exe (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62303 finisher ceph NOFD /proc/60827/task/62303/fd (opendir: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62304 safe_time ceph cwd unknown /proc/60827/task/62304/cwd (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62304 safe_time ceph rtd unknown /proc/60827/task/62304/root (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62304 safe_time ceph txt unknown /proc/60827/task/62304/exe (readlink: Permission denied) 2024-09-21T02:55:14.094 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62304 safe_time ceph NOFD /proc/60827/task/62304/fd (opendir: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62305 safe_time ceph cwd unknown /proc/60827/task/62305/cwd (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62305 safe_time ceph rtd unknown /proc/60827/task/62305/root (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62305 safe_time ceph txt unknown /proc/60827/task/62305/exe (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62305 safe_time ceph NOFD /proc/60827/task/62305/fd (opendir: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62306 ceph-osd ceph cwd unknown /proc/60827/task/62306/cwd (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62306 ceph-osd ceph rtd unknown /proc/60827/task/62306/root (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62306 ceph-osd ceph txt unknown /proc/60827/task/62306/exe (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62306 ceph-osd ceph NOFD /proc/60827/task/62306/fd (opendir: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62307 osd_srv_a ceph cwd unknown /proc/60827/task/62307/cwd (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62307 osd_srv_a ceph rtd unknown /proc/60827/task/62307/root (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62307 osd_srv_a ceph txt unknown /proc/60827/task/62307/exe (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62307 osd_srv_a ceph NOFD /proc/60827/task/62307/fd (opendir: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62308 tp_osd_tp ceph cwd unknown /proc/60827/task/62308/cwd (readlink: Permission denied) 2024-09-21T02:55:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62308 tp_osd_tp ceph rtd unknown /proc/60827/task/62308/root (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62308 tp_osd_tp ceph txt unknown /proc/60827/task/62308/exe (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62308 tp_osd_tp ceph NOFD /proc/60827/task/62308/fd (opendir: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62309 tp_osd_tp ceph cwd unknown /proc/60827/task/62309/cwd (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62309 tp_osd_tp ceph rtd unknown /proc/60827/task/62309/root (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62309 tp_osd_tp ceph txt unknown /proc/60827/task/62309/exe (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62309 tp_osd_tp ceph NOFD /proc/60827/task/62309/fd (opendir: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62310 tp_osd_tp ceph cwd unknown /proc/60827/task/62310/cwd (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62310 tp_osd_tp ceph rtd unknown /proc/60827/task/62310/root (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62310 tp_osd_tp ceph txt unknown /proc/60827/task/62310/exe (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62310 tp_osd_tp ceph NOFD /proc/60827/task/62310/fd (opendir: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62311 tp_osd_tp ceph cwd unknown /proc/60827/task/62311/cwd (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62311 tp_osd_tp ceph rtd unknown /proc/60827/task/62311/root (readlink: Permission denied) 2024-09-21T02:55:14.096 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62311 tp_osd_tp ceph txt unknown /proc/60827/task/62311/exe (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62311 tp_osd_tp ceph NOFD /proc/60827/task/62311/fd (opendir: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62312 tp_osd_tp ceph cwd unknown /proc/60827/task/62312/cwd (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62312 tp_osd_tp ceph rtd unknown /proc/60827/task/62312/root (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62312 tp_osd_tp ceph txt unknown /proc/60827/task/62312/exe (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62312 tp_osd_tp ceph NOFD /proc/60827/task/62312/fd (opendir: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62313 tp_osd_tp ceph cwd unknown /proc/60827/task/62313/cwd (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62313 tp_osd_tp ceph rtd unknown /proc/60827/task/62313/root (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62313 tp_osd_tp ceph txt unknown /proc/60827/task/62313/exe (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62313 tp_osd_tp ceph NOFD /proc/60827/task/62313/fd (opendir: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62314 tp_osd_tp ceph cwd unknown /proc/60827/task/62314/cwd (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62314 tp_osd_tp ceph rtd unknown /proc/60827/task/62314/root (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62314 tp_osd_tp ceph txt unknown /proc/60827/task/62314/exe (readlink: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62314 tp_osd_tp ceph NOFD /proc/60827/task/62314/fd (opendir: Permission denied) 2024-09-21T02:55:14.097 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62315 tp_osd_tp ceph cwd unknown /proc/60827/task/62315/cwd (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62315 tp_osd_tp ceph rtd unknown /proc/60827/task/62315/root (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62315 tp_osd_tp ceph txt unknown /proc/60827/task/62315/exe (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62315 tp_osd_tp ceph NOFD /proc/60827/task/62315/fd (opendir: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62316 tp_osd_tp ceph cwd unknown /proc/60827/task/62316/cwd (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62316 tp_osd_tp ceph rtd unknown /proc/60827/task/62316/root (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62316 tp_osd_tp ceph txt unknown /proc/60827/task/62316/exe (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62316 tp_osd_tp ceph NOFD /proc/60827/task/62316/fd (opendir: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62317 tp_osd_tp ceph cwd unknown /proc/60827/task/62317/cwd (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62317 tp_osd_tp ceph rtd unknown /proc/60827/task/62317/root (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62317 tp_osd_tp ceph txt unknown /proc/60827/task/62317/exe (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62317 tp_osd_tp ceph NOFD /proc/60827/task/62317/fd (opendir: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62318 tp_osd_tp ceph cwd unknown /proc/60827/task/62318/cwd (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62318 tp_osd_tp ceph rtd unknown /proc/60827/task/62318/root (readlink: Permission denied) 2024-09-21T02:55:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62318 tp_osd_tp ceph txt unknown /proc/60827/task/62318/exe (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62318 tp_osd_tp ceph NOFD /proc/60827/task/62318/fd (opendir: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62319 tp_osd_tp ceph cwd unknown /proc/60827/task/62319/cwd (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62319 tp_osd_tp ceph rtd unknown /proc/60827/task/62319/root (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62319 tp_osd_tp ceph txt unknown /proc/60827/task/62319/exe (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62319 tp_osd_tp ceph NOFD /proc/60827/task/62319/fd (opendir: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62320 tp_osd_tp ceph cwd unknown /proc/60827/task/62320/cwd (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62320 tp_osd_tp ceph rtd unknown /proc/60827/task/62320/root (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62320 tp_osd_tp ceph txt unknown /proc/60827/task/62320/exe (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62320 tp_osd_tp ceph NOFD /proc/60827/task/62320/fd (opendir: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62321 tp_osd_tp ceph cwd unknown /proc/60827/task/62321/cwd (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62321 tp_osd_tp ceph rtd unknown /proc/60827/task/62321/root (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62321 tp_osd_tp ceph txt unknown /proc/60827/task/62321/exe (readlink: Permission denied) 2024-09-21T02:55:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62321 tp_osd_tp ceph NOFD /proc/60827/task/62321/fd (opendir: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62322 tp_osd_tp ceph cwd unknown /proc/60827/task/62322/cwd (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62322 tp_osd_tp ceph rtd unknown /proc/60827/task/62322/root (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62322 tp_osd_tp ceph txt unknown /proc/60827/task/62322/exe (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62322 tp_osd_tp ceph NOFD /proc/60827/task/62322/fd (opendir: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62323 tp_osd_tp ceph cwd unknown /proc/60827/task/62323/cwd (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62323 tp_osd_tp ceph rtd unknown /proc/60827/task/62323/root (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62323 tp_osd_tp ceph txt unknown /proc/60827/task/62323/exe (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62323 tp_osd_tp ceph NOFD /proc/60827/task/62323/fd (opendir: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62324 osd_srv_h ceph cwd unknown /proc/60827/task/62324/cwd (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62324 osd_srv_h ceph rtd unknown /proc/60827/task/62324/root (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62324 osd_srv_h ceph txt unknown /proc/60827/task/62324/exe (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 60827 62324 osd_srv_h ceph NOFD /proc/60827/task/62324/fd (opendir: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 61448 root cwd unknown /proc/61448/cwd (readlink: Permission denied) 2024-09-21T02:55:14.100 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 61448 root rtd unknown /proc/61448/root (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 61448 root txt unknown /proc/61448/exe (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 61448 root NOFD /proc/61448/fd (opendir: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 61479 root cwd unknown /proc/61479/cwd (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 61479 root rtd unknown /proc/61479/root (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 61479 root txt unknown /proc/61479/exe (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 61479 root NOFD /proc/61479/fd (opendir: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 64900 root cwd unknown /proc/64900/cwd (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 64900 root rtd unknown /proc/64900/root (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 64900 root txt unknown /proc/64900/exe (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 64900 root NOFD /proc/64900/fd (opendir: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64929 root cwd unknown /proc/64929/cwd (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64929 root rtd unknown /proc/64929/root (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64929 root txt unknown /proc/64929/exe (readlink: Permission denied) 2024-09-21T02:55:14.101 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64929 root NOFD /proc/64929/fd (opendir: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64930 root cwd unknown /proc/64930/cwd (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64930 root rtd unknown /proc/64930/root (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64930 root txt unknown /proc/64930/exe (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 64930 root NOFD /proc/64930/fd (opendir: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 66915 root cwd unknown /proc/66915/cwd (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 66915 root rtd unknown /proc/66915/root (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 66915 root txt unknown /proc/66915/exe (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/4 66915 root NOFD /proc/66915/fd (opendir: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 67094 root cwd unknown /proc/67094/cwd (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 67094 root rtd unknown /proc/67094/root (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 67094 root txt unknown /proc/67094/exe (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 67094 root NOFD /proc/67094/fd (opendir: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 67143 root cwd unknown /proc/67143/cwd (readlink: Permission denied) 2024-09-21T02:55:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 67143 root rtd unknown /proc/67143/root (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 67143 root txt unknown /proc/67143/exe (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 67143 root NOFD /proc/67143/fd (opendir: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 67487 root cwd unknown /proc/67487/cwd (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 67487 root rtd unknown /proc/67487/root (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 67487 root txt unknown /proc/67487/exe (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 67487 root NOFD /proc/67487/fd (opendir: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 67489 root cwd unknown /proc/67489/cwd (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 67489 root rtd unknown /proc/67489/root (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 67489 root txt unknown /proc/67489/exe (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 67489 root NOFD /proc/67489/fd (opendir: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 ceph cwd unknown /proc/67491/cwd (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 ceph rtd unknown /proc/67491/root (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 ceph txt unknown /proc/67491/exe (readlink: Permission denied) 2024-09-21T02:55:14.103 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 ceph NOFD /proc/67491/fd (opendir: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67492 log ceph cwd unknown /proc/67491/task/67492/cwd (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67492 log ceph rtd unknown /proc/67491/task/67492/root (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67492 log ceph txt unknown /proc/67491/task/67492/exe (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67492 log ceph NOFD /proc/67491/task/67492/fd (opendir: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67494 msgr-work ceph cwd unknown /proc/67491/task/67494/cwd (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67494 msgr-work ceph rtd unknown /proc/67491/task/67494/root (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67494 msgr-work ceph txt unknown /proc/67491/task/67494/exe (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67494 msgr-work ceph NOFD /proc/67491/task/67494/fd (opendir: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67495 msgr-work ceph cwd unknown /proc/67491/task/67495/cwd (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67495 msgr-work ceph rtd unknown /proc/67491/task/67495/root (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67495 msgr-work ceph txt unknown /proc/67491/task/67495/exe (readlink: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67495 msgr-work ceph NOFD /proc/67491/task/67495/fd (opendir: Permission denied) 2024-09-21T02:55:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67496 msgr-work ceph cwd unknown /proc/67491/task/67496/cwd (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67496 msgr-work ceph rtd unknown /proc/67491/task/67496/root (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67496 msgr-work ceph txt unknown /proc/67491/task/67496/exe (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67496 msgr-work ceph NOFD /proc/67491/task/67496/fd (opendir: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67500 service ceph cwd unknown /proc/67491/task/67500/cwd (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67500 service ceph rtd unknown /proc/67491/task/67500/root (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67500 service ceph txt unknown /proc/67491/task/67500/exe (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67500 service ceph NOFD /proc/67491/task/67500/fd (opendir: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67501 admin_soc ceph cwd unknown /proc/67491/task/67501/cwd (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67501 admin_soc ceph rtd unknown /proc/67491/task/67501/root (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67501 admin_soc ceph txt unknown /proc/67491/task/67501/exe (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67501 admin_soc ceph NOFD /proc/67491/task/67501/fd (opendir: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67755 signal_ha ceph cwd unknown /proc/67491/task/67755/cwd (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67755 signal_ha ceph rtd unknown /proc/67491/task/67755/root (readlink: Permission denied) 2024-09-21T02:55:14.105 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67755 signal_ha ceph txt unknown /proc/67491/task/67755/exe (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67755 signal_ha ceph NOFD /proc/67491/task/67755/fd (opendir: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67756 io_contex ceph cwd unknown /proc/67491/task/67756/cwd (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67756 io_contex ceph rtd unknown /proc/67491/task/67756/root (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67756 io_contex ceph txt unknown /proc/67491/task/67756/exe (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67756 io_contex ceph NOFD /proc/67491/task/67756/fd (opendir: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67757 io_contex ceph cwd unknown /proc/67491/task/67757/cwd (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67757 io_contex ceph rtd unknown /proc/67491/task/67757/root (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67757 io_contex ceph txt unknown /proc/67491/task/67757/exe (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67757 io_contex ceph NOFD /proc/67491/task/67757/fd (opendir: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67771 OpHistory ceph cwd unknown /proc/67491/task/67771/cwd (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67771 OpHistory ceph rtd unknown /proc/67491/task/67771/root (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67771 OpHistory ceph txt unknown /proc/67491/task/67771/exe (readlink: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67771 OpHistory ceph NOFD /proc/67491/task/67771/fd (opendir: Permission denied) 2024-09-21T02:55:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67772 ceph_time ceph cwd unknown /proc/67491/task/67772/cwd (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67772 ceph_time ceph rtd unknown /proc/67491/task/67772/root (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67772 ceph_time ceph txt unknown /proc/67491/task/67772/exe (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67772 ceph_time ceph NOFD /proc/67491/task/67772/fd (opendir: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67809 ceph-osd ceph cwd unknown /proc/67491/task/67809/cwd (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67809 ceph-osd ceph rtd unknown /proc/67491/task/67809/root (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67809 ceph-osd ceph txt unknown /proc/67491/task/67809/exe (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67809 ceph-osd ceph NOFD /proc/67491/task/67809/fd (opendir: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67814 ceph-osd ceph cwd unknown /proc/67491/task/67814/cwd (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67814 ceph-osd ceph rtd unknown /proc/67491/task/67814/root (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67814 ceph-osd ceph txt unknown /proc/67491/task/67814/exe (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67814 ceph-osd ceph NOFD /proc/67491/task/67814/fd (opendir: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67853 ceph-osd ceph cwd unknown /proc/67491/task/67853/cwd (readlink: Permission denied) 2024-09-21T02:55:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67853 ceph-osd ceph rtd unknown /proc/67491/task/67853/root (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67853 ceph-osd ceph txt unknown /proc/67491/task/67853/exe (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67853 ceph-osd ceph NOFD /proc/67491/task/67853/fd (opendir: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67883 ceph-osd ceph cwd unknown /proc/67491/task/67883/cwd (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67883 ceph-osd ceph rtd unknown /proc/67491/task/67883/root (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67883 ceph-osd ceph txt unknown /proc/67491/task/67883/exe (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67883 ceph-osd ceph NOFD /proc/67491/task/67883/fd (opendir: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67964 ceph-osd ceph cwd unknown /proc/67491/task/67964/cwd (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67964 ceph-osd ceph rtd unknown /proc/67491/task/67964/root (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67964 ceph-osd ceph txt unknown /proc/67491/task/67964/exe (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67964 ceph-osd ceph NOFD /proc/67491/task/67964/fd (opendir: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67994 ceph-osd ceph cwd unknown /proc/67491/task/67994/cwd (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67994 ceph-osd ceph rtd unknown /proc/67491/task/67994/root (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67994 ceph-osd ceph txt unknown /proc/67491/task/67994/exe (readlink: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 67994 ceph-osd ceph NOFD /proc/67491/task/67994/fd (opendir: Permission denied) 2024-09-21T02:55:14.108 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68031 ceph-osd ceph cwd unknown /proc/67491/task/68031/cwd (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68031 ceph-osd ceph rtd unknown /proc/67491/task/68031/root (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68031 ceph-osd ceph txt unknown /proc/67491/task/68031/exe (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68031 ceph-osd ceph NOFD /proc/67491/task/68031/fd (opendir: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68082 ceph-osd ceph cwd unknown /proc/67491/task/68082/cwd (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68082 ceph-osd ceph rtd unknown /proc/67491/task/68082/root (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68082 ceph-osd ceph txt unknown /proc/67491/task/68082/exe (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68082 ceph-osd ceph NOFD /proc/67491/task/68082/fd (opendir: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68084 ceph-osd ceph cwd unknown /proc/67491/task/68084/cwd (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68084 ceph-osd ceph rtd unknown /proc/67491/task/68084/root (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68084 ceph-osd ceph txt unknown /proc/67491/task/68084/exe (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68084 ceph-osd ceph NOFD /proc/67491/task/68084/fd (opendir: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68085 safe_time ceph cwd unknown /proc/67491/task/68085/cwd (readlink: Permission denied) 2024-09-21T02:55:14.109 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68085 safe_time ceph rtd unknown /proc/67491/task/68085/root (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68085 safe_time ceph txt unknown /proc/67491/task/68085/exe (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68085 safe_time ceph NOFD /proc/67491/task/68085/fd (opendir: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68086 safe_time ceph cwd unknown /proc/67491/task/68086/cwd (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68086 safe_time ceph rtd unknown /proc/67491/task/68086/root (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68086 safe_time ceph txt unknown /proc/67491/task/68086/exe (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68086 safe_time ceph NOFD /proc/67491/task/68086/fd (opendir: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68087 safe_time ceph cwd unknown /proc/67491/task/68087/cwd (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68087 safe_time ceph rtd unknown /proc/67491/task/68087/root (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68087 safe_time ceph txt unknown /proc/67491/task/68087/exe (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68087 safe_time ceph NOFD /proc/67491/task/68087/fd (opendir: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68088 safe_time ceph cwd unknown /proc/67491/task/68088/cwd (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68088 safe_time ceph rtd unknown /proc/67491/task/68088/root (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68088 safe_time ceph txt unknown /proc/67491/task/68088/exe (readlink: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68088 safe_time ceph NOFD /proc/67491/task/68088/fd (opendir: Permission denied) 2024-09-21T02:55:14.110 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68089 fn_anonym ceph cwd unknown /proc/67491/task/68089/cwd (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68089 fn_anonym ceph rtd unknown /proc/67491/task/68089/root (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68089 fn_anonym ceph txt unknown /proc/67491/task/68089/exe (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68089 fn_anonym ceph NOFD /proc/67491/task/68089/fd (opendir: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68176 rocksdb:l ceph cwd unknown /proc/67491/task/68176/cwd (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68176 rocksdb:l ceph rtd unknown /proc/67491/task/68176/root (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68176 rocksdb:l ceph txt unknown /proc/67491/task/68176/exe (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68176 rocksdb:l ceph NOFD /proc/67491/task/68176/fd (opendir: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68177 rocksdb:l ceph cwd unknown /proc/67491/task/68177/cwd (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68177 rocksdb:l ceph rtd unknown /proc/67491/task/68177/root (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68177 rocksdb:l ceph txt unknown /proc/67491/task/68177/exe (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68177 rocksdb:l ceph NOFD /proc/67491/task/68177/fd (opendir: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68178 rocksdb:l ceph cwd unknown /proc/67491/task/68178/cwd (readlink: Permission denied) 2024-09-21T02:55:14.111 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68178 rocksdb:l ceph rtd unknown /proc/67491/task/68178/root (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68178 rocksdb:l ceph txt unknown /proc/67491/task/68178/exe (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68178 rocksdb:l ceph NOFD /proc/67491/task/68178/fd (opendir: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68179 rocksdb:h ceph cwd unknown /proc/67491/task/68179/cwd (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68179 rocksdb:h ceph rtd unknown /proc/67491/task/68179/root (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68179 rocksdb:h ceph txt unknown /proc/67491/task/68179/exe (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68179 rocksdb:h ceph NOFD /proc/67491/task/68179/fd (opendir: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68578 bstore_ai ceph cwd unknown /proc/67491/task/68578/cwd (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68578 bstore_ai ceph rtd unknown /proc/67491/task/68578/root (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68578 bstore_ai ceph txt unknown /proc/67491/task/68578/exe (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68578 bstore_ai ceph NOFD /proc/67491/task/68578/fd (opendir: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68781 bstore_ai ceph cwd unknown /proc/67491/task/68781/cwd (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68781 bstore_ai ceph rtd unknown /proc/67491/task/68781/root (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68781 bstore_ai ceph txt unknown /proc/67491/task/68781/exe (readlink: Permission denied) 2024-09-21T02:55:14.112 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68781 bstore_ai ceph NOFD /proc/67491/task/68781/fd (opendir: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68963 ceph-osd ceph cwd unknown /proc/67491/task/68963/cwd (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68963 ceph-osd ceph rtd unknown /proc/67491/task/68963/root (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68963 ceph-osd ceph txt unknown /proc/67491/task/68963/exe (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68963 ceph-osd ceph NOFD /proc/67491/task/68963/fd (opendir: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68964 cfin ceph cwd unknown /proc/67491/task/68964/cwd (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68964 cfin ceph rtd unknown /proc/67491/task/68964/root (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68964 cfin ceph txt unknown /proc/67491/task/68964/exe (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68964 cfin ceph NOFD /proc/67491/task/68964/fd (opendir: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68965 bstore_kv ceph cwd unknown /proc/67491/task/68965/cwd (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68965 bstore_kv ceph rtd unknown /proc/67491/task/68965/root (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68965 bstore_kv ceph txt unknown /proc/67491/task/68965/exe (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68965 bstore_kv ceph NOFD /proc/67491/task/68965/fd (opendir: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68966 bstore_kv ceph cwd unknown /proc/67491/task/68966/cwd (readlink: Permission denied) 2024-09-21T02:55:14.113 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68966 bstore_kv ceph rtd unknown /proc/67491/task/68966/root (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68966 bstore_kv ceph txt unknown /proc/67491/task/68966/exe (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68966 bstore_kv ceph NOFD /proc/67491/task/68966/fd (opendir: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68967 bstore_me ceph cwd unknown /proc/67491/task/68967/cwd (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68967 bstore_me ceph rtd unknown /proc/67491/task/68967/root (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68967 bstore_me ceph txt unknown /proc/67491/task/68967/exe (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68967 bstore_me ceph NOFD /proc/67491/task/68967/fd (opendir: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68968 ms_dispat ceph cwd unknown /proc/67491/task/68968/cwd (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68968 ms_dispat ceph rtd unknown /proc/67491/task/68968/root (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68968 ms_dispat ceph txt unknown /proc/67491/task/68968/exe (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68968 ms_dispat ceph NOFD /proc/67491/task/68968/fd (opendir: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68969 ms_local ceph cwd unknown /proc/67491/task/68969/cwd (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68969 ms_local ceph rtd unknown /proc/67491/task/68969/root (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68969 ms_local ceph txt unknown /proc/67491/task/68969/exe (readlink: Permission denied) 2024-09-21T02:55:14.114 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68969 ms_local ceph NOFD /proc/67491/task/68969/fd (opendir: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68970 safe_time ceph cwd unknown /proc/67491/task/68970/cwd (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68970 safe_time ceph rtd unknown /proc/67491/task/68970/root (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68970 safe_time ceph txt unknown /proc/67491/task/68970/exe (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68970 safe_time ceph NOFD /proc/67491/task/68970/fd (opendir: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68971 safe_time ceph cwd unknown /proc/67491/task/68971/cwd (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68971 safe_time ceph rtd unknown /proc/67491/task/68971/root (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68971 safe_time ceph txt unknown /proc/67491/task/68971/exe (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68971 safe_time ceph NOFD /proc/67491/task/68971/fd (opendir: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68972 ms_dispat ceph cwd unknown /proc/67491/task/68972/cwd (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68972 ms_dispat ceph rtd unknown /proc/67491/task/68972/root (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68972 ms_dispat ceph txt unknown /proc/67491/task/68972/exe (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68972 ms_dispat ceph NOFD /proc/67491/task/68972/fd (opendir: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68973 ms_local ceph cwd unknown /proc/67491/task/68973/cwd (readlink: Permission denied) 2024-09-21T02:55:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68973 ms_local ceph rtd unknown /proc/67491/task/68973/root (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68973 ms_local ceph txt unknown /proc/67491/task/68973/exe (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68973 ms_local ceph NOFD /proc/67491/task/68973/fd (opendir: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68974 ms_dispat ceph cwd unknown /proc/67491/task/68974/cwd (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68974 ms_dispat ceph rtd unknown /proc/67491/task/68974/root (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68974 ms_dispat ceph txt unknown /proc/67491/task/68974/exe (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68974 ms_dispat ceph NOFD /proc/67491/task/68974/fd (opendir: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68975 ms_local ceph cwd unknown /proc/67491/task/68975/cwd (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68975 ms_local ceph rtd unknown /proc/67491/task/68975/root (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68975 ms_local ceph txt unknown /proc/67491/task/68975/exe (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68975 ms_local ceph NOFD /proc/67491/task/68975/fd (opendir: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68976 ms_dispat ceph cwd unknown /proc/67491/task/68976/cwd (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68976 ms_dispat ceph rtd unknown /proc/67491/task/68976/root (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68976 ms_dispat ceph txt unknown /proc/67491/task/68976/exe (readlink: Permission denied) 2024-09-21T02:55:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68976 ms_dispat ceph NOFD /proc/67491/task/68976/fd (opendir: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68977 ms_local ceph cwd unknown /proc/67491/task/68977/cwd (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68977 ms_local ceph rtd unknown /proc/67491/task/68977/root (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68977 ms_local ceph txt unknown /proc/67491/task/68977/exe (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68977 ms_local ceph NOFD /proc/67491/task/68977/fd (opendir: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68978 ms_dispat ceph cwd unknown /proc/67491/task/68978/cwd (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68978 ms_dispat ceph rtd unknown /proc/67491/task/68978/root (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68978 ms_dispat ceph txt unknown /proc/67491/task/68978/exe (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68978 ms_dispat ceph NOFD /proc/67491/task/68978/fd (opendir: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68979 ms_local ceph cwd unknown /proc/67491/task/68979/cwd (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68979 ms_local ceph rtd unknown /proc/67491/task/68979/root (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68979 ms_local ceph txt unknown /proc/67491/task/68979/exe (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68979 ms_local ceph NOFD /proc/67491/task/68979/fd (opendir: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68980 ms_dispat ceph cwd unknown /proc/67491/task/68980/cwd (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68980 ms_dispat ceph rtd unknown /proc/67491/task/68980/root (readlink: Permission denied) 2024-09-21T02:55:14.117 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68980 ms_dispat ceph txt unknown /proc/67491/task/68980/exe (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68980 ms_dispat ceph NOFD /proc/67491/task/68980/fd (opendir: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68981 ms_local ceph cwd unknown /proc/67491/task/68981/cwd (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68981 ms_local ceph rtd unknown /proc/67491/task/68981/root (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68981 ms_local ceph txt unknown /proc/67491/task/68981/exe (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68981 ms_local ceph NOFD /proc/67491/task/68981/fd (opendir: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68982 ms_dispat ceph cwd unknown /proc/67491/task/68982/cwd (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68982 ms_dispat ceph rtd unknown /proc/67491/task/68982/root (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68982 ms_dispat ceph txt unknown /proc/67491/task/68982/exe (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68982 ms_dispat ceph NOFD /proc/67491/task/68982/fd (opendir: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68983 ms_local ceph cwd unknown /proc/67491/task/68983/cwd (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68983 ms_local ceph rtd unknown /proc/67491/task/68983/root (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68983 ms_local ceph txt unknown /proc/67491/task/68983/exe (readlink: Permission denied) 2024-09-21T02:55:14.118 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68983 ms_local ceph NOFD /proc/67491/task/68983/fd (opendir: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68984 fn_anonym ceph cwd unknown /proc/67491/task/68984/cwd (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68984 fn_anonym ceph rtd unknown /proc/67491/task/68984/root (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68984 fn_anonym ceph txt unknown /proc/67491/task/68984/exe (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68984 fn_anonym ceph NOFD /proc/67491/task/68984/fd (opendir: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68985 finisher ceph cwd unknown /proc/67491/task/68985/cwd (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68985 finisher ceph rtd unknown /proc/67491/task/68985/root (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68985 finisher ceph txt unknown /proc/67491/task/68985/exe (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68985 finisher ceph NOFD /proc/67491/task/68985/fd (opendir: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68986 safe_time ceph cwd unknown /proc/67491/task/68986/cwd (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68986 safe_time ceph rtd unknown /proc/67491/task/68986/root (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68986 safe_time ceph txt unknown /proc/67491/task/68986/exe (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68986 safe_time ceph NOFD /proc/67491/task/68986/fd (opendir: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68987 safe_time ceph cwd unknown /proc/67491/task/68987/cwd (readlink: Permission denied) 2024-09-21T02:55:14.119 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68987 safe_time ceph rtd unknown /proc/67491/task/68987/root (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68987 safe_time ceph txt unknown /proc/67491/task/68987/exe (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68987 safe_time ceph NOFD /proc/67491/task/68987/fd (opendir: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68988 ceph-osd ceph cwd unknown /proc/67491/task/68988/cwd (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68988 ceph-osd ceph rtd unknown /proc/67491/task/68988/root (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68988 ceph-osd ceph txt unknown /proc/67491/task/68988/exe (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68988 ceph-osd ceph NOFD /proc/67491/task/68988/fd (opendir: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68989 osd_srv_a ceph cwd unknown /proc/67491/task/68989/cwd (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68989 osd_srv_a ceph rtd unknown /proc/67491/task/68989/root (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68989 osd_srv_a ceph txt unknown /proc/67491/task/68989/exe (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68989 osd_srv_a ceph NOFD /proc/67491/task/68989/fd (opendir: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68990 tp_osd_tp ceph cwd unknown /proc/67491/task/68990/cwd (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68990 tp_osd_tp ceph rtd unknown /proc/67491/task/68990/root (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68990 tp_osd_tp ceph txt unknown /proc/67491/task/68990/exe (readlink: Permission denied) 2024-09-21T02:55:14.120 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68990 tp_osd_tp ceph NOFD /proc/67491/task/68990/fd (opendir: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68991 tp_osd_tp ceph cwd unknown /proc/67491/task/68991/cwd (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68991 tp_osd_tp ceph rtd unknown /proc/67491/task/68991/root (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68991 tp_osd_tp ceph txt unknown /proc/67491/task/68991/exe (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68991 tp_osd_tp ceph NOFD /proc/67491/task/68991/fd (opendir: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68992 tp_osd_tp ceph cwd unknown /proc/67491/task/68992/cwd (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68992 tp_osd_tp ceph rtd unknown /proc/67491/task/68992/root (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68992 tp_osd_tp ceph txt unknown /proc/67491/task/68992/exe (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68992 tp_osd_tp ceph NOFD /proc/67491/task/68992/fd (opendir: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68993 tp_osd_tp ceph cwd unknown /proc/67491/task/68993/cwd (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68993 tp_osd_tp ceph rtd unknown /proc/67491/task/68993/root (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68993 tp_osd_tp ceph txt unknown /proc/67491/task/68993/exe (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68993 tp_osd_tp ceph NOFD /proc/67491/task/68993/fd (opendir: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68994 tp_osd_tp ceph cwd unknown /proc/67491/task/68994/cwd (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68994 tp_osd_tp ceph rtd unknown /proc/67491/task/68994/root (readlink: Permission denied) 2024-09-21T02:55:14.121 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68994 tp_osd_tp ceph txt unknown /proc/67491/task/68994/exe (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68994 tp_osd_tp ceph NOFD /proc/67491/task/68994/fd (opendir: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68995 tp_osd_tp ceph cwd unknown /proc/67491/task/68995/cwd (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68995 tp_osd_tp ceph rtd unknown /proc/67491/task/68995/root (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68995 tp_osd_tp ceph txt unknown /proc/67491/task/68995/exe (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68995 tp_osd_tp ceph NOFD /proc/67491/task/68995/fd (opendir: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68996 tp_osd_tp ceph cwd unknown /proc/67491/task/68996/cwd (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68996 tp_osd_tp ceph rtd unknown /proc/67491/task/68996/root (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68996 tp_osd_tp ceph txt unknown /proc/67491/task/68996/exe (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68996 tp_osd_tp ceph NOFD /proc/67491/task/68996/fd (opendir: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68997 tp_osd_tp ceph cwd unknown /proc/67491/task/68997/cwd (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68997 tp_osd_tp ceph rtd unknown /proc/67491/task/68997/root (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68997 tp_osd_tp ceph txt unknown /proc/67491/task/68997/exe (readlink: Permission denied) 2024-09-21T02:55:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68997 tp_osd_tp ceph NOFD /proc/67491/task/68997/fd (opendir: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68998 tp_osd_tp ceph cwd unknown /proc/67491/task/68998/cwd (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68998 tp_osd_tp ceph rtd unknown /proc/67491/task/68998/root (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68998 tp_osd_tp ceph txt unknown /proc/67491/task/68998/exe (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68998 tp_osd_tp ceph NOFD /proc/67491/task/68998/fd (opendir: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68999 tp_osd_tp ceph cwd unknown /proc/67491/task/68999/cwd (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68999 tp_osd_tp ceph rtd unknown /proc/67491/task/68999/root (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68999 tp_osd_tp ceph txt unknown /proc/67491/task/68999/exe (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 68999 tp_osd_tp ceph NOFD /proc/67491/task/68999/fd (opendir: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69000 tp_osd_tp ceph cwd unknown /proc/67491/task/69000/cwd (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69000 tp_osd_tp ceph rtd unknown /proc/67491/task/69000/root (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69000 tp_osd_tp ceph txt unknown /proc/67491/task/69000/exe (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69000 tp_osd_tp ceph NOFD /proc/67491/task/69000/fd (opendir: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69001 tp_osd_tp ceph cwd unknown /proc/67491/task/69001/cwd (readlink: Permission denied) 2024-09-21T02:55:14.123 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69001 tp_osd_tp ceph rtd unknown /proc/67491/task/69001/root (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69001 tp_osd_tp ceph txt unknown /proc/67491/task/69001/exe (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69001 tp_osd_tp ceph NOFD /proc/67491/task/69001/fd (opendir: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69002 tp_osd_tp ceph cwd unknown /proc/67491/task/69002/cwd (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69002 tp_osd_tp ceph rtd unknown /proc/67491/task/69002/root (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69002 tp_osd_tp ceph txt unknown /proc/67491/task/69002/exe (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69002 tp_osd_tp ceph NOFD /proc/67491/task/69002/fd (opendir: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69003 tp_osd_tp ceph cwd unknown /proc/67491/task/69003/cwd (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69003 tp_osd_tp ceph rtd unknown /proc/67491/task/69003/root (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69003 tp_osd_tp ceph txt unknown /proc/67491/task/69003/exe (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69003 tp_osd_tp ceph NOFD /proc/67491/task/69003/fd (opendir: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69004 tp_osd_tp ceph cwd unknown /proc/67491/task/69004/cwd (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69004 tp_osd_tp ceph rtd unknown /proc/67491/task/69004/root (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69004 tp_osd_tp ceph txt unknown /proc/67491/task/69004/exe (readlink: Permission denied) 2024-09-21T02:55:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69004 tp_osd_tp ceph NOFD /proc/67491/task/69004/fd (opendir: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69005 tp_osd_tp ceph cwd unknown /proc/67491/task/69005/cwd (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69005 tp_osd_tp ceph rtd unknown /proc/67491/task/69005/root (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69005 tp_osd_tp ceph txt unknown /proc/67491/task/69005/exe (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69005 tp_osd_tp ceph NOFD /proc/67491/task/69005/fd (opendir: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69006 osd_srv_h ceph cwd unknown /proc/67491/task/69006/cwd (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69006 osd_srv_h ceph rtd unknown /proc/67491/task/69006/root (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69006 osd_srv_h ceph txt unknown /proc/67491/task/69006/exe (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-osd 67491 69006 osd_srv_h ceph NOFD /proc/67491/task/69006/fd (opendir: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 68139 root cwd unknown /proc/68139/cwd (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 68139 root rtd unknown /proc/68139/root (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 68139 root txt unknown /proc/68139/exe (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 68139 root NOFD /proc/68139/fd (opendir: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 68158 root cwd unknown /proc/68158/cwd (readlink: Permission denied) 2024-09-21T02:55:14.125 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 68158 root rtd unknown /proc/68158/root (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 68158 root txt unknown /proc/68158/exe (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 68158 root NOFD /proc/68158/fd (opendir: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 71612 root cwd unknown /proc/71612/cwd (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 71612 root rtd unknown /proc/71612/root (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 71612 root txt unknown /proc/71612/exe (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/u 71612 root NOFD /proc/71612/fd (opendir: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 71613 root cwd unknown /proc/71613/cwd (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 71613 root rtd unknown /proc/71613/root (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 71613 root txt unknown /proc/71613/exe (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/7 71613 root NOFD /proc/71613/fd (opendir: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72222 root cwd unknown /proc/72222/cwd (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72222 root rtd unknown /proc/72222/root (readlink: Permission denied) 2024-09-21T02:55:14.126 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72222 root txt unknown /proc/72222/exe (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72222 root NOFD /proc/72222/fd (opendir: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72239 root cwd unknown /proc/72239/cwd (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72239 root rtd unknown /proc/72239/root (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72239 root txt unknown /proc/72239/exe (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72239 root NOFD /proc/72239/fd (opendir: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72240 root cwd unknown /proc/72240/cwd (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72240 root rtd unknown /proc/72240/root (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72240 root txt unknown /proc/72240/exe (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72240 root NOFD /proc/72240/fd (opendir: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72257 root cwd unknown /proc/72257/cwd (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72257 root rtd unknown /proc/72257/root (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72257 root txt unknown /proc/72257/exe (readlink: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72257 root NOFD /proc/72257/fd (opendir: Permission denied) 2024-09-21T02:55:14.127 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72258 root cwd unknown /proc/72258/cwd (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72258 root rtd unknown /proc/72258/root (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72258 root txt unknown /proc/72258/exe (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72258 root NOFD /proc/72258/fd (opendir: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72275 root cwd unknown /proc/72275/cwd (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72275 root rtd unknown /proc/72275/root (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72275 root txt unknown /proc/72275/exe (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72275 root NOFD /proc/72275/fd (opendir: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72276 root cwd unknown /proc/72276/cwd (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72276 root rtd unknown /proc/72276/root (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72276 root txt unknown /proc/72276/exe (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 72276 root NOFD /proc/72276/fd (opendir: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72293 root cwd unknown /proc/72293/cwd (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72293 root rtd unknown /proc/72293/root (readlink: Permission denied) 2024-09-21T02:55:14.128 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72293 root txt unknown /proc/72293/exe (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:journalct 72293 root NOFD /proc/72293/fd (opendir: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 72764 root cwd unknown /proc/72764/cwd (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 72764 root rtd unknown /proc/72764/root (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 72764 root txt unknown /proc/72764/exe (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 72764 root NOFD /proc/72764/fd (opendir: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 72770 root cwd unknown /proc/72770/cwd (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 72770 root rtd unknown /proc/72770/root (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 72770 root txt unknown /proc/72770/exe (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 72770 root NOFD /proc/72770/fd (opendir: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 ceph cwd unknown /proc/72782/cwd (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 ceph rtd unknown /proc/72782/root (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 ceph txt unknown /proc/72782/exe (readlink: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 ceph NOFD /proc/72782/fd (opendir: Permission denied) 2024-09-21T02:55:14.129 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72783 log ceph cwd unknown /proc/72782/task/72783/cwd (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72783 log ceph rtd unknown /proc/72782/task/72783/root (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72783 log ceph txt unknown /proc/72782/task/72783/exe (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72783 log ceph NOFD /proc/72782/task/72783/fd (opendir: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72785 msgr-work ceph cwd unknown /proc/72782/task/72785/cwd (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72785 msgr-work ceph rtd unknown /proc/72782/task/72785/root (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72785 msgr-work ceph txt unknown /proc/72782/task/72785/exe (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72785 msgr-work ceph NOFD /proc/72782/task/72785/fd (opendir: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72786 msgr-work ceph cwd unknown /proc/72782/task/72786/cwd (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72786 msgr-work ceph rtd unknown /proc/72782/task/72786/root (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72786 msgr-work ceph txt unknown /proc/72782/task/72786/exe (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72786 msgr-work ceph NOFD /proc/72782/task/72786/fd (opendir: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72787 msgr-work ceph cwd unknown /proc/72782/task/72787/cwd (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72787 msgr-work ceph rtd unknown /proc/72782/task/72787/root (readlink: Permission denied) 2024-09-21T02:55:14.130 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72787 msgr-work ceph txt unknown /proc/72782/task/72787/exe (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72787 msgr-work ceph NOFD /proc/72782/task/72787/fd (opendir: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72791 service ceph cwd unknown /proc/72782/task/72791/cwd (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72791 service ceph rtd unknown /proc/72782/task/72791/root (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72791 service ceph txt unknown /proc/72782/task/72791/exe (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72791 service ceph NOFD /proc/72782/task/72791/fd (opendir: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72792 admin_soc ceph cwd unknown /proc/72782/task/72792/cwd (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72792 admin_soc ceph rtd unknown /proc/72782/task/72792/root (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72792 admin_soc ceph txt unknown /proc/72782/task/72792/exe (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72792 admin_soc ceph NOFD /proc/72782/task/72792/fd (opendir: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72793 signal_ha ceph cwd unknown /proc/72782/task/72793/cwd (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72793 signal_ha ceph rtd unknown /proc/72782/task/72793/root (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72793 signal_ha ceph txt unknown /proc/72782/task/72793/exe (readlink: Permission denied) 2024-09-21T02:55:14.131 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72793 signal_ha ceph NOFD /proc/72782/task/72793/fd (opendir: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72794 io_contex ceph cwd unknown /proc/72782/task/72794/cwd (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72794 io_contex ceph rtd unknown /proc/72782/task/72794/root (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72794 io_contex ceph txt unknown /proc/72782/task/72794/exe (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72794 io_contex ceph NOFD /proc/72782/task/72794/fd (opendir: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72795 io_contex ceph cwd unknown /proc/72782/task/72795/cwd (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72795 io_contex ceph rtd unknown /proc/72782/task/72795/root (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72795 io_contex ceph txt unknown /proc/72782/task/72795/exe (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72795 io_contex ceph NOFD /proc/72782/task/72795/fd (opendir: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72796 ms_dispat ceph cwd unknown /proc/72782/task/72796/cwd (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72796 ms_dispat ceph rtd unknown /proc/72782/task/72796/root (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72796 ms_dispat ceph txt unknown /proc/72782/task/72796/exe (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72796 ms_dispat ceph NOFD /proc/72782/task/72796/fd (opendir: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72797 ms_local ceph cwd unknown /proc/72782/task/72797/cwd (readlink: Permission denied) 2024-09-21T02:55:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72797 ms_local ceph rtd unknown /proc/72782/task/72797/root (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72797 ms_local ceph txt unknown /proc/72782/task/72797/exe (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72797 ms_local ceph NOFD /proc/72782/task/72797/fd (opendir: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72798 safe_time ceph cwd unknown /proc/72782/task/72798/cwd (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72798 safe_time ceph rtd unknown /proc/72782/task/72798/root (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72798 safe_time ceph txt unknown /proc/72782/task/72798/exe (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72798 safe_time ceph NOFD /proc/72782/task/72798/fd (opendir: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72799 safe_time ceph cwd unknown /proc/72782/task/72799/cwd (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72799 safe_time ceph rtd unknown /proc/72782/task/72799/root (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72799 safe_time ceph txt unknown /proc/72782/task/72799/exe (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72799 safe_time ceph NOFD /proc/72782/task/72799/fd (opendir: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72800 ceph-mds ceph cwd unknown /proc/72782/task/72800/cwd (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72800 ceph-mds ceph rtd unknown /proc/72782/task/72800/root (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72800 ceph-mds ceph txt unknown /proc/72782/task/72800/exe (readlink: Permission denied) 2024-09-21T02:55:14.133 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72800 ceph-mds ceph NOFD /proc/72782/task/72800/fd (opendir: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72973 safe_time ceph cwd unknown /proc/72782/task/72973/cwd (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72973 safe_time ceph rtd unknown /proc/72782/task/72973/root (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72973 safe_time ceph txt unknown /proc/72782/task/72973/exe (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 72782 72973 safe_time ceph NOFD /proc/72782/task/72973/fd (opendir: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73141 root cwd unknown /proc/73141/cwd (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73141 root rtd unknown /proc/73141/root (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73141 root txt unknown /proc/73141/exe (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73141 root NOFD /proc/73141/fd (opendir: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73143 root cwd unknown /proc/73143/cwd (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73143 root rtd unknown /proc/73143/root (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73143 root txt unknown /proc/73143/exe (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73143 root NOFD /proc/73143/fd (opendir: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 ceph cwd unknown /proc/73145/cwd (readlink: Permission denied) 2024-09-21T02:55:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 ceph rtd unknown /proc/73145/root (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 ceph txt unknown /proc/73145/exe (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 ceph NOFD /proc/73145/fd (opendir: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73146 log ceph cwd unknown /proc/73145/task/73146/cwd (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73146 log ceph rtd unknown /proc/73145/task/73146/root (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73146 log ceph txt unknown /proc/73145/task/73146/exe (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73146 log ceph NOFD /proc/73145/task/73146/fd (opendir: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73148 msgr-work ceph cwd unknown /proc/73145/task/73148/cwd (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73148 msgr-work ceph rtd unknown /proc/73145/task/73148/root (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73148 msgr-work ceph txt unknown /proc/73145/task/73148/exe (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73148 msgr-work ceph NOFD /proc/73145/task/73148/fd (opendir: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73149 msgr-work ceph cwd unknown /proc/73145/task/73149/cwd (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73149 msgr-work ceph rtd unknown /proc/73145/task/73149/root (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73149 msgr-work ceph txt unknown /proc/73145/task/73149/exe (readlink: Permission denied) 2024-09-21T02:55:14.135 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73149 msgr-work ceph NOFD /proc/73145/task/73149/fd (opendir: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73150 msgr-work ceph cwd unknown /proc/73145/task/73150/cwd (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73150 msgr-work ceph rtd unknown /proc/73145/task/73150/root (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73150 msgr-work ceph txt unknown /proc/73145/task/73150/exe (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73150 msgr-work ceph NOFD /proc/73145/task/73150/fd (opendir: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73154 service ceph cwd unknown /proc/73145/task/73154/cwd (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73154 service ceph rtd unknown /proc/73145/task/73154/root (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73154 service ceph txt unknown /proc/73145/task/73154/exe (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73154 service ceph NOFD /proc/73145/task/73154/fd (opendir: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73155 admin_soc ceph cwd unknown /proc/73145/task/73155/cwd (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73155 admin_soc ceph rtd unknown /proc/73145/task/73155/root (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73155 admin_soc ceph txt unknown /proc/73145/task/73155/exe (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73155 admin_soc ceph NOFD /proc/73145/task/73155/fd (opendir: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73156 signal_ha ceph cwd unknown /proc/73145/task/73156/cwd (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73156 signal_ha ceph rtd unknown /proc/73145/task/73156/root (readlink: Permission denied) 2024-09-21T02:55:14.136 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73156 signal_ha ceph txt unknown /proc/73145/task/73156/exe (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73156 signal_ha ceph NOFD /proc/73145/task/73156/fd (opendir: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73157 io_contex ceph cwd unknown /proc/73145/task/73157/cwd (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73157 io_contex ceph rtd unknown /proc/73145/task/73157/root (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73157 io_contex ceph txt unknown /proc/73145/task/73157/exe (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73157 io_contex ceph NOFD /proc/73145/task/73157/fd (opendir: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73158 io_contex ceph cwd unknown /proc/73145/task/73158/cwd (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73158 io_contex ceph rtd unknown /proc/73145/task/73158/root (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73158 io_contex ceph txt unknown /proc/73145/task/73158/exe (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73158 io_contex ceph NOFD /proc/73145/task/73158/fd (opendir: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73159 ms_dispat ceph cwd unknown /proc/73145/task/73159/cwd (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73159 ms_dispat ceph rtd unknown /proc/73145/task/73159/root (readlink: Permission denied) 2024-09-21T02:55:14.137 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73159 ms_dispat ceph txt unknown /proc/73145/task/73159/exe (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73159 ms_dispat ceph NOFD /proc/73145/task/73159/fd (opendir: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73160 ms_local ceph cwd unknown /proc/73145/task/73160/cwd (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73160 ms_local ceph rtd unknown /proc/73145/task/73160/root (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73160 ms_local ceph txt unknown /proc/73145/task/73160/exe (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73160 ms_local ceph NOFD /proc/73145/task/73160/fd (opendir: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73161 safe_time ceph cwd unknown /proc/73145/task/73161/cwd (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73161 safe_time ceph rtd unknown /proc/73145/task/73161/root (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73161 safe_time ceph txt unknown /proc/73145/task/73161/exe (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73161 safe_time ceph NOFD /proc/73145/task/73161/fd (opendir: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73162 safe_time ceph cwd unknown /proc/73145/task/73162/cwd (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73162 safe_time ceph rtd unknown /proc/73145/task/73162/root (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73162 safe_time ceph txt unknown /proc/73145/task/73162/exe (readlink: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73162 safe_time ceph NOFD /proc/73145/task/73162/fd (opendir: Permission denied) 2024-09-21T02:55:14.138 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73163 ceph-mds ceph cwd unknown /proc/73145/task/73163/cwd (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73163 ceph-mds ceph rtd unknown /proc/73145/task/73163/root (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73163 ceph-mds ceph txt unknown /proc/73145/task/73163/exe (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73163 ceph-mds ceph NOFD /proc/73145/task/73163/fd (opendir: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73186 safe_time ceph cwd unknown /proc/73145/task/73186/cwd (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73186 safe_time ceph rtd unknown /proc/73145/task/73186/root (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73186 safe_time ceph txt unknown /proc/73145/task/73186/exe (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73145 73186 safe_time ceph NOFD /proc/73145/task/73186/fd (opendir: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73903 root cwd unknown /proc/73903/cwd (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73903 root rtd unknown /proc/73903/root (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73903 root txt unknown /proc/73903/exe (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 73903 root NOFD /proc/73903/fd (opendir: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73905 root cwd unknown /proc/73905/cwd (readlink: Permission denied) 2024-09-21T02:55:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73905 root rtd unknown /proc/73905/root (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73905 root txt unknown /proc/73905/exe (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 73905 root NOFD /proc/73905/fd (opendir: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 ceph cwd unknown /proc/73919/cwd (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 ceph rtd unknown /proc/73919/root (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 ceph txt unknown /proc/73919/exe (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 ceph NOFD /proc/73919/fd (opendir: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73920 log ceph cwd unknown /proc/73919/task/73920/cwd (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73920 log ceph rtd unknown /proc/73919/task/73920/root (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73920 log ceph txt unknown /proc/73919/task/73920/exe (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73920 log ceph NOFD /proc/73919/task/73920/fd (opendir: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73922 msgr-work ceph cwd unknown /proc/73919/task/73922/cwd (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73922 msgr-work ceph rtd unknown /proc/73919/task/73922/root (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73922 msgr-work ceph txt unknown /proc/73919/task/73922/exe (readlink: Permission denied) 2024-09-21T02:55:14.140 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73922 msgr-work ceph NOFD /proc/73919/task/73922/fd (opendir: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73923 msgr-work ceph cwd unknown /proc/73919/task/73923/cwd (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73923 msgr-work ceph rtd unknown /proc/73919/task/73923/root (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73923 msgr-work ceph txt unknown /proc/73919/task/73923/exe (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73923 msgr-work ceph NOFD /proc/73919/task/73923/fd (opendir: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73924 msgr-work ceph cwd unknown /proc/73919/task/73924/cwd (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73924 msgr-work ceph rtd unknown /proc/73919/task/73924/root (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73924 msgr-work ceph txt unknown /proc/73919/task/73924/exe (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73924 msgr-work ceph NOFD /proc/73919/task/73924/fd (opendir: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73928 service ceph cwd unknown /proc/73919/task/73928/cwd (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73928 service ceph rtd unknown /proc/73919/task/73928/root (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73928 service ceph txt unknown /proc/73919/task/73928/exe (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73928 service ceph NOFD /proc/73919/task/73928/fd (opendir: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73929 admin_soc ceph cwd unknown /proc/73919/task/73929/cwd (readlink: Permission denied) 2024-09-21T02:55:14.141 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73929 admin_soc ceph rtd unknown /proc/73919/task/73929/root (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73929 admin_soc ceph txt unknown /proc/73919/task/73929/exe (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73929 admin_soc ceph NOFD /proc/73919/task/73929/fd (opendir: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73930 signal_ha ceph cwd unknown /proc/73919/task/73930/cwd (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73930 signal_ha ceph rtd unknown /proc/73919/task/73930/root (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73930 signal_ha ceph txt unknown /proc/73919/task/73930/exe (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73930 signal_ha ceph NOFD /proc/73919/task/73930/fd (opendir: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73931 io_contex ceph cwd unknown /proc/73919/task/73931/cwd (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73931 io_contex ceph rtd unknown /proc/73919/task/73931/root (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73931 io_contex ceph txt unknown /proc/73919/task/73931/exe (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73931 io_contex ceph NOFD /proc/73919/task/73931/fd (opendir: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73932 io_contex ceph cwd unknown /proc/73919/task/73932/cwd (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73932 io_contex ceph rtd unknown /proc/73919/task/73932/root (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73932 io_contex ceph txt unknown /proc/73919/task/73932/exe (readlink: Permission denied) 2024-09-21T02:55:14.142 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73932 io_contex ceph NOFD /proc/73919/task/73932/fd (opendir: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73933 ms_dispat ceph cwd unknown /proc/73919/task/73933/cwd (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73933 ms_dispat ceph rtd unknown /proc/73919/task/73933/root (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73933 ms_dispat ceph txt unknown /proc/73919/task/73933/exe (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73933 ms_dispat ceph NOFD /proc/73919/task/73933/fd (opendir: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73934 ms_local ceph cwd unknown /proc/73919/task/73934/cwd (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73934 ms_local ceph rtd unknown /proc/73919/task/73934/root (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73934 ms_local ceph txt unknown /proc/73919/task/73934/exe (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73934 ms_local ceph NOFD /proc/73919/task/73934/fd (opendir: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73935 safe_time ceph cwd unknown /proc/73919/task/73935/cwd (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73935 safe_time ceph rtd unknown /proc/73919/task/73935/root (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73935 safe_time ceph txt unknown /proc/73919/task/73935/exe (readlink: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73935 safe_time ceph NOFD /proc/73919/task/73935/fd (opendir: Permission denied) 2024-09-21T02:55:14.143 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73936 safe_time ceph cwd unknown /proc/73919/task/73936/cwd (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73936 safe_time ceph rtd unknown /proc/73919/task/73936/root (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73936 safe_time ceph txt unknown /proc/73919/task/73936/exe (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73936 safe_time ceph NOFD /proc/73919/task/73936/fd (opendir: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73937 ceph-mds ceph cwd unknown /proc/73919/task/73937/cwd (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73937 ceph-mds ceph rtd unknown /proc/73919/task/73937/root (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73937 ceph-mds ceph txt unknown /proc/73919/task/73937/exe (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 73937 ceph-mds ceph NOFD /proc/73919/task/73937/fd (opendir: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 74051 safe_time ceph cwd unknown /proc/73919/task/74051/cwd (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 74051 safe_time ceph rtd unknown /proc/73919/task/74051/root (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 74051 safe_time ceph txt unknown /proc/73919/task/74051/exe (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 74051 safe_time ceph NOFD /proc/73919/task/74051/fd (opendir: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76284 ceph_time ceph cwd unknown /proc/73919/task/76284/cwd (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76284 ceph_time ceph rtd unknown /proc/73919/task/76284/root (readlink: Permission denied) 2024-09-21T02:55:14.144 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76284 ceph_time ceph txt unknown /proc/73919/task/76284/exe (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76284 ceph_time ceph NOFD /proc/73919/task/76284/fd (opendir: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76286 OpHistory ceph cwd unknown /proc/73919/task/76286/cwd (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76286 OpHistory ceph rtd unknown /proc/73919/task/76286/root (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76286 OpHistory ceph txt unknown /proc/73919/task/76286/exe (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76286 OpHistory ceph NOFD /proc/73919/task/76286/fd (opendir: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76288 ms_dispat ceph cwd unknown /proc/73919/task/76288/cwd (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76288 ms_dispat ceph rtd unknown /proc/73919/task/76288/root (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76288 ms_dispat ceph txt unknown /proc/73919/task/76288/exe (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76288 ms_dispat ceph NOFD /proc/73919/task/76288/fd (opendir: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76290 ms_dispat ceph cwd unknown /proc/73919/task/76290/cwd (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76290 ms_dispat ceph rtd unknown /proc/73919/task/76290/root (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76290 ms_dispat ceph txt unknown /proc/73919/task/76290/exe (readlink: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76290 ms_dispat ceph NOFD /proc/73919/task/76290/fd (opendir: Permission denied) 2024-09-21T02:55:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76293 mds_rank_ ceph cwd unknown /proc/73919/task/76293/cwd (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76293 mds_rank_ ceph rtd unknown /proc/73919/task/76293/root (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76293 mds_rank_ ceph txt unknown /proc/73919/task/76293/exe (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76293 mds_rank_ ceph NOFD /proc/73919/task/76293/fd (opendir: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76296 PQ_Finish ceph cwd unknown /proc/73919/task/76296/cwd (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76296 PQ_Finish ceph rtd unknown /proc/73919/task/76296/root (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76296 PQ_Finish ceph txt unknown /proc/73919/task/76296/exe (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76296 PQ_Finish ceph NOFD /proc/73919/task/76296/fd (opendir: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76297 safe_time ceph cwd unknown /proc/73919/task/76297/cwd (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76297 safe_time ceph rtd unknown /proc/73919/task/76297/root (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76297 safe_time ceph txt unknown /proc/73919/task/76297/exe (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76297 safe_time ceph NOFD /proc/73919/task/76297/fd (opendir: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76298 MR_Finish ceph cwd unknown /proc/73919/task/76298/cwd (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76298 MR_Finish ceph rtd unknown /proc/73919/task/76298/root (readlink: Permission denied) 2024-09-21T02:55:14.146 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76298 MR_Finish ceph txt unknown /proc/73919/task/76298/exe (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76298 MR_Finish ceph NOFD /proc/73919/task/76298/fd (opendir: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76302 md_submit ceph cwd unknown /proc/73919/task/76302/cwd (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76302 md_submit ceph rtd unknown /proc/73919/task/76302/root (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76302 md_submit ceph txt unknown /proc/73919/task/76302/exe (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 73919 76302 md_submit ceph NOFD /proc/73919/task/76302/fd (opendir: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 74267 root cwd unknown /proc/74267/cwd (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 74267 root rtd unknown /proc/74267/root (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 74267 root txt unknown /proc/74267/exe (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 74267 root NOFD /proc/74267/fd (opendir: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 74269 root cwd unknown /proc/74269/cwd (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 74269 root rtd unknown /proc/74269/root (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 74269 root txt unknown /proc/74269/exe (readlink: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:podman-in 74269 root NOFD /proc/74269/fd (opendir: Permission denied) 2024-09-21T02:55:14.147 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 ceph cwd unknown /proc/74271/cwd (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 ceph rtd unknown /proc/74271/root (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 ceph txt unknown /proc/74271/exe (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 ceph NOFD /proc/74271/fd (opendir: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74272 log ceph cwd unknown /proc/74271/task/74272/cwd (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74272 log ceph rtd unknown /proc/74271/task/74272/root (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74272 log ceph txt unknown /proc/74271/task/74272/exe (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74272 log ceph NOFD /proc/74271/task/74272/fd (opendir: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74274 msgr-work ceph cwd unknown /proc/74271/task/74274/cwd (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74274 msgr-work ceph rtd unknown /proc/74271/task/74274/root (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74274 msgr-work ceph txt unknown /proc/74271/task/74274/exe (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74274 msgr-work ceph NOFD /proc/74271/task/74274/fd (opendir: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74275 msgr-work ceph cwd unknown /proc/74271/task/74275/cwd (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74275 msgr-work ceph rtd unknown /proc/74271/task/74275/root (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74275 msgr-work ceph txt unknown /proc/74271/task/74275/exe (readlink: Permission denied) 2024-09-21T02:55:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74275 msgr-work ceph NOFD /proc/74271/task/74275/fd (opendir: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74276 msgr-work ceph cwd unknown /proc/74271/task/74276/cwd (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74276 msgr-work ceph rtd unknown /proc/74271/task/74276/root (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74276 msgr-work ceph txt unknown /proc/74271/task/74276/exe (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74276 msgr-work ceph NOFD /proc/74271/task/74276/fd (opendir: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74280 service ceph cwd unknown /proc/74271/task/74280/cwd (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74280 service ceph rtd unknown /proc/74271/task/74280/root (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74280 service ceph txt unknown /proc/74271/task/74280/exe (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74280 service ceph NOFD /proc/74271/task/74280/fd (opendir: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74281 admin_soc ceph cwd unknown /proc/74271/task/74281/cwd (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74281 admin_soc ceph rtd unknown /proc/74271/task/74281/root (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74281 admin_soc ceph txt unknown /proc/74271/task/74281/exe (readlink: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74281 admin_soc ceph NOFD /proc/74271/task/74281/fd (opendir: Permission denied) 2024-09-21T02:55:14.149 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74282 signal_ha ceph cwd unknown /proc/74271/task/74282/cwd (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74282 signal_ha ceph rtd unknown /proc/74271/task/74282/root (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74282 signal_ha ceph txt unknown /proc/74271/task/74282/exe (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74282 signal_ha ceph NOFD /proc/74271/task/74282/fd (opendir: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74283 io_contex ceph cwd unknown /proc/74271/task/74283/cwd (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74283 io_contex ceph rtd unknown /proc/74271/task/74283/root (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74283 io_contex ceph txt unknown /proc/74271/task/74283/exe (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74283 io_contex ceph NOFD /proc/74271/task/74283/fd (opendir: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74284 io_contex ceph cwd unknown /proc/74271/task/74284/cwd (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74284 io_contex ceph rtd unknown /proc/74271/task/74284/root (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74284 io_contex ceph txt unknown /proc/74271/task/74284/exe (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74284 io_contex ceph NOFD /proc/74271/task/74284/fd (opendir: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74285 ms_dispat ceph cwd unknown /proc/74271/task/74285/cwd (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74285 ms_dispat ceph rtd unknown /proc/74271/task/74285/root (readlink: Permission denied) 2024-09-21T02:55:14.150 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74285 ms_dispat ceph txt unknown /proc/74271/task/74285/exe (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74285 ms_dispat ceph NOFD /proc/74271/task/74285/fd (opendir: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74286 ms_local ceph cwd unknown /proc/74271/task/74286/cwd (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74286 ms_local ceph rtd unknown /proc/74271/task/74286/root (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74286 ms_local ceph txt unknown /proc/74271/task/74286/exe (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74286 ms_local ceph NOFD /proc/74271/task/74286/fd (opendir: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74287 safe_time ceph cwd unknown /proc/74271/task/74287/cwd (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74287 safe_time ceph rtd unknown /proc/74271/task/74287/root (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74287 safe_time ceph txt unknown /proc/74271/task/74287/exe (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74287 safe_time ceph NOFD /proc/74271/task/74287/fd (opendir: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74288 safe_time ceph cwd unknown /proc/74271/task/74288/cwd (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74288 safe_time ceph rtd unknown /proc/74271/task/74288/root (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74288 safe_time ceph txt unknown /proc/74271/task/74288/exe (readlink: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74288 safe_time ceph NOFD /proc/74271/task/74288/fd (opendir: Permission denied) 2024-09-21T02:55:14.151 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74289 ceph-mds ceph cwd unknown /proc/74271/task/74289/cwd (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74289 ceph-mds ceph rtd unknown /proc/74271/task/74289/root (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74289 ceph-mds ceph txt unknown /proc/74271/task/74289/exe (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74289 ceph-mds ceph NOFD /proc/74271/task/74289/fd (opendir: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74294 safe_time ceph cwd unknown /proc/74271/task/74294/cwd (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74294 safe_time ceph rtd unknown /proc/74271/task/74294/root (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74294 safe_time ceph txt unknown /proc/74271/task/74294/exe (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 74294 safe_time ceph NOFD /proc/74271/task/74294/fd (opendir: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76283 ceph_time ceph cwd unknown /proc/74271/task/76283/cwd (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76283 ceph_time ceph rtd unknown /proc/74271/task/76283/root (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76283 ceph_time ceph txt unknown /proc/74271/task/76283/exe (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76283 ceph_time ceph NOFD /proc/74271/task/76283/fd (opendir: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76285 OpHistory ceph cwd unknown /proc/74271/task/76285/cwd (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76285 OpHistory ceph rtd unknown /proc/74271/task/76285/root (readlink: Permission denied) 2024-09-21T02:55:14.152 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76285 OpHistory ceph txt unknown /proc/74271/task/76285/exe (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76285 OpHistory ceph NOFD /proc/74271/task/76285/fd (opendir: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76287 ms_dispat ceph cwd unknown /proc/74271/task/76287/cwd (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76287 ms_dispat ceph rtd unknown /proc/74271/task/76287/root (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76287 ms_dispat ceph txt unknown /proc/74271/task/76287/exe (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76287 ms_dispat ceph NOFD /proc/74271/task/76287/fd (opendir: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76289 ms_dispat ceph cwd unknown /proc/74271/task/76289/cwd (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76289 ms_dispat ceph rtd unknown /proc/74271/task/76289/root (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76289 ms_dispat ceph txt unknown /proc/74271/task/76289/exe (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76289 ms_dispat ceph NOFD /proc/74271/task/76289/fd (opendir: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76291 mds_rank_ ceph cwd unknown /proc/74271/task/76291/cwd (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76291 mds_rank_ ceph rtd unknown /proc/74271/task/76291/root (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76291 mds_rank_ ceph txt unknown /proc/74271/task/76291/exe (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76291 mds_rank_ ceph NOFD /proc/74271/task/76291/fd (opendir: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76292 PQ_Finish ceph cwd unknown /proc/74271/task/76292/cwd (readlink: Permission denied) 2024-09-21T02:55:14.153 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76292 PQ_Finish ceph rtd unknown /proc/74271/task/76292/root (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76292 PQ_Finish ceph txt unknown /proc/74271/task/76292/exe (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76292 PQ_Finish ceph NOFD /proc/74271/task/76292/fd (opendir: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76294 safe_time ceph cwd unknown /proc/74271/task/76294/cwd (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76294 safe_time ceph rtd unknown /proc/74271/task/76294/root (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76294 safe_time ceph txt unknown /proc/74271/task/76294/exe (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76294 safe_time ceph NOFD /proc/74271/task/76294/fd (opendir: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76295 MR_Finish ceph cwd unknown /proc/74271/task/76295/cwd (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76295 MR_Finish ceph rtd unknown /proc/74271/task/76295/root (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76295 MR_Finish ceph txt unknown /proc/74271/task/76295/exe (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76295 MR_Finish ceph NOFD /proc/74271/task/76295/fd (opendir: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76300 md_submit ceph cwd unknown /proc/74271/task/76300/cwd (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76300 md_submit ceph rtd unknown /proc/74271/task/76300/root (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76300 md_submit ceph txt unknown /proc/74271/task/76300/exe (readlink: Permission denied) 2024-09-21T02:55:14.154 INFO:tasks.workunit.client.0.smithi031.stdout:ceph-mds 74271 76300 md_submit ceph NOFD /proc/74271/task/76300/fd (opendir: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 75749 root cwd unknown /proc/75749/cwd (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 75749 root rtd unknown /proc/75749/root (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 75749 root txt unknown /proc/75749/exe (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/6 75749 root NOFD /proc/75749/fd (opendir: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 75866 root cwd unknown /proc/75866/cwd (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 75866 root rtd unknown /proc/75866/root (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 75866 root txt unknown /proc/75866/exe (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/2 75866 root NOFD /proc/75866/fd (opendir: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84140 root cwd unknown /proc/84140/cwd (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84140 root rtd unknown /proc/84140/root (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84140 root txt unknown /proc/84140/exe (readlink: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84140 root NOFD /proc/84140/fd (opendir: Permission denied) 2024-09-21T02:55:14.155 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84141 root cwd unknown /proc/84141/cwd (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84141 root rtd unknown /proc/84141/root (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84141 root txt unknown /proc/84141/exe (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/5 84141 root NOFD /proc/84141/fd (opendir: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85305 root cwd unknown /proc/85305/cwd (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85305 root rtd unknown /proc/85305/root (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85305 root txt unknown /proc/85305/exe (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85305 root NOFD /proc/85305/fd (opendir: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85306 root cwd unknown /proc/85306/cwd (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85306 root rtd unknown /proc/85306/root (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85306 root txt unknown /proc/85306/exe (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85306 root NOFD /proc/85306/fd (opendir: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85307 root cwd unknown /proc/85307/cwd (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85307 root rtd unknown /proc/85307/root (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85307 root txt unknown /proc/85307/exe (readlink: Permission denied) 2024-09-21T02:55:14.156 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/R 85307 root NOFD /proc/85307/fd (opendir: Permission denied) 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 87946 root cwd unknown /proc/87946/cwd (readlink: Permission denied) 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 87946 root rtd unknown /proc/87946/root (readlink: Permission denied) 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 87946 root txt unknown /proc/87946/exe (readlink: Permission denied) 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/0 87946 root NOFD /proc/87946/fd (opendir: Permission denied) 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu cwd DIR 0,120 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu txt REG 8,1 41128 11685 /usr/bin/timeout 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 2586930 19889 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 54 19892 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 3284 32609 /usr/lib/locale/en_US.utf8/LC_TIME 2024-09-21T02:55:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 286 32606 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 57 30370 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 34 32659 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 77 19891 /usr/lib/locale/en_US.utf8/LC_NAME 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 167 32601 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 59 32608 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 23 32605 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 369 32604 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu 0r FIFO 0,13 0t0 240868 pipe 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu 1w FIFO 0,13 0t0 240869 pipe 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:timeout 88479 ubuntu 2w FIFO 0,13 0t0 240870 pipe 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu cwd DIR 0,120 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:55:14.158 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu txt REG 8,1 1389072 11254 /usr/bin/bash 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 2586930 19889 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 54 19892 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 3284 32609 /usr/lib/locale/en_US.utf8/LC_TIME 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 286 32606 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 57 30370 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 34 32659 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 77 19891 /usr/lib/locale/en_US.utf8/LC_NAME 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 167 32601 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 59 32608 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 23 32605 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-09-21T02:55:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 195080 11252 /usr/lib64/libtinfo.so.6.2 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 369 32604 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu 0r FIFO 0,13 0t0 240868 pipe 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu 1w FIFO 0,13 0t0 240869 pipe 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu 2w FIFO 0,13 0t0 240870 pipe 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:fsync-tes 88497 ubuntu 255r REG 8,1 207 532139 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 88641 root cwd unknown /proc/88641/cwd (readlink: Permission denied) 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 88641 root rtd unknown /proc/88641/root (readlink: Permission denied) 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 88641 root txt unknown /proc/88641/exe (readlink: Permission denied) 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 88641 root NOFD /proc/88641/fd (opendir: Permission denied) 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 89461 root cwd unknown /proc/89461/cwd (readlink: Permission denied) 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 89461 root rtd unknown /proc/89461/root (readlink: Permission denied) 2024-09-21T02:55:14.160 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 89461 root txt unknown /proc/89461/exe (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/3 89461 root NOFD /proc/89461/fd (opendir: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 90273 root cwd unknown /proc/90273/cwd (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 90273 root rtd unknown /proc/90273/root (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 90273 root txt unknown /proc/90273/exe (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:kworker/1 90273 root NOFD /proc/90273/fd (opendir: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 91984 root cwd unknown /proc/91984/cwd (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 91984 root rtd unknown /proc/91984/root (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 91984 root txt unknown /proc/91984/exe (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:sudo 91984 root NOFD /proc/91984/fd (opendir: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:cephadm 92001 root cwd unknown /proc/92001/cwd (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:cephadm 92001 root rtd unknown /proc/92001/root (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:cephadm 92001 root txt unknown /proc/92001/exe (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:cephadm 92001 root NOFD /proc/92001/fd (opendir: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 root cwd unknown /proc/92058/cwd (readlink: Permission denied) 2024-09-21T02:55:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 root rtd unknown /proc/92058/root (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 root txt unknown /proc/92058/exe (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 root NOFD /proc/92058/fd (opendir: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92059 podman root cwd unknown /proc/92058/task/92059/cwd (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92059 podman root rtd unknown /proc/92058/task/92059/root (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92059 podman root txt unknown /proc/92058/task/92059/exe (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92059 podman root NOFD /proc/92058/task/92059/fd (opendir: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92060 podman root cwd unknown /proc/92058/task/92060/cwd (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92060 podman root rtd unknown /proc/92058/task/92060/root (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92060 podman root txt unknown /proc/92058/task/92060/exe (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92060 podman root NOFD /proc/92058/task/92060/fd (opendir: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92061 podman root cwd unknown /proc/92058/task/92061/cwd (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92061 podman root rtd unknown /proc/92058/task/92061/root (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92061 podman root txt unknown /proc/92058/task/92061/exe (readlink: Permission denied) 2024-09-21T02:55:14.162 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92061 podman root NOFD /proc/92058/task/92061/fd (opendir: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92062 podman root cwd unknown /proc/92058/task/92062/cwd (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92062 podman root rtd unknown /proc/92058/task/92062/root (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92062 podman root txt unknown /proc/92058/task/92062/exe (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92062 podman root NOFD /proc/92058/task/92062/fd (opendir: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92063 podman root cwd unknown /proc/92058/task/92063/cwd (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92063 podman root rtd unknown /proc/92058/task/92063/root (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92063 podman root txt unknown /proc/92058/task/92063/exe (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92063 podman root NOFD /proc/92058/task/92063/fd (opendir: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92064 podman root cwd unknown /proc/92058/task/92064/cwd (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92064 podman root rtd unknown /proc/92058/task/92064/root (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92064 podman root txt unknown /proc/92058/task/92064/exe (readlink: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92064 podman root NOFD /proc/92058/task/92064/fd (opendir: Permission denied) 2024-09-21T02:55:14.163 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92065 podman root cwd unknown /proc/92058/task/92065/cwd (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92065 podman root rtd unknown /proc/92058/task/92065/root (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92065 podman root txt unknown /proc/92058/task/92065/exe (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92065 podman root NOFD /proc/92058/task/92065/fd (opendir: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92066 podman root cwd unknown /proc/92058/task/92066/cwd (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92066 podman root rtd unknown /proc/92058/task/92066/root (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92066 podman root txt unknown /proc/92058/task/92066/exe (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92066 podman root NOFD /proc/92058/task/92066/fd (opendir: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92067 podman root cwd unknown /proc/92058/task/92067/cwd (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92067 podman root rtd unknown /proc/92058/task/92067/root (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92067 podman root txt unknown /proc/92058/task/92067/exe (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92067 podman root NOFD /proc/92058/task/92067/fd (opendir: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92068 podman root cwd unknown /proc/92058/task/92068/cwd (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92068 podman root rtd unknown /proc/92058/task/92068/root (readlink: Permission denied) 2024-09-21T02:55:14.164 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92068 podman root txt unknown /proc/92058/task/92068/exe (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92068 podman root NOFD /proc/92058/task/92068/fd (opendir: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92069 podman root cwd unknown /proc/92058/task/92069/cwd (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92069 podman root rtd unknown /proc/92058/task/92069/root (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92069 podman root txt unknown /proc/92058/task/92069/exe (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92069 podman root NOFD /proc/92058/task/92069/fd (opendir: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92070 podman root cwd unknown /proc/92058/task/92070/cwd (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92070 podman root rtd unknown /proc/92058/task/92070/root (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92070 podman root txt unknown /proc/92058/task/92070/exe (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92070 podman root NOFD /proc/92058/task/92070/fd (opendir: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92071 podman root cwd unknown /proc/92058/task/92071/cwd (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92071 podman root rtd unknown /proc/92058/task/92071/root (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92071 podman root txt unknown /proc/92058/task/92071/exe (readlink: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92058 92071 podman root NOFD /proc/92058/task/92071/fd (opendir: Permission denied) 2024-09-21T02:55:14.165 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 92073 root cwd unknown /proc/92073/cwd (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 92073 root rtd unknown /proc/92073/root (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 92073 root txt unknown /proc/92073/exe (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:conmon 92073 root NOFD /proc/92073/fd (opendir: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 root cwd unknown /proc/92099/cwd (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 root rtd unknown /proc/92099/root (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 root txt unknown /proc/92099/exe (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 root NOFD /proc/92099/fd (opendir: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92100 podman root cwd unknown /proc/92099/task/92100/cwd (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92100 podman root rtd unknown /proc/92099/task/92100/root (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92100 podman root txt unknown /proc/92099/task/92100/exe (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92100 podman root NOFD /proc/92099/task/92100/fd (opendir: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92101 podman root cwd unknown /proc/92099/task/92101/cwd (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92101 podman root rtd unknown /proc/92099/task/92101/root (readlink: Permission denied) 2024-09-21T02:55:14.166 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92101 podman root txt unknown /proc/92099/task/92101/exe (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92101 podman root NOFD /proc/92099/task/92101/fd (opendir: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92102 podman root cwd unknown /proc/92099/task/92102/cwd (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92102 podman root rtd unknown /proc/92099/task/92102/root (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92102 podman root txt unknown /proc/92099/task/92102/exe (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92102 podman root NOFD /proc/92099/task/92102/fd (opendir: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92103 podman root cwd unknown /proc/92099/task/92103/cwd (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92103 podman root rtd unknown /proc/92099/task/92103/root (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92103 podman root txt unknown /proc/92099/task/92103/exe (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92103 podman root NOFD /proc/92099/task/92103/fd (opendir: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92104 podman root cwd unknown /proc/92099/task/92104/cwd (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92104 podman root rtd unknown /proc/92099/task/92104/root (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92104 podman root txt unknown /proc/92099/task/92104/exe (readlink: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92104 podman root NOFD /proc/92099/task/92104/fd (opendir: Permission denied) 2024-09-21T02:55:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92105 podman root cwd unknown /proc/92099/task/92105/cwd (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92105 podman root rtd unknown /proc/92099/task/92105/root (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92105 podman root txt unknown /proc/92099/task/92105/exe (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92105 podman root NOFD /proc/92099/task/92105/fd (opendir: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92106 podman root cwd unknown /proc/92099/task/92106/cwd (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92106 podman root rtd unknown /proc/92099/task/92106/root (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92106 podman root txt unknown /proc/92099/task/92106/exe (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92106 podman root NOFD /proc/92099/task/92106/fd (opendir: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92107 podman root cwd unknown /proc/92099/task/92107/cwd (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92107 podman root rtd unknown /proc/92099/task/92107/root (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92107 podman root txt unknown /proc/92099/task/92107/exe (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92107 podman root NOFD /proc/92099/task/92107/fd (opendir: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92108 podman root cwd unknown /proc/92099/task/92108/cwd (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92108 podman root rtd unknown /proc/92099/task/92108/root (readlink: Permission denied) 2024-09-21T02:55:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92108 podman root txt unknown /proc/92099/task/92108/exe (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92108 podman root NOFD /proc/92099/task/92108/fd (opendir: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92113 podman root cwd unknown /proc/92099/task/92113/cwd (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92113 podman root rtd unknown /proc/92099/task/92113/root (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92113 podman root txt unknown /proc/92099/task/92113/exe (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92113 podman root NOFD /proc/92099/task/92113/fd (opendir: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92114 podman root cwd unknown /proc/92099/task/92114/cwd (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92114 podman root rtd unknown /proc/92099/task/92114/root (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92114 podman root txt unknown /proc/92099/task/92114/exe (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92114 podman root NOFD /proc/92099/task/92114/fd (opendir: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92115 podman root cwd unknown /proc/92099/task/92115/cwd (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92115 podman root rtd unknown /proc/92099/task/92115/root (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92115 podman root txt unknown /proc/92099/task/92115/exe (readlink: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92115 podman root NOFD /proc/92099/task/92115/fd (opendir: Permission denied) 2024-09-21T02:55:14.169 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92116 podman root cwd unknown /proc/92099/task/92116/cwd (readlink: Permission denied) 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92116 podman root rtd unknown /proc/92099/task/92116/root (readlink: Permission denied) 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92116 podman root txt unknown /proc/92099/task/92116/exe (readlink: Permission denied) 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:podman 92099 92116 podman root NOFD /proc/92099/task/92116/fd (opendir: Permission denied) 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu cwd DIR 0,120 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu txt REG 8,1 179232 13934 /usr/bin/lsof 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 5436544 2392 /usr/lib64/libcrypto.so.3.2.2 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 102672 11274 /usr/lib64/libz.so.1.2.11 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 70880 11221 /usr/lib64/libresolv.so.2 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 66320 12998 /usr/lib64/libkrb5support.so.0.1 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 99112 12990 /usr/lib64/libk5crypto.so.3.1 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 906368 12996 /usr/lib64/libkrb5.so.3.3 2024-09-21T02:55:14.170 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 358560 12986 /usr/lib64/libgssapi_krb5.so.2.2 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 636848 11327 /usr/lib64/libpcre2-8.so.0.11.0 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 23992 11348 /usr/lib64/libkeyutils.so.1.10 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 23872 11298 /usr/lib64/libcom_err.so.2.1 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 196224 13024 /usr/lib64/libtirpc.so.3.0.0 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 175744 11412 /usr/lib64/libselinux.so.1 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 0r FIFO 0,13 0t0 240868 pipe 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 1w FIFO 0,13 0t0 240869 pipe 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 2w FIFO 0,13 0t0 240870 pipe 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 3r DIR 0,20 0 1 /proc 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 4r DIR 0,20 0 244703 /proc/92111/fd 2024-09-21T02:55:14.171 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 5w FIFO 0,13 0t0 244708 pipe 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92111 ubuntu 6r FIFO 0,13 0t0 244709 pipe 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu cwd DIR 0,120 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu rtd DIR 8,1 4096 2 / 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu txt REG 8,1 179232 13934 /usr/bin/lsof 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 5436544 2392 /usr/lib64/libcrypto.so.3.2.2 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 346132 32690 /usr/lib/locale/C.utf8/LC_CTYPE 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 102672 11274 /usr/lib64/libz.so.1.2.11 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 70880 11221 /usr/lib64/libresolv.so.2 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 66320 12998 /usr/lib64/libkrb5support.so.0.1 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 99112 12990 /usr/lib64/libk5crypto.so.3.1 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 906368 12996 /usr/lib64/libkrb5.so.3.3 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 358560 12986 /usr/lib64/libgssapi_krb5.so.2.2 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 636848 11327 /usr/lib64/libpcre2-8.so.0.11.0 2024-09-21T02:55:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 2544296 11212 /usr/lib64/libc.so.6 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 23992 11348 /usr/lib64/libkeyutils.so.1.10 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 23872 11298 /usr/lib64/libcom_err.so.2.1 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 196224 13024 /usr/lib64/libtirpc.so.3.0.0 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 175744 11412 /usr/lib64/libselinux.so.1 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 26988 31782 /usr/lib64/gconv/gconv-modules.cache 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu mem REG 8,1 877424 15688 /usr/lib64/ld-linux-x86-64.so.2 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu 4r FIFO 0,13 0t0 244708 pipe 2024-09-21T02:55:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:lsof 92112 ubuntu 7w FIFO 0,13 0t0 244709 pipe 2024-09-21T02:55:14.173 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T02:55:14.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T02:55:14.174 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:13 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3424857267' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:55:14.175 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:13 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3424857267' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:55:14.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:55:14.260 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-09-21T02:55:14.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T02:55:14.484 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:14.625 DEBUG:teuthology.parallel:result is None 2024-09-21T02:55:14.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T02:55:14.662 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T02:55:14.663 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T02:55:14.679 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T02:55:14.679 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T02:55:14.679 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T02:55:14.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T02:55:14.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T02:55:14.932 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:14 smithi031 ceph-mon[31209]: pgmap v395: 97 pgs: 97 active+clean; 260 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.1 MiB/s wr, 589 op/s 2024-09-21T02:55:14.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:14 smithi165 ceph-mon[34818]: pgmap v395: 97 pgs: 97 active+clean; 260 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.1 MiB/s wr, 589 op/s 2024-09-21T02:55:14.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:14 smithi178 ceph-mon[34759]: pgmap v395: 97 pgs: 97 active+clean; 260 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.1 MiB/s wr, 589 op/s 2024-09-21T02:55:15.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.026+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/3628539296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4103b60 msgr2=0x7f0cb4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.026+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/3628539296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4103b60 0x7f0cb4103f70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/3628539296 shutdown_connections 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/3628539296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cb41094a0 0x7f0cb41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/3628539296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cb4109df0 0x7f0cb4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/3628539296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4103b60 0x7f0cb4103f70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/3628539296 >> 172.21.15.31:0/3628539296 conn(0x7f0cb407b850 msgr2=0x7f0cb407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/3628539296 shutdown_connections 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/3628539296 wait complete. 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.027+0000 7f0cb8d2a640 1 Processor -- start 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 -- start start 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cb4103b60 0x7f0cb419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cb41094a0 0x7f0cb419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 0x7f0cb419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4113ab0 con 0x7f0cb4109df0 2024-09-21T02:55:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4113930 con 0x7f0cb41094a0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb8d2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4113c30 con 0x7f0cb4103b60 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 0x7f0cb419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 0x7f0cb419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40596/0 (socket says 172.21.15.31:40596) 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1207276198 learned_addr learned my addr 172.21.15.31:0/1207276198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cb41094a0 0x7f0cb419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cb4103b60 0x7f0cb419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1207276198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cb4103b60 msgr2=0x7f0cb419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cb4103b60 0x7f0cb419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1207276198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cb41094a0 msgr2=0x7f0cb419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cb41094a0 0x7f0cb419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1207276198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41a0310 con 0x7f0cb4109df0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.028+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 0x7f0cb419fca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00c970 tx=0x7f0cac00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.029+0000 7f0ca37fe640 1 -- 172.21.15.31:0/1207276198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00ae60 con 0x7f0cb4109df0 2024-09-21T02:55:15.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.029+0000 7f0ca37fe640 1 -- 172.21.15.31:0/1207276198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac0054d0 con 0x7f0cb4109df0 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.029+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb41a0600 con 0x7f0cb4109df0 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.029+0000 7f0ca37fe640 1 -- 172.21.15.31:0/1207276198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac004d60 con 0x7f0cb4109df0 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.029+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb41a6e20 con 0x7f0cb4109df0 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.031+0000 7f0ca37fe640 1 -- 172.21.15.31:0/1207276198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0cac004fe0 con 0x7f0cb4109df0 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.031+0000 7f0cbf29e740 1 Processor -- start 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 -- start start 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 0x563a477646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47764be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a47765120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563a476daea0 con 0x563a473bd070 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563a476ca2d0 con 0x563a4739c390 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a476ce0e0 con 0x563a473941b0 2024-09-21T02:55:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 0x563a477646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 0x563a477646a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40608/0 (socket says 172.21.15.31:40608) 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/543057203 learned_addr learned my addr 172.21.15.31:0/543057203 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb259c640 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47764be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/543057203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 msgr2=0x563a47764be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47764be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/543057203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 msgr2=0x563a47765120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.032+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a47765120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/543057203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a47428650 con 0x563a473bd070 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0ca37fe640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0c9c0b66b0 0x7f0c9c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 0x563a477646a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800acc0 tx=0x7f0ca80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d53d0ab7e09f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0c9c0b66b0 0x7f0c9c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0ca37fe640 1 -- 172.21.15.31:0/1207276198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0cac0e7410 con 0x7f0cb4109df0 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0ca0ff9640 1 -- 172.21.15.31:0/543057203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8012450 con 0x563a473bd070 2024-09-21T02:55:15.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0ca0ff9640 1 -- 172.21.15.31:0/543057203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8012aa0 con 0x563a473bd070 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.033+0000 7f0ca0ff9640 1 -- 172.21.15.31:0/543057203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800f8f0 con 0x563a473bd070 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 -- 172.21.15.31:0/543057203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 msgr2=0x563a477646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0c9c0b66b0 0x7f0c9c0b8bc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ca402f500 tx=0x7f0ca40094a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 0x563a477646a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800acc0 tx=0x7f0ca80117b0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 -- 172.21.15.31:0/543057203 shutdown_connections 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47764be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a47765120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/543057203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a473bd070 0x563a477646a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 -- 172.21.15.31:0/543057203 >> 172.21.15.31:0/543057203 conn(0x563a473b1460 msgr2=0x563a47762590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 -- 172.21.15.31:0/543057203 shutdown_connections 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.034+0000 7f0cbf29e740 1 -- 172.21.15.31:0/543057203 wait complete. 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 -- start start 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 Processor -- start 2024-09-21T02:55:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a473941b0 0x563a4776fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473bd070 0x563a4777c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563a4752b700 con 0x563a4739c390 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563a47665160 con 0x563a473941b0 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a47772e80 con 0x563a473bd070 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cbf29e740 10 client.0 _ll_register_callbacks cb 0x563a4767e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40624/0 (socket says 172.21.15.31:40624) 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/1293829157 learned_addr learned my addr 172.21.15.31:0/1293829157 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a473941b0 0x563a4776fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473bd070 0x563a4777c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/1293829157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473bd070 msgr2=0x563a4777c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473bd070 0x563a4777c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a473941b0 msgr2=0x563a4776fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a473941b0 0x563a4776fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb1d9b640 1 -- 172.21.15.31:0/1293829157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a47692c90 con 0x563a4739c390 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473bd070 0x563a4777c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.035+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a473941b0 0x563a4776fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477702b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80058c0 tx=0x7f0ca8012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d85a58fec75556 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1293829157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8005240 con 0x563a4739c390 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1293829157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0ca80053e0 con 0x563a4739c390 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca8012aa0 con 0x563a4739c390 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1293829157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ca80128f0 con 0x563a4739c390 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0cbf29e740 10 client.16188 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a4777eba0 con 0x563a4739c390 2024-09-21T02:55:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1293829157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8012aa0 con 0x563a4739c390 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1293829157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0ca802b020 con 0x563a4739c390 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0cbf29e740 10 client.16188 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.036+0000 7f0cbf29e740 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0cbf29e740 20 client.16188 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a4737b9f0 0x563a47781f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0cbf29e740 4 client.16188 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a476dcdf0 con 0x563a4737b9f0 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a4737b9f0 0x563a47781f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0c99ffb640 10 client.16188 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.037+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a4737b9f0 0x563a47781f00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2d3e05b1d102fed server_cookie=a3b4454c9194e2b9 in_seq=0 out_seq=0 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1293829157 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a476dcdf0 con 0x563a4737b9f0 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0c99ffb640 10 client.16188 handle_command_reply: tid=0 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 client.16188 shutdown 2024-09-21T02:55:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 msgr2=0x563a477702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477702b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80058c0 tx=0x7f0ca8012700 comp rx=0 tx=0).stop 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 shutdown_connections 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a4737b9f0 0x563a47781f00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473bd070 0x563a4777c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477702b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0c99ffb640 0 client.16188 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1293829157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a473941b0 0x563a4776fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 >> 172.21.15.31:0/1293829157 conn(0x563a473b1460 msgr2=0x563a47767880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 shutdown_connections 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1293829157 wait complete. 2024-09-21T02:55:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.038+0000 7f0cbf29e740 20 client.16188 trim_cache size 0 max 16384 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 Processor -- start 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 -- start start 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 0x563a47770230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a4777c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a4776cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41af660 con 0x563a4737b9f0 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b11a0 con 0x563a4739c390 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41afd30 con 0x563a473941b0 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cb259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 0x563a47770230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cb259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 0x563a47770230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40626/0 (socket says 172.21.15.31:40626) 2024-09-21T02:55:15.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cb259c640 1 -- 172.21.15.31:0/611405590 learned_addr learned my addr 172.21.15.31:0/611405590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a4777c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.052+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a4776cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb259c640 1 -- 172.21.15.31:0/611405590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 msgr2=0x563a4777c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb259c640 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a4777c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb259c640 1 -- 172.21.15.31:0/611405590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 msgr2=0x563a4776cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb259c640 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a4776cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb259c640 1 -- 172.21.15.31:0/611405590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a47766bc0 con 0x563a4737b9f0 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a4776cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0cb259c640 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 0x563a47770230 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffa1ce05b14f580 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0c9bfff640 1 -- 172.21.15.31:0/611405590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4003c30 con 0x563a4737b9f0 2024-09-21T02:55:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0c9bfff640 1 -- 172.21.15.31:0/611405590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca4036040 con 0x563a4737b9f0 2024-09-21T02:55:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.053+0000 7f0c9bfff640 1 -- 172.21.15.31:0/611405590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca402faa0 con 0x563a4737b9f0 2024-09-21T02:55:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 -- 172.21.15.31:0/611405590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 msgr2=0x563a47770230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 0x563a47770230 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca403b000 comp rx=0 tx=0).stop 2024-09-21T02:55:15.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 -- 172.21.15.31:0/611405590 shutdown_connections 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a4777c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4739c390 0x563a4776cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/611405590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4737b9f0 0x563a47770230 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 -- 172.21.15.31:0/611405590 >> 172.21.15.31:0/611405590 conn(0x563a473b1460 msgr2=0x563a4776fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 -- 172.21.15.31:0/611405590 shutdown_connections 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 -- 172.21.15.31:0/611405590 wait complete. 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 -- start start 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.054+0000 7f0cbf29e740 1 Processor -- start 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4737b9f0 0x563a47782f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47783480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b0ad0 con 0x563a4739c390 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b0400 con 0x563a4737b9f0 2024-09-21T02:55:15.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41ad640 con 0x563a473941b0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cbf29e740 10 client.0 _ll_register_callbacks cb 0x7f0ca800ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477839c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40636/0 (socket says 172.21.15.31:40636) 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1425748909 learned_addr learned my addr 172.21.15.31:0/1425748909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb1d9b640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47783480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4737b9f0 0x563a47782f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1425748909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 msgr2=0x563a47783480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47783480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4737b9f0 msgr2=0x563a47782f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4737b9f0 0x563a47782f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.055+0000 7f0cb2d9d640 1 -- 172.21.15.31:0/1425748909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a477eb680 con 0x563a4739c390 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477839c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0a9850 tx=0x7f0cac0a9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2938050714c4b69a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0c99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:15.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1425748909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac01d040 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1425748909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0cac0a8de0 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca4036040 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1425748909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0cac0acd90 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 10 client.16194 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a477ec8d0 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1425748909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca4036040 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1425748909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0cac0b2ef0 con 0x563a4739c390 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 10 client.16194 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 10 client.16194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 20 client.16194 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a473bd070 0x563a477ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 4 client.16194 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.056+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a476d88a0 con 0x563a473bd070 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a473bd070 0x563a477ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:15.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cb259c640 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a473bd070 0x563a477ef110 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a400dd43d99d4a2 server_cookie=1a9e7403c72bf21d in_seq=0 out_seq=0 2024-09-21T02:55:15.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0c99ffb640 10 client.16194 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:55:15.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0c99ffb640 1 -- 172.21.15.31:0/1425748909 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a476d88a0 con 0x563a473bd070 2024-09-21T02:55:15.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0c99ffb640 10 client.16194 handle_command_reply: tid=0 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 client.16194 shutdown 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 msgr2=0x563a477839c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477839c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0a9850 tx=0x7f0cac0a9780 comp rx=0 tx=0).stop 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 shutdown_connections 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x563a473bd070 0x563a477ef110 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a473941b0 0x563a47783480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a4739c390 0x563a477839c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 --2- 172.21.15.31:0/1425748909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a4737b9f0 0x563a47782f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.057+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 >> 172.21.15.31:0/1425748909 conn(0x563a473b1460 msgr2=0x563a4776e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0c99ffb640 0 client.16194 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 shutdown_connections 2024-09-21T02:55:15.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cbf29e740 1 -- 172.21.15.31:0/1425748909 wait complete. 2024-09-21T02:55:15.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cbf29e740 20 client.16194 trim_cache size 0 max 16384 2024-09-21T02:55:15.064 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:15.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0c9c0b66b0 msgr2=0x7f0c9c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0c9c0b66b0 0x7f0c9c0b8bc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ca402f500 tx=0x7f0ca40094a0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 msgr2=0x7f0cb419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 0x7f0cb419fca0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00c970 tx=0x7f0cac00cd70 comp rx=0 tx=0).stop 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 shutdown_connections 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0c9c0b66b0 0x7f0c9c0b8bc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4109df0 0x7f0cb419fca0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0cb41094a0 0x7f0cb419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 --2- 172.21.15.31:0/1207276198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0cb4103b60 0x7f0cb419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 >> 172.21.15.31:0/1207276198 conn(0x7f0cb407b850 msgr2=0x7f0cb4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 shutdown_connections 2024-09-21T02:55:15.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:15.058+0000 7f0cb8d2a640 1 -- 172.21.15.31:0/1207276198 wait complete. 2024-09-21T02:55:15.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:55:16.178 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.627+0000 7f48a8be8640 1 -- 172.21.15.31:0/1875582597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 msgr2=0x7f48a41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.627+0000 7f48a8be8640 1 --2- 172.21.15.31:0/1875582597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a41087c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4898009900 tx=0x7f489802a410 comp rx=0 tx=0).stop 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.627+0000 7f48a8be8640 1 -- 172.21.15.31:0/1875582597 shutdown_connections 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.627+0000 7f48a8be8640 1 --2- 172.21.15.31:0/1875582597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a4108d00 0x7f48a410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.627+0000 7f48a8be8640 1 --2- 172.21.15.31:0/1875582597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a41087c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.627+0000 7f48a8be8640 1 --2- 172.21.15.31:0/1875582597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4102a70 0x7f48a4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.628+0000 7f48a8be8640 1 -- 172.21.15.31:0/1875582597 >> 172.21.15.31:0/1875582597 conn(0x7f48a4079710 msgr2=0x7f48a4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.628+0000 7f48a8be8640 1 -- 172.21.15.31:0/1875582597 shutdown_connections 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.628+0000 7f48a8be8640 1 -- 172.21.15.31:0/1875582597 wait complete. 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.628+0000 7f48a8be8640 1 Processor -- start 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.628+0000 7f48a8be8640 1 -- start start 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a8be8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4102a70 0x7f48a419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a8be8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a8be8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a4108d00 0x7f48a419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a8be8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112a10 con 0x7f48a41083b0 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a8be8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112890 con 0x7f48a4108d00 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a8be8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112b90 con 0x7f48a4102a70 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40656/0 (socket says 172.21.15.31:40656) 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 -- 172.21.15.31:0/27189824 learned_addr learned my addr 172.21.15.31:0/27189824 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a259c640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4102a70 0x7f48a419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a4108d00 0x7f48a419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 -- 172.21.15.31:0/27189824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4102a70 msgr2=0x7f48a419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4102a70 0x7f48a419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 -- 172.21.15.31:0/27189824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a4108d00 msgr2=0x7f48a419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a4108d00 0x7f48a419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 -- 172.21.15.31:0/27189824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a41a65e0 con 0x7f48a41083b0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.629+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a419b860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4898005ab0 tx=0x7f4898003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.630+0000 7f48937fe640 1 -- 172.21.15.31:0/27189824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489803f070 con 0x7f48a41083b0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.630+0000 7f48937fe640 1 -- 172.21.15.31:0/27189824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4898031d60 con 0x7f48a41083b0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.630+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a41a6870 con 0x7f48a41083b0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.630+0000 7f48937fe640 1 -- 172.21.15.31:0/27189824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898035030 con 0x7f48a41083b0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.630+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a41a6d50 con 0x7f48a41083b0 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.632+0000 7f48af15c740 1 Processor -- start 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.632+0000 7f48af15c740 1 -- start start 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48af15c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 0x5603675662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48af15c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671961b0 0x560367566800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48af15c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56036719e390 0x560367566d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48af15c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5603674dcea0 con 0x5603671bf070 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48af15c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5603674cc2d0 con 0x56036719e390 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48af15c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5603674d00e0 con 0x5603671961b0 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 0x5603675662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 0x5603675662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40670/0 (socket says 172.21.15.31:40670) 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a2d9d640 1 -- 172.21.15.31:0/3634191113 learned_addr learned my addr 172.21.15.31:0/3634191113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48937fe640 1 -- 172.21.15.31:0/27189824 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4898042070 con 0x7f48a41083b0 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a259c640 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671961b0 0x560367566800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a259c640 1 -- 172.21.15.31:0/3634191113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56036719e390 msgr2=0x560367566d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a259c640 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56036719e390 0x560367566d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a259c640 1 -- 172.21.15.31:0/3634191113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 msgr2=0x5603675662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a259c640 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 0x5603675662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a259c640 1 -- 172.21.15.31:0/3634191113 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036722a650 con 0x5603671961b0 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.633+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 0x5603675662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f48a259c640 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671961b0 0x560367566800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48940058c0 tx=0x7f489400e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b5214e1869f32b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f48937fe640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f488c0b6680 0x7f488c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f4890ff9640 1 -- 172.21.15.31:0/3634191113 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894005020 con 0x5603671961b0 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f4890ff9640 1 -- 172.21.15.31:0/3634191113 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48940047a0 con 0x5603671961b0 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f4890ff9640 1 -- 172.21.15.31:0/3634191113 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894010690 con 0x5603671961b0 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f488c0b6680 0x7f488c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.634+0000 7f48937fe640 1 -- 172.21.15.31:0/27189824 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4898107fe0 con 0x7f48a41083b0 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 -- 172.21.15.31:0/3634191113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671961b0 msgr2=0x560367566800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f488c0b6680 0x7f488c0b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fd0 tx=0x7f489c009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671961b0 0x560367566800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48940058c0 tx=0x7f489400e350 comp rx=0 tx=0).stop 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 -- 172.21.15.31:0/3634191113 shutdown_connections 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671961b0 0x560367566800 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56036719e390 0x560367566d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 --2- 172.21.15.31:0/3634191113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5603671bf070 0x5603675662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 -- 172.21.15.31:0/3634191113 >> 172.21.15.31:0/3634191113 conn(0x5603671b3460 msgr2=0x5603675641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 -- 172.21.15.31:0/3634191113 shutdown_connections 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.635+0000 7f48af15c740 1 -- 172.21.15.31:0/3634191113 wait complete. 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 -- start start 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 Processor -- start 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367575db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 0x5603675762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671bf070 0x560367576870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56036732d700 con 0x56036719e390 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560367467160 con 0x5603671961b0 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560367578e00 con 0x5603671bf070 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48af15c740 10 client.0 _ll_register_callbacks cb 0x5603674809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48a1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 0x5603675762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48a1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 0x5603675762f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40672/0 (socket says 172.21.15.31:40672) 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48a1d9b640 1 -- 172.21.15.31:0/921662312 learned_addr learned my addr 172.21.15.31:0/921662312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.636+0000 7f48a259c640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367575db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671bf070 0x560367576870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a1d9b640 1 -- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671bf070 msgr2=0x560367576870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671bf070 0x560367576870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a1d9b640 1 -- 172.21.15.31:0/921662312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 msgr2=0x560367575db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367575db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a1d9b640 1 -- 172.21.15.31:0/921662312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560367494c90 con 0x56036719e390 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 0x5603675762f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f48981057b0 tx=0x7f489802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad81320243f239bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f4889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f4889ffb640 1 -- 172.21.15.31:0/921662312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898049070 con 0x56036719e390 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f4889ffb640 1 -- 172.21.15.31:0/921662312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f48980b50b0 con 0x56036719e390 2024-09-21T02:55:16.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48940047a0 con 0x56036719e390 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f4889ffb640 1 -- 172.21.15.31:0/921662312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48980efe60 con 0x56036719e390 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48af15c740 10 client.16206 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.637+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560367584b10 con 0x56036719e390 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f4889ffb640 1 -- 172.21.15.31:0/921662312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48940047a0 con 0x56036719e390 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f4889ffb640 1 -- 172.21.15.31:0/921662312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f489810a0d0 con 0x56036719e390 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48af15c740 10 client.16206 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48af15c740 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48af15c740 20 client.16206 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48af15c740 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56036717d9f0 0x560367587f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48af15c740 4 client.16206 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603674dedf0 con 0x56036717d9f0 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.638+0000 7f48a259c640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56036717d9f0 0x560367587f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f48a259c640 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56036717d9f0 0x560367587f30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=65f79ec2afd31c8f server_cookie=91ea2fbc0421618f in_seq=0 out_seq=0 2024-09-21T02:55:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f4889ffb640 10 client.16206 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f4889ffb640 1 -- 172.21.15.31:0/921662312 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603674dedf0 con 0x56036717d9f0 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f4889ffb640 10 client.16206 handle_command_reply: tid=0 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f48af15c740 1 client.16206 shutdown 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 msgr2=0x5603675762f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.639+0000 7f48af15c740 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 0x5603675762f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f48981057b0 tx=0x7f489802a410 comp rx=0 tx=0).stop 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 shutdown_connections 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56036717d9f0 0x560367587f30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5603671bf070 0x560367576870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036719e390 0x5603675762f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f4889ffb640 0 client.16206 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 --2- 172.21.15.31:0/921662312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367575db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 >> 172.21.15.31:0/921662312 conn(0x5603671b3460 msgr2=0x560367567cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 shutdown_connections 2024-09-21T02:55:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 1 -- 172.21.15.31:0/921662312 wait complete. 2024-09-21T02:55:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.640+0000 7f48af15c740 20 client.16206 trim_cache size 0 max 16384 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 Processor -- start 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 -- start start 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x560367574630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367563ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x560367564010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41ad4e0 con 0x56036717d9f0 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41af020 con 0x5603671961b0 2024-09-21T02:55:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48af15c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41adbb0 con 0x56036719e390 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x560367574630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.653+0000 7f48a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x560367574630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40678/0 (socket says 172.21.15.31:40678) 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 -- 172.21.15.31:0/3608088867 learned_addr learned my addr 172.21.15.31:0/3608088867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x560367564010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 -- 172.21.15.31:0/3608088867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 msgr2=0x560367564010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x560367564010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 -- 172.21.15.31:0/3608088867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 msgr2=0x560367563ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367563ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 -- 172.21.15.31:0/3608088867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560367567100 con 0x56036717d9f0 2024-09-21T02:55:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f48a259c640 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x560367574630 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f489400ca70 tx=0x7f489400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b64815b5968193 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f488bfff640 1 -- 172.21.15.31:0/3608088867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894027450 con 0x56036717d9f0 2024-09-21T02:55:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f488bfff640 1 -- 172.21.15.31:0/3608088867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4894027aa0 con 0x56036717d9f0 2024-09-21T02:55:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.654+0000 7f488bfff640 1 -- 172.21.15.31:0/3608088867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894010b10 con 0x56036717d9f0 2024-09-21T02:55:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 -- 172.21.15.31:0/3608088867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 msgr2=0x560367574630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x560367574630 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f489400ca70 tx=0x7f489400c480 comp rx=0 tx=0).stop 2024-09-21T02:55:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 -- 172.21.15.31:0/3608088867 shutdown_connections 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x560367564010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x560367563ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 --2- 172.21.15.31:0/3608088867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x560367574630 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 -- 172.21.15.31:0/3608088867 >> 172.21.15.31:0/3608088867 conn(0x5603671b3460 msgr2=0x560367570f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 -- 172.21.15.31:0/3608088867 shutdown_connections 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 -- 172.21.15.31:0/3608088867 wait complete. 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.655+0000 7f48af15c740 1 -- start start 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 Processor -- start 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x56036757a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x56036757aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x5603675f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41ae950 con 0x56036717d9f0 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41ae280 con 0x5603671961b0 2024-09-21T02:55:16.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41ab4c0 con 0x56036719e390 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48af15c740 10 client.0 _ll_register_callbacks cb 0x7f4898005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x56036757a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x56036757a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40680/0 (socket says 172.21.15.31:40680) 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 -- 172.21.15.31:0/3416713974 learned_addr learned my addr 172.21.15.31:0/3416713974 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a2d9d640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x5603675f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x56036757aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 -- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 msgr2=0x5603675f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x5603675f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 -- 172.21.15.31:0/3416713974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 msgr2=0x56036757aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x56036757aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.656+0000 7f48a259c640 1 -- 172.21.15.31:0/3416713974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603675f1dd0 con 0x56036717d9f0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48a259c640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x56036757a4f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4894005d90 tx=0x7f48940047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0f25f67ccecd3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f4889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:16.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f4889ffb640 1 -- 172.21.15.31:0/3416713974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489403c030 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4894027aa0 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f4889ffb640 1 -- 172.21.15.31:0/3416713974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4894027450 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f4889ffb640 1 -- 172.21.15.31:0/3416713974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4894033040 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 10 client.16212 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603675f2f60 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f4889ffb640 1 -- 172.21.15.31:0/3416713974 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4894027aa0 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f4889ffb640 1 -- 172.21.15.31:0/3416713974 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f48940384a0 con 0x56036717d9f0 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 10 client.16212 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 20 client.16212 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5603671bf070 0x5603675f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 4 client.16212 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.657+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603674da8a0 con 0x5603671bf070 2024-09-21T02:55:16.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5603671bf070 0x5603675f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48a1d9b640 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5603671bf070 0x5603675f5570 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5834d9788abab42 server_cookie=a7bca8f90702fc39 in_seq=0 out_seq=0 2024-09-21T02:55:16.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f4889ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f4889ffb640 1 -- 172.21.15.31:0/3416713974 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603674da8a0 con 0x5603671bf070 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f4889ffb640 10 client.16212 handle_command_reply: tid=0 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48af15c740 1 client.16212 shutdown 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 msgr2=0x56036757a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48af15c740 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x56036757a4f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4894005d90 tx=0x7f48940047a0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 shutdown_connections 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.658+0000 7f48af15c740 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5603671bf070 0x5603675f5570 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56036719e390 0x5603675f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603671961b0 0x56036757aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f4889ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:55:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 1 --2- 172.21.15.31:0/3416713974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036717d9f0 0x56036757a4f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 >> 172.21.15.31:0/3416713974 conn(0x5603671b3460 msgr2=0x560367575b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 shutdown_connections 2024-09-21T02:55:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 1 -- 172.21.15.31:0/3416713974 wait complete. 2024-09-21T02:55:16.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48af15c740 20 client.16212 trim_cache size 0 max 16384 2024-09-21T02:55:16.665 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f488c0b6680 msgr2=0x7f488c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f488c0b6680 0x7f488c0b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fd0 tx=0x7f489c009760 comp rx=0 tx=0).stop 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 msgr2=0x7f48a419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a419b860 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4898005ab0 tx=0x7f4898003cd0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 shutdown_connections 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f488c0b6680 0x7f488c0b8b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a4108d00 0x7f48a419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a41083b0 0x7f48a419b860 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 --2- 172.21.15.31:0/27189824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4102a70 0x7f48a419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 >> 172.21.15.31:0/27189824 conn(0x7f48a4079710 msgr2=0x7f48a4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.659+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 shutdown_connections 2024-09-21T02:55:16.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:16.660+0000 7f48a8be8640 1 -- 172.21.15.31:0/27189824 wait complete. 2024-09-21T02:55:16.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:16 smithi165 ceph-mon[34818]: pgmap v396: 97 pgs: 97 active+clean; 260 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 742 op/s 2024-09-21T02:55:16.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:16 smithi178 ceph-mon[34759]: pgmap v396: 97 pgs: 97 active+clean; 260 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 742 op/s 2024-09-21T02:55:16.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:16 smithi031 ceph-mon[31209]: pgmap v396: 97 pgs: 97 active+clean; 260 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 742 op/s 2024-09-21T02:55:17.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:55:17.634 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 -- 172.21.15.31:0/1913736931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af0108cf0 msgr2=0x7f5af010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 --2- 172.21.15.31:0/1913736931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af0108cf0 0x7f5af010f520 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009ca0 tx=0x7f5ae802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 -- 172.21.15.31:0/1913736931 shutdown_connections 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 --2- 172.21.15.31:0/1913736931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af01083a0 0x7f5af01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 --2- 172.21.15.31:0/1913736931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af0108cf0 0x7f5af010f520 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 --2- 172.21.15.31:0/1913736931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af0102a60 0x7f5af0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 -- 172.21.15.31:0/1913736931 >> 172.21.15.31:0/1913736931 conn(0x7f5af0079730 msgr2=0x7f5af0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:18.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 -- 172.21.15.31:0/1913736931 shutdown_connections 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.136+0000 7f5af490e640 1 -- 172.21.15.31:0/1913736931 wait complete. 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 Processor -- start 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 -- start start 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af0102a60 0x7f5af019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 0x7f5af019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af0108cf0 0x7f5af019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0113a80 con 0x7f5af01083a0 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0113900 con 0x7f5af0108cf0 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5af490e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0113c00 con 0x7f5af0102a60 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.137+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 0x7f5af019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 0x7f5af019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40700/0 (socket says 172.21.15.31:40700) 2024-09-21T02:55:18.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 -- 172.21.15.31:0/119296630 learned_addr learned my addr 172.21.15.31:0/119296630 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aee59c640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af0102a60 0x7f5af019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af0108cf0 0x7f5af019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 -- 172.21.15.31:0/119296630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af0102a60 msgr2=0x7f5af019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af0102a60 0x7f5af019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 -- 172.21.15.31:0/119296630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af0108cf0 msgr2=0x7f5af019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af0108cf0 0x7f5af019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 -- 172.21.15.31:0/119296630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af01a02c0 con 0x7f5af01083a0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 0x7f5af019a9b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c970 tx=0x7f5ae400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5adf7fe640 1 -- 172.21.15.31:0/119296630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4007bb0 con 0x7f5af01083a0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af01a05b0 con 0x7f5af01083a0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5adf7fe640 1 -- 172.21.15.31:0/119296630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4007d50 con 0x7f5af01083a0 2024-09-21T02:55:18.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5adf7fe640 1 -- 172.21.15.31:0/119296630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4018800 con 0x7f5af01083a0 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.138+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01a6dd0 con 0x7f5af01083a0 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.141+0000 7f5afae82740 1 Processor -- start 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 -- start start 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b707f070 0x5617b74262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b7426800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7426d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5617b739cea0 con 0x5617b705e390 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5617b738c2d0 con 0x5617b707f070 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5afae82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5617b73900e0 con 0x5617b70561b0 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5adf7fe640 1 -- 172.21.15.31:0/119296630 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5ae400fa20 con 0x7f5af01083a0 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7426d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5aeed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b707f070 0x5617b74262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7426d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40710/0 (socket says 172.21.15.31:40710) 2024-09-21T02:55:18.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.142+0000 7f5aedd9b640 1 -- 172.21.15.31:0/3768425396 learned_addr learned my addr 172.21.15.31:0/3768425396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aee59c640 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b7426800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aedd9b640 1 -- 172.21.15.31:0/3768425396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 msgr2=0x5617b7426800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b7426800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aedd9b640 1 -- 172.21.15.31:0/3768425396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b707f070 msgr2=0x5617b74262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b707f070 0x5617b74262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aedd9b640 1 -- 172.21.15.31:0/3768425396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b70ea650 con 0x5617b705e390 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7426d40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40e4e70 tx=0x7f5ae40ef690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaf56d6b8219de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5adf7fe640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5ad80b66b0 0x7f5ad80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5adcff9640 1 -- 172.21.15.31:0/3768425396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40f6070 con 0x5617b705e390 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.143+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5ad80b66b0 0x7f5ad80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.144+0000 7f5adcff9640 1 -- 172.21.15.31:0/3768425396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae40b80a0 con 0x5617b705e390 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.144+0000 7f5adcff9640 1 -- 172.21.15.31:0/3768425396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40acd70 con 0x5617b705e390 2024-09-21T02:55:18.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.144+0000 7f5adf7fe640 1 -- 172.21.15.31:0/119296630 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5ae40e7790 con 0x7f5af01083a0 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.144+0000 7f5afae82740 1 -- 172.21.15.31:0/3768425396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 msgr2=0x5617b7426d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.144+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5ad80b66b0 0x7f5ad80b8bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80098d0 tx=0x7f5ae8009900 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.144+0000 7f5afae82740 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7426d40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40e4e70 tx=0x7f5ae40ef690 comp rx=0 tx=0).stop 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 -- 172.21.15.31:0/3768425396 shutdown_connections 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b7426800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7426d40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 --2- 172.21.15.31:0/3768425396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b707f070 0x5617b74262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 -- 172.21.15.31:0/3768425396 >> 172.21.15.31:0/3768425396 conn(0x5617b7073460 msgr2=0x5617b74241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 -- 172.21.15.31:0/3768425396 shutdown_connections 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 -- 172.21.15.31:0/3768425396 wait complete. 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 -- start start 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 Processor -- start 2024-09-21T02:55:18.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.145+0000 7f5afae82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b74318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7431e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b707f070 0x5617b7432380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5617b71ed700 con 0x5617b70561b0 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5617b7327160 con 0x5617b705e390 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5617b7434aa0 con 0x5617b707f070 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 10 client.0 _ll_register_callbacks cb 0x5617b73409e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b74318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b74318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40714/0 (socket says 172.21.15.31:40714) 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 -- 172.21.15.31:0/956192463 learned_addr learned my addr 172.21.15.31:0/956192463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b707f070 0x5617b7432380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 -- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b707f070 msgr2=0x5617b7432380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7431e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b707f070 0x5617b7432380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 -- 172.21.15.31:0/956192463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 msgr2=0x5617b7431e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7431e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 -- 172.21.15.31:0/956192463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b7354c90 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5aee59c640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b74318e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0010a50 tx=0x7f5ae0010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6233df9b6e6ca0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5ad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/956192463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae001e060 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/956192463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5ae000fb10 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.146+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae40b80a0 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/956192463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ae000fd10 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 10 client.16224 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5617b74407b0 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/956192463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae40b80a0 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/956192463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ae0021420 con 0x5617b70561b0 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 10 client.16224 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 10 client.16224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:55:18.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 20 client.16224 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b703d9f0 0x5617b7443d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 4 client.16224 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.147+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b739edf0 con 0x5617b703d9f0 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.148+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b703d9f0 0x5617b7443d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.148+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b703d9f0 0x5617b7443d30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b2b18ec0836a7cc server_cookie=d0ddb21d523dfbc3 in_seq=0 out_seq=0 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.148+0000 7f5ad5ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/956192463 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617b739edf0 con 0x5617b703d9f0 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5ad5ffb640 10 client.16224 handle_command_reply: tid=0 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 client.16224 shutdown 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 msgr2=0x5617b74318e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b74318e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0010a50 tx=0x7f5ae0010f10 comp rx=0 tx=0).stop 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 shutdown_connections 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b703d9f0 0x5617b7443d30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5ad5ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:18.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b707f070 0x5617b7432380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7431e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 --2- 172.21.15.31:0/956192463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b74318e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 >> 172.21.15.31:0/956192463 conn(0x5617b7073460 msgr2=0x5617b7429450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:18.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 shutdown_connections 2024-09-21T02:55:18.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 1 -- 172.21.15.31:0/956192463 wait complete. 2024-09-21T02:55:18.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.149+0000 7f5afae82740 20 client.16224 trim_cache size 0 max 16384 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 Processor -- start 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 -- start start 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b703d9f0 0x5617b7431da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b7423760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7423ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ad450 con 0x5617b70561b0 2024-09-21T02:55:18.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01aef90 con 0x5617b705e390 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5afae82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01adb20 con 0x5617b703d9f0 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b7423760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b7423760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40720/0 (socket says 172.21.15.31:40720) 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 -- 172.21.15.31:0/1758276770 learned_addr learned my addr 172.21.15.31:0/1758276770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aee59c640 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b703d9f0 0x5617b7431da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7423ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 -- 172.21.15.31:0/1758276770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b703d9f0 msgr2=0x5617b7431da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b703d9f0 0x5617b7431da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 -- 172.21.15.31:0/1758276770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 msgr2=0x5617b7423ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7423ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aedd9b640 1 -- 172.21.15.31:0/1758276770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b743dcc0 con 0x5617b70561b0 2024-09-21T02:55:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.163+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7423ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.164+0000 7f5aedd9b640 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b7423760 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40ad590 tx=0x7f5ae400fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e708686216a459 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.164+0000 7f5ad7fff640 1 -- 172.21.15.31:0/1758276770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4088780 con 0x5617b70561b0 2024-09-21T02:55:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.164+0000 7f5ad7fff640 1 -- 172.21.15.31:0/1758276770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae40efe90 con 0x5617b70561b0 2024-09-21T02:55:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.164+0000 7f5ad7fff640 1 -- 172.21.15.31:0/1758276770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4088970 con 0x5617b70561b0 2024-09-21T02:55:18.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.164+0000 7f5afae82740 1 -- 172.21.15.31:0/1758276770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 msgr2=0x5617b7423760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.164+0000 7f5afae82740 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b7423760 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40ad590 tx=0x7f5ae400fdc0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- 172.21.15.31:0/1758276770 shutdown_connections 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b705e390 0x5617b7423ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b70561b0 0x5617b7423760 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 --2- 172.21.15.31:0/1758276770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b703d9f0 0x5617b7431da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- 172.21.15.31:0/1758276770 >> 172.21.15.31:0/1758276770 conn(0x5617b7073460 msgr2=0x5617b7431690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- 172.21.15.31:0/1758276770 shutdown_connections 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- 172.21.15.31:0/1758276770 wait complete. 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- start start 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 Processor -- start 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b703d9f0 0x5617b7435e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b74363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7436910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ae8c0 con 0x5617b705e390 2024-09-21T02:55:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ae1f0 con 0x5617b703d9f0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ab430 con 0x5617b70561b0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.165+0000 7f5afae82740 10 client.0 _ll_register_callbacks cb 0x7f5ae000a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7436910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b74363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7436910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40728/0 (socket says 172.21.15.31:40728) 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 -- 172.21.15.31:0/2178955196 learned_addr learned my addr 172.21.15.31:0/2178955196 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aee59c640 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b703d9f0 0x5617b7435e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 -- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 msgr2=0x5617b74363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b74363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 -- 172.21.15.31:0/2178955196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b703d9f0 msgr2=0x5617b7435e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b703d9f0 0x5617b7435e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 -- 172.21.15.31:0/2178955196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b74ad8e0 con 0x5617b705e390 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5aeed9d640 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7436910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8031a00 tx=0x7f5ae802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263010dca2865fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5ad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/2178955196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8031e60 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/2178955196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5ae8046030 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.166+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae40efe90 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/2178955196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ae8040a30 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 10 client.16230 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5617b74aeab0 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/2178955196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae40efe90 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/2178955196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ae805f070 con 0x5617b705e390 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 10 client.16230 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 10 client.16230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 20 client.16230 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b707f070 0x5617b74b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 4 client.16230 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617b739a8a0 con 0x5617b707f070 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.167+0000 7f5aee59c640 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b707f070 0x5617b74b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5aee59c640 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b707f070 0x5617b74b12d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de447b144aca85e8 server_cookie=2416a9d90777cc0b in_seq=0 out_seq=0 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5ad5ffb640 10 client.16230 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5ad5ffb640 1 -- 172.21.15.31:0/2178955196 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617b739a8a0 con 0x5617b707f070 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5ad5ffb640 10 client.16230 handle_command_reply: tid=0 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 client.16230 shutdown 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 msgr2=0x5617b7436910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7436910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8031a00 tx=0x7f5ae802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 shutdown_connections 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5617b707f070 0x5617b74b12d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617b70561b0 0x5617b74363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5617b705e390 0x5617b7436910 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 --2- 172.21.15.31:0/2178955196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617b703d9f0 0x5617b7435e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 >> 172.21.15.31:0/2178955196 conn(0x5617b7073460 msgr2=0x5617b74272f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5ad5ffb640 0 client.16230 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 shutdown_connections 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 1 -- 172.21.15.31:0/2178955196 wait complete. 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.168+0000 7f5afae82740 20 client.16230 trim_cache size 0 max 16384 2024-09-21T02:55:18.175 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5ad80b66b0 msgr2=0x7f5ad80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5ad80b66b0 0x7f5ad80b8bc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80098d0 tx=0x7f5ae8009900 comp rx=0 tx=0).stop 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 msgr2=0x7f5af019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 0x7f5af019a9b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c970 tx=0x7f5ae400cd70 comp rx=0 tx=0).stop 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 shutdown_connections 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5ad80b66b0 0x7f5ad80b8bc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5af0108cf0 0x7f5af019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5af01083a0 0x7f5af019a9b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 --2- 172.21.15.31:0/119296630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af0102a60 0x7f5af019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 >> 172.21.15.31:0/119296630 conn(0x7f5af0079730 msgr2=0x7f5af0105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 shutdown_connections 2024-09-21T02:55:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:18.169+0000 7f5af490e640 1 -- 172.21.15.31:0/119296630 wait complete. 2024-09-21T02:55:18.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:55:18.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:18 smithi165 ceph-mon[34818]: pgmap v397: 97 pgs: 97 active+clean; 259 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.5 MiB/s wr, 653 op/s 2024-09-21T02:55:18.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:18 smithi178 ceph-mon[34759]: pgmap v397: 97 pgs: 97 active+clean; 259 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.5 MiB/s wr, 653 op/s 2024-09-21T02:55:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:18 smithi031 ceph-mon[31209]: pgmap v397: 97 pgs: 97 active+clean; 259 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.5 MiB/s wr, 653 op/s 2024-09-21T02:55:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/3737303845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd0108d20 msgr2=0x7f6bd010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/3737303845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd0108d20 0x7f6bd010f550 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009ca0 tx=0x7f6bc802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/3737303845 shutdown_connections 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/3737303845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd01083d0 0x7f6bd01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/3737303845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd0108d20 0x7f6bd010f550 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/3737303845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0102ae0 0x7f6bd0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/3737303845 >> 172.21.15.31:0/3737303845 conn(0x7f6bd00797b0 msgr2=0x7f6bd0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/3737303845 shutdown_connections 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.588+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/3737303845 wait complete. 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 Processor -- start 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 -- start start 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd0102ae0 0x7f6bd019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 0x7f6bd019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0108d20 0x7f6bd019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01129e0 con 0x7f6bd01083d0 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0112860 con 0x7f6bd0108d20 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd6aa7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0112b60 con 0x7f6bd0102ae0 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 0x7f6bd019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 0x7f6bd019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49910/0 (socket says 172.21.15.31:49910) 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 -- 172.21.15.31:0/2976215409 learned_addr learned my addr 172.21.15.31:0/2976215409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd0102ae0 0x7f6bd019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0108d20 0x7f6bd019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 -- 172.21.15.31:0/2976215409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd0102ae0 msgr2=0x7f6bd019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd0102ae0 0x7f6bd019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 -- 172.21.15.31:0/2976215409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0108d20 msgr2=0x7f6bd019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0108d20 0x7f6bd019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.589+0000 7f6bcffff640 1 -- 172.21.15.31:0/2976215409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd01a03d0 con 0x7f6bd01083d0 2024-09-21T02:55:19.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0108d20 0x7f6bd019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 0x7f6bd019aa00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400e9c0 tx=0x7f6bc400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bcdffb640 1 -- 172.21.15.31:0/2976215409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400ccc0 con 0x7f6bd01083d0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bcdffb640 1 -- 172.21.15.31:0/2976215409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bc40054d0 con 0x7f6bd01083d0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd01a06c0 con 0x7f6bd01083d0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bcdffb640 1 -- 172.21.15.31:0/2976215409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4010430 con 0x7f6bd01083d0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.590+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a6e20 con 0x7f6bd01083d0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.593+0000 7f6bdd01b740 1 Processor -- start 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.593+0000 7f6bdd01b740 1 -- start start 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bcdffb640 1 -- 172.21.15.31:0/2976215409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6bc4005020 con 0x7f6bd01083d0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c37800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5582a2badea0 con 0x5582a28671b0 2024-09-21T02:55:19.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5582a2b9d2d0 con 0x5582a286f390 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5582a2ba10e0 con 0x5582a2890070 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c37800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c37800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49924/0 (socket says 172.21.15.31:49924) 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 -- 172.21.15.31:0/2680958135 learned_addr learned my addr 172.21.15.31:0/2680958135 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 -- 172.21.15.31:0/2680958135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 msgr2=0x5582a2c372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c372c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 -- 172.21.15.31:0/2680958135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 msgr2=0x5582a2c37d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c37d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.594+0000 7f6bd4843640 1 -- 172.21.15.31:0/2680958135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582a28fb650 con 0x5582a28671b0 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c37800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000bd50 tx=0x7f6bc00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73df0c08a1dd87a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bcdffb640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6bb80b6680 0x7f6bb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bb6ffd640 1 -- 172.21.15.31:0/2680958135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0005220 con 0x5582a28671b0 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6bb80b6680 0x7f6bb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bb6ffd640 1 -- 172.21.15.31:0/2680958135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bc000be40 con 0x5582a28671b0 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bb6ffd640 1 -- 172.21.15.31:0/2680958135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00117c0 con 0x5582a28671b0 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.595+0000 7f6bcdffb640 1 -- 172.21.15.31:0/2976215409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6bc40e7a60 con 0x7f6bd01083d0 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2680958135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 msgr2=0x5582a2c37800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6bb80b6680 0x7f6bb80b8b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8004af0 tx=0x7f6bc8005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c37800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000bd50 tx=0x7f6bc00107a0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2680958135 shutdown_connections 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c37d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c37800 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2680958135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2680958135 >> 172.21.15.31:0/2680958135 conn(0x5582a2884460 msgr2=0x5582a2c351b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:19.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2680958135 shutdown_connections 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2680958135 wait complete. 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 -- start start 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.596+0000 7f6bdd01b740 1 Processor -- start 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c42f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c4edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5582a29fe700 con 0x5582a286f390 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5582a2b38160 con 0x5582a28671b0 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5582a2c45aa0 con 0x5582a2890070 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bdd01b740 10 client.0 _ll_register_callbacks cb 0x5582a2b519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c42f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49928/0 (socket says 172.21.15.31:49928) 2024-09-21T02:55:19.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 -- 172.21.15.31:0/2263427486 learned_addr learned my addr 172.21.15.31:0/2263427486 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c4edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 -- 172.21.15.31:0/2263427486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 msgr2=0x5582a2c4edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c4edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 -- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 msgr2=0x5582a2c429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bcffff640 1 -- 172.21.15.31:0/2263427486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582a2b65c90 con 0x5582a286f390 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c4edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.597+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c42f10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40e5230 tx=0x7f6bc40aa050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143f22e0a9194cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6baffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6baffff640 1 -- 172.21.15.31:0/2263427486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40ad040 con 0x5582a286f390 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6baffff640 1 -- 172.21.15.31:0/2263427486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6bc40acd40 con 0x5582a286f390 2024-09-21T02:55:19.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582a2bafdf0 con 0x5582a286f390 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6baffff640 1 -- 172.21.15.31:0/2263427486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6bc400ce60 con 0x5582a286f390 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 10 client.16242 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582a2c517e0 con 0x5582a286f390 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6baffff640 1 -- 172.21.15.31:0/2263427486 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5582a2bafdf0 con 0x5582a286f390 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6baffff640 1 -- 172.21.15.31:0/2263427486 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6bc40d3020 con 0x5582a286f390 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 10 client.16242 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 20 client.16242 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a284e9f0 0x5582a2c54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 4 client.16242 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.598+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582a2bab8a0 con 0x5582a284e9f0 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.599+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a284e9f0 0x5582a2c54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.599+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a284e9f0 0x5582a2c54d70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5daa87aed6bb2c43 server_cookie=4815a1b96803fca9 in_seq=0 out_seq=0 2024-09-21T02:55:19.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.599+0000 7f6baffff640 10 client.16242 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6baffff640 1 -- 172.21.15.31:0/2263427486 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582a2bab8a0 con 0x5582a284e9f0 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6baffff640 10 client.16242 handle_command_reply: tid=0 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 client.16242 shutdown 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 msgr2=0x5582a2c42f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c42f10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40e5230 tx=0x7f6bc40aa050 comp rx=0 tx=0).stop 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 shutdown_connections 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a284e9f0 0x5582a2c54d70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c4edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c42f10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2263427486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 >> 172.21.15.31:0/2263427486 conn(0x5582a2884460 msgr2=0x5582a2c3a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6baffff640 0 client.16242 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:19.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.600+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 shutdown_connections 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.601+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2263427486 wait complete. 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.601+0000 7f6bdd01b740 20 client.16242 trim_cache size 0 max 16384 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 Processor -- start 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 -- start start 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c44600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c4ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c3fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01af750 con 0x5582a286f390 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01b1290 con 0x5582a28671b0 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01afe20 con 0x5582a2890070 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c4ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c4ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49930/0 (socket says 172.21.15.31:49930) 2024-09-21T02:55:19.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 -- 172.21.15.31:0/3296144314 learned_addr learned my addr 172.21.15.31:0/3296144314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 -- 172.21.15.31:0/3296144314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 msgr2=0x5582a2c3fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bd5044640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c3fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bd4843640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c44600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c3fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 -- 172.21.15.31:0/3296144314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 msgr2=0x5582a2c44600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c44600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bcffff640 1 -- 172.21.15.31:0/3296144314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582a2c43c90 con 0x5582a286f390 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bd5044640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c3fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.614+0000 7f6bd4843640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c44600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.615+0000 7f6bcffff640 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c4ecc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40ad5d0 tx=0x7f6bc400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bbbef584ff2936 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.615+0000 7f6bb67fc640 1 -- 172.21.15.31:0/3296144314 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40ad040 con 0x5582a286f390 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.615+0000 7f6bb67fc640 1 -- 172.21.15.31:0/3296144314 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bc4005680 con 0x5582a286f390 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.615+0000 7f6bb67fc640 1 -- 172.21.15.31:0/3296144314 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40ad6a0 con 0x5582a286f390 2024-09-21T02:55:19.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 -- 172.21.15.31:0/3296144314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 msgr2=0x5582a2c4ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c4ecc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40ad5d0 tx=0x7f6bc400efd0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 -- 172.21.15.31:0/3296144314 shutdown_connections 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c3fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a286f390 0x5582a2c4ecc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/3296144314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a28671b0 0x5582a2c44600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 -- 172.21.15.31:0/3296144314 >> 172.21.15.31:0/3296144314 conn(0x5582a2884460 msgr2=0x5582a2c43ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 -- 172.21.15.31:0/3296144314 shutdown_connections 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 -- 172.21.15.31:0/3296144314 wait complete. 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 -- start start 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.616+0000 7f6bdd01b740 1 Processor -- start 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c46c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c47180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01b0bc0 con 0x5582a28671b0 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01b04f0 con 0x5582a286f390 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01ad730 con 0x5582a2890070 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 10 client.0 _ll_register_callbacks cb 0x7f6bc40096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c46c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c46c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49936/0 (socket says 172.21.15.31:49936) 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 -- 172.21.15.31:0/2393329309 learned_addr learned my addr 172.21.15.31:0/2393329309 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd5044640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 -- 172.21.15.31:0/2393329309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 msgr2=0x5582a2c476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 -- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 msgr2=0x5582a2c47180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:19.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c47180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 -- 172.21.15.31:0/2393329309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582a2cbe730 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bd4843640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c46c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0005d00 tx=0x7f6bc000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f0e09d24b3d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6baffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6baffff640 1 -- 172.21.15.31:0/2393329309 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0032060 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6baffff640 1 -- 172.21.15.31:0/2393329309 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6bc00126d0 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.617+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bc4005680 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6baffff640 1 -- 172.21.15.31:0/2393329309 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6bc00128d0 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5582a2cbf7d0 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6baffff640 1 -- 172.21.15.31:0/2393329309 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bc4005680 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6baffff640 1 -- 172.21.15.31:0/2393329309 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6bc0011b30 con 0x5582a28671b0 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 10 client.16248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:55:19.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 20 client.16248 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a2823da0 0x5582a2cc1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 4 client.16248 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582a2ba8cd0 con 0x5582a2823da0 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.618+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a2823da0 0x5582a2cc1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6baffff640 10 client.16248 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bcffff640 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a2823da0 0x5582a2cc1c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6a385607887b7ed server_cookie=3ae42741460a9d24 in_seq=0 out_seq=0 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6baffff640 1 -- 172.21.15.31:0/2393329309 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582a2ba8cd0 con 0x5582a2823da0 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6baffff640 10 client.16248 handle_command_reply: tid=0 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 client.16248 shutdown 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 msgr2=0x5582a2c46c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c46c40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0005d00 tx=0x7f6bc000b630 comp rx=0 tx=0).stop 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 shutdown_connections 2024-09-21T02:55:19.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5582a2823da0 0x5582a2cc1c60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6baffff640 0 client.16248 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582a2890070 0x5582a2c476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582a286f390 0x5582a2c47180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 --2- 172.21.15.31:0/2393329309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5582a28671b0 0x5582a2c46c40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 >> 172.21.15.31:0/2393329309 conn(0x5582a2884460 msgr2=0x5582a2c38a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 shutdown_connections 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.619+0000 7f6bdd01b740 1 -- 172.21.15.31:0/2393329309 wait complete. 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bdd01b740 20 client.16248 trim_cache size 0 max 16384 2024-09-21T02:55:19.797 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6bb80b6680 msgr2=0x7f6bb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6bb80b6680 0x7f6bb80b8b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8004af0 tx=0x7f6bc8005c30 comp rx=0 tx=0).stop 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 msgr2=0x7f6bd019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 0x7f6bd019aa00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400e9c0 tx=0x7f6bc400ee80 comp rx=0 tx=0).stop 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 shutdown_connections 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6bb80b6680 0x7f6bb80b8b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6bd0108d20 0x7f6bd019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bd01083d0 0x7f6bd019aa00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 --2- 172.21.15.31:0/2976215409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd0102ae0 0x7f6bd019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 >> 172.21.15.31:0/2976215409 conn(0x7f6bd00797b0 msgr2=0x7f6bd0101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 shutdown_connections 2024-09-21T02:55:19.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:19.620+0000 7f6bd6aa7640 1 -- 172.21.15.31:0/2976215409 wait complete. 2024-09-21T02:55:20.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:20 smithi165 ceph-mon[34818]: pgmap v398: 97 pgs: 97 active+clean; 259 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.7 MiB/s wr, 546 op/s 2024-09-21T02:55:20.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:20 smithi031 ceph-mon[31209]: pgmap v398: 97 pgs: 97 active+clean; 259 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.7 MiB/s wr, 546 op/s 2024-09-21T02:55:20.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:20 smithi178 ceph-mon[34759]: pgmap v398: 97 pgs: 97 active+clean; 259 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.7 MiB/s wr, 546 op/s 2024-09-21T02:55:21.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:55:21.695 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/615143058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a01049d0 msgr2=0x7fb8a0104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/615143058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104de0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb890009900 tx=0x7fb89002a410 comp rx=0 tx=0).stop 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/615143058 shutdown_connections 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/615143058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a0105320 0x7fb8a010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/615143058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a010de30 0x7fb8a0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/615143058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104de0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/615143058 >> 172.21.15.31:0/615143058 conn(0x7fb8a0079740 msgr2=0x7fb8a0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/615143058 shutdown_connections 2024-09-21T02:55:22.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.531+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/615143058 wait complete. 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.532+0000 7fb8a6b1c640 1 Processor -- start 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 -- start start 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a0105320 0x7fb8a0102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 0x7fb8a0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113a90 con 0x7fb8a010de30 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113910 con 0x7fb8a01049d0 2024-09-21T02:55:22.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a6b1c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113c10 con 0x7fb8a0105320 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 0x7fb8a0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a48b8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 0x7fb8a0102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49962/0 (socket says 172.21.15.31:49962) 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb8a50b9640 1 -- 172.21.15.31:0/3823643367 learned_addr learned my addr 172.21.15.31:0/3823643367 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.533+0000 7fb89ffff640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a0105320 0x7fb8a0102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a48b8640 1 -- 172.21.15.31:0/3823643367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a0105320 msgr2=0x7fb8a0102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a48b8640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a0105320 0x7fb8a0102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a48b8640 1 -- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 msgr2=0x7fb8a0102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a48b8640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 0x7fb8a0102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a48b8640 1 -- 172.21.15.31:0/3823643367 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a0103470 con 0x7fb8a01049d0 2024-09-21T02:55:22.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 0x7fb8a0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a48b8640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb890006010 tx=0x7fb890004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb89dffb640 1 -- 172.21.15.31:0/3823643367 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89003f070 con 0x7fb8a01049d0 2024-09-21T02:55:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a01ace20 con 0x7fb8a01049d0 2024-09-21T02:55:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb89dffb640 1 -- 172.21.15.31:0/3823643367 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb890003ca0 con 0x7fb8a01049d0 2024-09-21T02:55:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.534+0000 7fb89dffb640 1 -- 172.21.15.31:0/3823643367 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890035070 con 0x7fb8a01049d0 2024-09-21T02:55:22.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.535+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01ad280 con 0x7fb8a01049d0 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 Processor -- start 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 -- start start 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 0x55f2b2b692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27991b0 0x55f2b2b69800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27a1390 0x55f2b2b69d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2adfea0 con 0x55f2b27c2070 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2acf2d0 con 0x55f2b27991b0 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.537+0000 7fb8ad090740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2ad30e0 con 0x55f2b27a1390 2024-09-21T02:55:22.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 0x55f2b2b692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 0x55f2b2b692c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49970/0 (socket says 172.21.15.31:49970) 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 -- 172.21.15.31:0/4090626197 learned_addr learned my addr 172.21.15.31:0/4090626197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb89ffff640 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27a1390 0x55f2b2b69d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 -- 172.21.15.31:0/4090626197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27a1390 msgr2=0x55f2b2b69d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27a1390 0x55f2b2b69d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 -- 172.21.15.31:0/4090626197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27991b0 msgr2=0x55f2b2b69800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27991b0 0x55f2b2b69800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 -- 172.21.15.31:0/4090626197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b282d650 con 0x55f2b27c2070 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 0x55f2b2b692c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb898011890 tx=0x7fb898011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c2707633c295c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb89dffb640 1 -- 172.21.15.31:0/3823643367 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb89003b460 con 0x7fb8a01049d0 2024-09-21T02:55:22.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb886ffd640 1 -- 172.21.15.31:0/4090626197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898011310 con 0x55f2b27c2070 2024-09-21T02:55:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb886ffd640 1 -- 172.21.15.31:0/4090626197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89800f9a0 con 0x55f2b27c2070 2024-09-21T02:55:22.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.538+0000 7fb886ffd640 1 -- 172.21.15.31:0/4090626197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898005600 con 0x55f2b27c2070 2024-09-21T02:55:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb89dffb640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8880b6680 0x7fb8880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 -- 172.21.15.31:0/4090626197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 msgr2=0x55f2b2b692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 0x55f2b2b692c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb898011890 tx=0x7fb898011d50 comp rx=0 tx=0).stop 2024-09-21T02:55:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8a48b8640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8880b6680 0x7fb8880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 -- 172.21.15.31:0/4090626197 shutdown_connections 2024-09-21T02:55:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27a1390 0x55f2b2b69d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27991b0 0x55f2b2b69800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 --2- 172.21.15.31:0/4090626197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27c2070 0x55f2b2b692c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.539+0000 7fb8ad090740 1 -- 172.21.15.31:0/4090626197 >> 172.21.15.31:0/4090626197 conn(0x55f2b27b6460 msgr2=0x55f2b2b671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.540+0000 7fb8ad090740 1 -- 172.21.15.31:0/4090626197 shutdown_connections 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.540+0000 7fb89dffb640 1 -- 172.21.15.31:0/3823643367 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb890107b70 con 0x7fb8a01049d0 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.540+0000 7fb8ad090740 1 -- 172.21.15.31:0/4090626197 wait complete. 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.540+0000 7fb8a48b8640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8880b6680 0x7fb8880b8b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f2b2b67f40 tx=0x7fb8900ca1b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.540+0000 7fb8ad090740 1 -- start start 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.540+0000 7fb8ad090740 1 Processor -- start 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b74910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b74e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27c2070 0x55f2b2b75390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2930700 con 0x55f2b27991b0 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2a6a160 con 0x55f2b27a1390 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2b2b77aa0 con 0x55f2b27c2070 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8ad090740 10 client.0 _ll_register_callbacks cb 0x55f2b2a839e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b74910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b74910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49984/0 (socket says 172.21.15.31:49984) 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 -- 172.21.15.31:0/1162233557 learned_addr learned my addr 172.21.15.31:0/1162233557 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb89ffff640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b74e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27c2070 0x55f2b2b75390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 -- 172.21.15.31:0/1162233557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27c2070 msgr2=0x55f2b2b75390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27c2070 0x55f2b2b75390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 -- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 msgr2=0x55f2b2b74e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b74e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb8a50b9640 1 -- 172.21.15.31:0/1162233557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b2a97c90 con 0x55f2b27991b0 2024-09-21T02:55:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.541+0000 7fb89ffff640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b74e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b74910 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb898009a70 tx=0x7fb898005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3018479253d8e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb87ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb87ffff640 1 -- 172.21.15.31:0/1162233557 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89802d060 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb87ffff640 1 -- 172.21.15.31:0/1162233557 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb898011490 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb89800f9a0 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb87ffff640 1 -- 172.21.15.31:0/1162233557 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb89801e040 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb8ad090740 10 client.16260 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2b2b83720 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb87ffff640 1 -- 172.21.15.31:0/1162233557 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89800f9a0 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.542+0000 7fb87ffff640 1 -- 172.21.15.31:0/1162233557 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb898018070 con 0x55f2b27991b0 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb8ad090740 10 client.16260 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb8ad090740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb8ad090740 20 client.16260 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb8ad090740 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27809f0 0x55f2b2b86b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb8ad090740 4 client.16260 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b2ae1df0 con 0x55f2b27809f0 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb89ffff640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27809f0 0x55f2b2b86b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb87ffff640 10 client.16260 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.543+0000 7fb89ffff640 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27809f0 0x55f2b2b86b90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e617c3442bcdfbcf server_cookie=28cfb5b996092148 in_seq=0 out_seq=0 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb87ffff640 1 -- 172.21.15.31:0/1162233557 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2b2ae1df0 con 0x55f2b27809f0 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb87ffff640 10 client.16260 handle_command_reply: tid=0 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 client.16260 shutdown 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 msgr2=0x55f2b2b74910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b74910 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb898009a70 tx=0x7fb898005240 comp rx=0 tx=0).stop 2024-09-21T02:55:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 shutdown_connections 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27809f0 0x55f2b2b86b90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb87ffff640 0 client.16260 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27c2070 0x55f2b2b75390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b74e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 --2- 172.21.15.31:0/1162233557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b74910 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.544+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 >> 172.21.15.31:0/1162233557 conn(0x55f2b27b6460 msgr2=0x55f2b2b6c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.545+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 shutdown_connections 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.545+0000 7fb8ad090740 1 -- 172.21.15.31:0/1162233557 wait complete. 2024-09-21T02:55:22.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.545+0000 7fb8ad090740 20 client.16260 trim_cache size 0 max 16384 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 Processor -- start 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 -- start start 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27809f0 0x55f2b2b6acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27991b0 0x55f2b2b76540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 0x55f2b2b66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b5d10 con 0x55f2b27a1390 2024-09-21T02:55:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b7850 con 0x55f2b27809f0 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8ad090740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b63e0 con 0x55f2b27991b0 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 0x55f2b2b66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8a50b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 0x55f2b2b66760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49990/0 (socket says 172.21.15.31:49990) 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.558+0000 7fb8a50b9640 1 -- 172.21.15.31:0/3847547401 learned_addr learned my addr 172.21.15.31:0/3847547401 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27809f0 0x55f2b2b6acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb89ffff640 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27991b0 0x55f2b2b76540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 -- 172.21.15.31:0/3847547401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27991b0 msgr2=0x55f2b2b76540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27991b0 0x55f2b2b76540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 -- 172.21.15.31:0/3847547401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27809f0 msgr2=0x55f2b2b6acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27809f0 0x55f2b2b6acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 -- 172.21.15.31:0/3847547401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b2b80cc0 con 0x55f2b27a1390 2024-09-21T02:55:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 0x55f2b2b66760 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb898005d70 tx=0x7fb898011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acca50dc79def3ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8867fc640 1 -- 172.21.15.31:0/3847547401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89802d040 con 0x55f2b27a1390 2024-09-21T02:55:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8867fc640 1 -- 172.21.15.31:0/3847547401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8980114e0 con 0x55f2b27a1390 2024-09-21T02:55:22.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.559+0000 7fb8867fc640 1 -- 172.21.15.31:0/3847547401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8980052b0 con 0x55f2b27a1390 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 -- 172.21.15.31:0/3847547401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 msgr2=0x55f2b2b66760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 0x55f2b2b66760 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb898005d70 tx=0x7fb898011d50 comp rx=0 tx=0).stop 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 -- 172.21.15.31:0/3847547401 shutdown_connections 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27991b0 0x55f2b2b76540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27a1390 0x55f2b2b66760 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3847547401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27809f0 0x55f2b2b6acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 -- 172.21.15.31:0/3847547401 >> 172.21.15.31:0/3847547401 conn(0x55f2b27b6460 msgr2=0x55f2b2b75e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 -- 172.21.15.31:0/3847547401 shutdown_connections 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 -- 172.21.15.31:0/3847547401 wait complete. 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.560+0000 7fb8ad090740 1 -- start start 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 Processor -- start 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27809f0 0x55f2b2b7d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b7180 con 0x55f2b27991b0 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b6ab0 con 0x55f2b27a1390 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b3cf0 con 0x55f2b27809f0 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8ad090740 10 client.0 _ll_register_callbacks cb 0x7fb898004bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b78a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50006/0 (socket says 172.21.15.31:50006) 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb89ffff640 1 -- 172.21.15.31:0/3277068311 learned_addr learned my addr 172.21.15.31:0/3277068311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27809f0 0x55f2b2b7d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.561+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb89ffff640 1 -- 172.21.15.31:0/3277068311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27809f0 msgr2=0x55f2b2b7d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb89ffff640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27809f0 0x55f2b2b7d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb89ffff640 1 -- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 msgr2=0x55f2b2b79070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb89ffff640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b79070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb89ffff640 1 -- 172.21.15.31:0/3277068311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b2b79850 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27809f0 0x55f2b2b7d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb89ffff640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b78a70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb8a006b6c0 tx=0x7fb8940039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1e4ec2c314100e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb87ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb87ffff640 1 -- 172.21.15.31:0/3277068311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894005220 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb87ffff640 1 -- 172.21.15.31:0/3277068311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb8940053c0 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8980114e0 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb87ffff640 1 -- 172.21.15.31:0/3277068311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8940055c0 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb8ad090740 10 client.16266 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.562+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2b2bf1410 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb87ffff640 1 -- 172.21.15.31:0/3277068311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8980114e0 con 0x55f2b27991b0 2024-09-21T02:55:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb87ffff640 1 -- 172.21.15.31:0/3277068311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb89402b020 con 0x55f2b27991b0 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8ad090740 10 client.16266 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8ad090740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8ad090740 20 client.16266 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27c2070 0x55f2b2bf3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8ad090740 4 client.16266 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2b2add8a0 con 0x55f2b27c2070 2024-09-21T02:55:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.563+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27c2070 0x55f2b2bf3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb87ffff640 10 client.16266 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8a50b9640 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27c2070 0x55f2b2bf3c50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d794a59de1bb274b server_cookie=5d52317209110746 in_seq=0 out_seq=0 2024-09-21T02:55:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb87ffff640 1 -- 172.21.15.31:0/3277068311 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2b2add8a0 con 0x55f2b27c2070 2024-09-21T02:55:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb87ffff640 10 client.16266 handle_command_reply: tid=0 2024-09-21T02:55:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 client.16266 shutdown 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 msgr2=0x55f2b2b78a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b78a70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb8a006b6c0 tx=0x7fb8940039c0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 shutdown_connections 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f2b27c2070 0x55f2b2bf3c50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2b27a1390 0x55f2b2b79070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f2b27991b0 0x55f2b2b78a70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb87ffff640 0 client.16266 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 --2- 172.21.15.31:0/3277068311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2b27809f0 0x55f2b2b7d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 >> 172.21.15.31:0/3277068311 conn(0x55f2b27b6460 msgr2=0x55f2b2b6aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 shutdown_connections 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 1 -- 172.21.15.31:0/3277068311 wait complete. 2024-09-21T02:55:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.564+0000 7fb8ad090740 20 client.16266 trim_cache size 0 max 16384 2024-09-21T02:55:22.571 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:55:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8880b6680 msgr2=0x7fb8880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8880b6680 0x7fb8880b8b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55f2b2b67f40 tx=0x7fb8900ca1b0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a01049d0 msgr2=0x7fb8a0104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb890006010 tx=0x7fb890004060 comp rx=0 tx=0).stop 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 shutdown_connections 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8880b6680 0x7fb8880b8b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8a0105320 0x7fb8a0102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8a010de30 0x7fb8a0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 --2- 172.21.15.31:0/3823643367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a01049d0 0x7fb8a0104120 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 >> 172.21.15.31:0/3823643367 conn(0x7fb8a0079740 msgr2=0x7fb8a01007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 shutdown_connections 2024-09-21T02:55:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:22.565+0000 7fb8a6b1c640 1 -- 172.21.15.31:0/3823643367 wait complete. 2024-09-21T02:55:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:22 smithi165 ceph-mon[34818]: pgmap v399: 97 pgs: 97 active+clean; 262 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 565 op/s 2024-09-21T02:55:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:22 smithi178 ceph-mon[34759]: pgmap v399: 97 pgs: 97 active+clean; 262 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 565 op/s 2024-09-21T02:55:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:22 smithi031 ceph-mon[31209]: pgmap v399: 97 pgs: 97 active+clean; 262 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 565 op/s 2024-09-21T02:55:24.356 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:55:24.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:55:24.629 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:24.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:24 smithi165 ceph-mon[34818]: pgmap v400: 97 pgs: 97 active+clean; 271 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.6 MiB/s wr, 761 op/s 2024-09-21T02:55:24.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:24 smithi031 ceph-mon[31209]: pgmap v400: 97 pgs: 97 active+clean; 271 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.6 MiB/s wr, 761 op/s 2024-09-21T02:55:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:24 smithi178 ceph-mon[34759]: pgmap v400: 97 pgs: 97 active+clean; 271 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.6 MiB/s wr, 761 op/s 2024-09-21T02:55:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 -- 172.21.15.31:0/503682882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4410ddf0 msgr2=0x7f2e44110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 --2- 172.21.15.31:0/503682882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4410ddf0 0x7f2e44110300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009ca0 tx=0x7f2e3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 -- 172.21.15.31:0/503682882 shutdown_connections 2024-09-21T02:55:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 --2- 172.21.15.31:0/503682882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e4410ddf0 0x7f2e44110300 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 --2- 172.21.15.31:0/503682882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 0x7f2e4410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 --2- 172.21.15.31:0/503682882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e44104840 0x7f2e44104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 -- 172.21.15.31:0/503682882 >> 172.21.15.31:0/503682882 conn(0x7f2e44079730 msgr2=0x7f2e44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 -- 172.21.15.31:0/503682882 shutdown_connections 2024-09-21T02:55:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.188+0000 7f2e493f7640 1 -- 172.21.15.31:0/503682882 wait complete. 2024-09-21T02:55:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.189+0000 7f2e493f7640 1 Processor -- start 2024-09-21T02:55:25.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.189+0000 7f2e493f7640 1 -- start start 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e493f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 0x7f2e44102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e493f7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 0x7f2e44102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e493f7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e4410ddf0 0x7f2e441031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e493f7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113a90 con 0x7f2e44104840 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e493f7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113910 con 0x7f2e44105270 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e493f7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113c10 con 0x7f2e4410ddf0 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 0x7f2e44102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 0x7f2e44102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50016/0 (socket says 172.21.15.31:50016) 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 -- 172.21.15.31:0/152292964 learned_addr learned my addr 172.21.15.31:0/152292964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e437fe640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e4410ddf0 0x7f2e441031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e427fc640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 0x7f2e44102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 -- 172.21.15.31:0/152292964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e4410ddf0 msgr2=0x7f2e441031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e4410ddf0 0x7f2e441031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 -- 172.21.15.31:0/152292964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 msgr2=0x7f2e44102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 0x7f2e44102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 -- 172.21.15.31:0/152292964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441a8720 con 0x7f2e44104840 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e427fc640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 0x7f2e44102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 0x7f2e44102750 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9c0 tx=0x7f2e3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.190+0000 7f2e2bfff640 1 -- 172.21.15.31:0/152292964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400cd80 con 0x7f2e44104840 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.191+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441a8a10 con 0x7f2e44104840 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.191+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a8f50 con 0x7f2e44104840 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.191+0000 7f2e2bfff640 1 -- 172.21.15.31:0/152292964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e340035f0 con 0x7f2e44104840 2024-09-21T02:55:25.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.191+0000 7f2e2bfff640 1 -- 172.21.15.31:0/152292964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34010690 con 0x7f2e44104840 2024-09-21T02:55:25.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.193+0000 7f2e2bfff640 1 -- 172.21.15.31:0/152292964 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2e34010830 con 0x7f2e44104840 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.193+0000 7f2e4f96b740 1 Processor -- start 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.193+0000 7f2e4f96b740 1 -- start start 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a1181b5350 0x55a11855c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11818c490 0x55a11855c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11855ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a1184d2e60 con 0x55a118194670 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a1184c2290 con 0x55a1181b5350 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a1184c60a0 con 0x55a11818c490 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11855ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11855ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50032/0 (socket says 172.21.15.31:50032) 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 -- 172.21.15.31:0/4171208399 learned_addr learned my addr 172.21.15.31:0/4171208399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e437fe640 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a1181b5350 0x55a11855c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11818c490 0x55a11855c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 -- 172.21.15.31:0/4171208399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11818c490 msgr2=0x55a11855c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11818c490 0x55a11855c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 -- 172.21.15.31:0/4171208399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a1181b5350 msgr2=0x55a11855c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a1181b5350 0x55a11855c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e427fc640 1 -- 172.21.15.31:0/4171208399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a118220930 con 0x55a118194670 2024-09-21T02:55:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.194+0000 7f2e2bfff640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2e2c0b6700 0x7f2e2c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.195+0000 7f2e427fc640 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11855ceb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bcd0 tx=0x7f2e3800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9f992e478687ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.195+0000 7f2e437fe640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2e2c0b6700 0x7f2e2c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.195+0000 7f2e2bfff640 1 -- 172.21.15.31:0/152292964 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2e340e7cb0 con 0x7f2e44104840 2024-09-21T02:55:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.195+0000 7f2e297fa640 1 -- 172.21.15.31:0/4171208399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38015070 con 0x55a118194670 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.195+0000 7f2e297fa640 1 -- 172.21.15.31:0/4171208399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e380118f0 con 0x55a118194670 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.195+0000 7f2e297fa640 1 -- 172.21.15.31:0/4171208399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38010930 con 0x55a118194670 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 -- 172.21.15.31:0/4171208399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 msgr2=0x55a11855ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11855ceb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bcd0 tx=0x7f2e3800f730 comp rx=0 tx=0).stop 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e437fe640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2e2c0b6700 0x7f2e2c0b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c02f500 tx=0x7f2e3c02f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 -- 172.21.15.31:0/4171208399 shutdown_connections 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a11818c490 0x55a11855c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11855ceb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/4171208399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a1181b5350 0x55a11855c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 -- 172.21.15.31:0/4171208399 >> 172.21.15.31:0/4171208399 conn(0x55a1181a9740 msgr2=0x55a11855a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 -- 172.21.15.31:0/4171208399 shutdown_connections 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.196+0000 7f2e4f96b740 1 -- 172.21.15.31:0/4171208399 wait complete. 2024-09-21T02:55:25.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 -- start start 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 Processor -- start 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118568070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118194670 0x55a118568630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1181b5350 0x55a118573f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a118323890 con 0x55a11818c490 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a11845cd40 con 0x55a118194670 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a11856ac10 con 0x55a1181b5350 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e4f96b740 10 client.0 _ll_register_callbacks cb 0x55a1184c0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e42ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118568070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e42ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118568070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50044/0 (socket says 172.21.15.31:50044) 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.197+0000 7f2e42ffd640 1 -- 172.21.15.31:0/3687359827 learned_addr learned my addr 172.21.15.31:0/3687359827 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118194670 0x55a118568630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e437fe640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1181b5350 0x55a118573f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e42ffd640 1 -- 172.21.15.31:0/3687359827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1181b5350 msgr2=0x55a118573f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1181b5350 0x55a118573f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e42ffd640 1 -- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118194670 msgr2=0x55a118568630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118194670 0x55a118568630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e42ffd640 1 -- 172.21.15.31:0/3687359827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a118574570 con 0x55a11818c490 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118568070 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c6a0 tx=0x7f2e340eb310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cec8026c21dac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3687359827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340ad480 con 0x55a11818c490 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3687359827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2e34010b20 con 0x55a11818c490 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e380118f0 con 0x55a11818c490 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3687359827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e340ad2a0 con 0x55a11818c490 2024-09-21T02:55:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e4f96b740 10 client.16278 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.198+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a118576ee0 con 0x55a11818c490 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3687359827 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e380118f0 con 0x55a11818c490 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3687359827 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2e340b20d0 con 0x55a11818c490 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e4f96b740 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e4f96b740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e4f96b740 20 client.16278 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a118173cd0 0x55a11857a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e4f96b740 4 client.16278 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1184d4db0 con 0x55a118173cd0 2024-09-21T02:55:25.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.199+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a118173cd0 0x55a11857a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.200+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a118173cd0 0x55a11857a2f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d6a7ce8feeff99 server_cookie=b45569ed55e7ed52 in_seq=0 out_seq=0 2024-09-21T02:55:25.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.200+0000 7f2e26ff5640 10 client.16278 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3687359827 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1184d4db0 con 0x55a118173cd0 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e26ff5640 10 client.16278 handle_command_reply: tid=0 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 client.16278 shutdown 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 msgr2=0x55a118568070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118568070 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c6a0 tx=0x7f2e340eb310 comp rx=0 tx=0).stop 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 shutdown_connections 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a118173cd0 0x55a11857a2f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e26ff5640 0 client.16278 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a1181b5350 0x55a118573f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118194670 0x55a118568630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3687359827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118568070 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 >> 172.21.15.31:0/3687359827 conn(0x55a1181a9740 msgr2=0x55a11855f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 shutdown_connections 2024-09-21T02:55:25.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3687359827 wait complete. 2024-09-21T02:55:25.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.201+0000 7f2e4f96b740 20 client.16278 trim_cache size 0 max 16384 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.220+0000 7f2e4f96b740 1 Processor -- start 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.220+0000 7f2e4f96b740 1 -- start start 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118173cd0 0x55a118567700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118569aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118194670 0x55a1185616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b18b0 con 0x55a11818c490 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b33f0 con 0x55a118173cd0 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b1f80 con 0x55a118194670 2024-09-21T02:55:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118569aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118569aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50052/0 (socket says 172.21.15.31:50052) 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 -- 172.21.15.31:0/3253361748 learned_addr learned my addr 172.21.15.31:0/3253361748 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e437fe640 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118194670 0x55a1185616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118173cd0 0x55a118567700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 -- 172.21.15.31:0/3253361748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118194670 msgr2=0x55a1185616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118194670 0x55a1185616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 -- 172.21.15.31:0/3253361748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118173cd0 msgr2=0x55a118567700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118173cd0 0x55a118567700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 -- 172.21.15.31:0/3253361748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a118559a70 con 0x55a11818c490 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118173cd0 0x55a118567700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.221+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118569aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bca0 tx=0x7f2e38010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1049fb6e110c4a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e28ff9640 1 -- 172.21.15.31:0/3253361748 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38015030 con 0x55a11818c490 2024-09-21T02:55:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e28ff9640 1 -- 172.21.15.31:0/3253361748 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e380054f0 con 0x55a11818c490 2024-09-21T02:55:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e28ff9640 1 -- 172.21.15.31:0/3253361748 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004c30 con 0x55a11818c490 2024-09-21T02:55:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3253361748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 msgr2=0x55a118569aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118569aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800bca0 tx=0x7f2e38010470 comp rx=0 tx=0).stop 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3253361748 shutdown_connections 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118194670 0x55a1185616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a11818c490 0x55a118569aa0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.222+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3253361748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a118173cd0 0x55a118567700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3253361748 >> 172.21.15.31:0/3253361748 conn(0x55a1181a9740 msgr2=0x55a118569390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3253361748 shutdown_connections 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3253361748 wait complete. 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- start start 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 Processor -- start 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118173cd0 0x55a118570260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a11818c490 0x55a118570820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11856af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b2d20 con 0x55a118194670 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b2650 con 0x55a11818c490 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441af890 con 0x55a118173cd0 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e4f96b740 10 client.0 _ll_register_callbacks cb 0x7f2e34003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.223+0000 7f2e437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11856af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11856af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50060/0 (socket says 172.21.15.31:50060) 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 -- 172.21.15.31:0/3179256482 learned_addr learned my addr 172.21.15.31:0/3179256482 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e427fc640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a11818c490 0x55a118570820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118173cd0 0x55a118570260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 -- 172.21.15.31:0/3179256482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118173cd0 msgr2=0x55a118570260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118173cd0 0x55a118570260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 -- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a11818c490 msgr2=0x55a118570820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:25.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a11818c490 0x55a118570820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 -- 172.21.15.31:0/3179256482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11856b710 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118173cd0 0x55a118570260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e437fe640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11856af80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c035000 tx=0x7f2e3c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681cfa816c9260f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3179256482 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c059060 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3179256482 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2e3c004040 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.224+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e380054f0 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3179256482 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c004240 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 10 client.16284 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1185e5790 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3179256482 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e380054f0 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3179256482 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2e3c02fd50 con 0x55a118194670 2024-09-21T02:55:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 10 client.16284 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 20 client.16284 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a1181b5350 0x55a1185e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 4 client.16284 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1184d0860 con 0x55a1181b5350 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.225+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a1181b5350 0x55a1185e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.226+0000 7f2e42ffd640 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a1181b5350 0x55a1185e7fb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fea37ea5451ee25 server_cookie=937a64c8c2f65fec in_seq=0 out_seq=0 2024-09-21T02:55:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.226+0000 7f2e26ff5640 10 client.16284 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e26ff5640 1 -- 172.21.15.31:0/3179256482 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1184d0860 con 0x55a1181b5350 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e26ff5640 10 client.16284 handle_command_reply: tid=0 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 client.16284 shutdown 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 msgr2=0x55a11856af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11856af80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c035000 tx=0x7f2e3c0095c0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 shutdown_connections 2024-09-21T02:55:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a1181b5350 0x55a1185e7fb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a118194670 0x55a11856af80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a11818c490 0x55a118570820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 --2- 172.21.15.31:0/3179256482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a118173cd0 0x55a118570260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 >> 172.21.15.31:0/3179256482 conn(0x55a1181a9740 msgr2=0x55a118569ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e26ff5640 0 client.16284 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 shutdown_connections 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 1 -- 172.21.15.31:0/3179256482 wait complete. 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.228+0000 7f2e4f96b740 20 client.16284 trim_cache size 0 max 16384 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:55:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "c5ad05d6-d975-460d-b1d3-f1af6af3b302", 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2e2c0b6700 msgr2=0x7f2e2c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2e2c0b6700 0x7f2e2c0b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c02f500 tx=0x7f2e3c02f310 comp rx=0 tx=0).stop 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 msgr2=0x7f2e44102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 0x7f2e44102750 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9c0 tx=0x7f2e3400ee80 comp rx=0 tx=0).stop 2024-09-21T02:55:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 shutdown_connections 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2e2c0b6700 0x7f2e2c0b8c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e4410ddf0 0x7f2e441031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e44105270 0x7f2e44102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 --2- 172.21.15.31:0/152292964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e44104840 0x7f2e44102750 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 >> 172.21.15.31:0/152292964 conn(0x7f2e44079730 msgr2=0x7f2e44107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 shutdown_connections 2024-09-21T02:55:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:25.229+0000 7f2e493f7640 1 -- 172.21.15.31:0/152292964 wait complete. 2024-09-21T02:55:25.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:55:26.140 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.641+0000 7efdfccfc640 1 -- 172.21.15.31:0/3467472535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf81051c0 msgr2=0x7efdf81055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.641+0000 7efdfccfc640 1 --2- 172.21.15.31:0/3467472535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf81051c0 0x7efdf81055d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec02a410 comp rx=0 tx=0).stop 2024-09-21T02:55:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.641+0000 7efdfccfc640 1 -- 172.21.15.31:0/3467472535 shutdown_connections 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.641+0000 7efdfccfc640 1 --2- 172.21.15.31:0/3467472535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf8105b10 0x7efdf810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.641+0000 7efdfccfc640 1 --2- 172.21.15.31:0/3467472535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf81051c0 0x7efdf81055d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.642+0000 7efdfccfc640 1 --2- 172.21.15.31:0/3467472535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdf81031b0 0x7efdf81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.642+0000 7efdfccfc640 1 -- 172.21.15.31:0/3467472535 >> 172.21.15.31:0/3467472535 conn(0x7efdf8079e40 msgr2=0x7efdf807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.642+0000 7efdfccfc640 1 -- 172.21.15.31:0/3467472535 shutdown_connections 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.642+0000 7efdfccfc640 1 -- 172.21.15.31:0/3467472535 wait complete. 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 Processor -- start 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 -- start start 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf81031b0 0x7efdf819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdf81051c0 0x7efdf819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 0x7efdf819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81129e0 con 0x7efdf8105b10 2024-09-21T02:55:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8112860 con 0x7efdf81051c0 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdfccfc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8112b60 con 0x7efdf81031b0 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdf6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 0x7efdf819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdf6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 0x7efdf819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50088/0 (socket says 172.21.15.31:50088) 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdf6d9d640 1 -- 172.21.15.31:0/2455545112 learned_addr learned my addr 172.21.15.31:0/2455545112 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdf659c640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf81031b0 0x7efdf819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.643+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdf81051c0 0x7efdf819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdf6d9d640 1 -- 172.21.15.31:0/2455545112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf81031b0 msgr2=0x7efdf819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdf6d9d640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf81031b0 0x7efdf819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdf6d9d640 1 -- 172.21.15.31:0/2455545112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdf81051c0 msgr2=0x7efdf819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdf6d9d640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdf81051c0 0x7efdf819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdf6d9d640 1 -- 172.21.15.31:0/2455545112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf81a6570 con 0x7efdf8105b10 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdf6d9d640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 0x7efdf819bd30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7efdf000eaa0 tx=0x7efdf000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efde77fe640 1 -- 172.21.15.31:0/2455545112 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf000cdb0 con 0x7efdf8105b10 2024-09-21T02:55:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efde77fe640 1 -- 172.21.15.31:0/2455545112 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf0005020 con 0x7efdf8105b10 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf81a6860 con 0x7efdf8105b10 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efde77fe640 1 -- 172.21.15.31:0/2455545112 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00116d0 con 0x7efdf8105b10 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.644+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf81a6d20 con 0x7efdf8105b10 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.647+0000 7efe03270740 1 Processor -- start 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.647+0000 7efde77fe640 1 -- 172.21.15.31:0/2455545112 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efdf0004b20 con 0x7efdf8105b10 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.647+0000 7efe03270740 1 -- start start 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efe03270740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e7359e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efe03270740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e7359eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efe03270740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e7359f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efe03270740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561e73514e50 con 0x561e731ce1b0 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efe03270740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e73504280 con 0x561e731d6390 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efe03270740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e73508090 con 0x561e731f7070 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e7359eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e7359eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50098/0 (socket says 172.21.15.31:50098) 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 -- 172.21.15.31:0/2072276949 learned_addr learned my addr 172.21.15.31:0/2072276949 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e7359f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e7359e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efde77fe640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efde00b6680 0x7efde00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 -- 172.21.15.31:0/2072276949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 msgr2=0x561e7359e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e7359e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 -- 172.21.15.31:0/2072276949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 msgr2=0x561e7359f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e7359f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf659c640 1 -- 172.21.15.31:0/2072276949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e73262650 con 0x561e731ce1b0 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e7359e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.648+0000 7efdf6d9d640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efde00b6680 0x7efde00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.649+0000 7efdf659c640 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e7359eb90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efde800abc0 tx=0x7efde800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8c66e1793e83fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.649+0000 7efde77fe640 1 -- 172.21.15.31:0/2455545112 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7efdf00e7940 con 0x7efdf8105b10 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.649+0000 7efde4ff9640 1 -- 172.21.15.31:0/2072276949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde801d070 con 0x561e731ce1b0 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.649+0000 7efde4ff9640 1 -- 172.21.15.31:0/2072276949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8018af0 con 0x561e731ce1b0 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.649+0000 7efde4ff9640 1 -- 172.21.15.31:0/2072276949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8010990 con 0x561e731ce1b0 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 -- 172.21.15.31:0/2072276949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 msgr2=0x561e7359eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e7359eb90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efde800abc0 tx=0x7efde800f790 comp rx=0 tx=0).stop 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efdf6d9d640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efde00b6680 0x7efde00b8b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efdf00e5160 tx=0x7efdf0022f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 -- 172.21.15.31:0/2072276949 shutdown_connections 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e7359f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e7359eb90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 --2- 172.21.15.31:0/2072276949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e7359e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 -- 172.21.15.31:0/2072276949 >> 172.21.15.31:0/2072276949 conn(0x561e731eb460 msgr2=0x561e7359c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 -- 172.21.15.31:0/2072276949 shutdown_connections 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 -- 172.21.15.31:0/2072276949 wait complete. 2024-09-21T02:55:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 -- start start 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.650+0000 7efe03270740 1 Processor -- start 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e735a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 0x561e735aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561e73365700 con 0x561e731d6390 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e7349f110 con 0x561e731ce1b0 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e735ace30 con 0x561e731f7070 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efe03270740 10 client.0 _ll_register_callbacks cb 0x561e734b8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 0x561e735aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 0x561e735aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50108/0 (socket says 172.21.15.31:50108) 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 -- 172.21.15.31:0/317851062 learned_addr learned my addr 172.21.15.31:0/317851062 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e735a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 -- 172.21.15.31:0/317851062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 msgr2=0x561e735b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 -- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 msgr2=0x561e735a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e735a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 -- 172.21.15.31:0/317851062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e734ccc40 con 0x561e731d6390 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdf659c640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 0x561e735aa260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efde8018670 tx=0x7efde80186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698157c72f4fe8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.651+0000 7efdddffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efdddffb640 1 -- 172.21.15.31:0/317851062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8027030 con 0x561e731d6390 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde8018af0 con 0x561e731d6390 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efdddffb640 1 -- 172.21.15.31:0/317851062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7efde8004020 con 0x561e731d6390 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efdddffb640 1 -- 172.21.15.31:0/317851062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efde8031050 con 0x561e731d6390 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 10 client.16296 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e735b8b60 con 0x561e731d6390 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efdddffb640 1 -- 172.21.15.31:0/317851062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8018af0 con 0x561e731d6390 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efdddffb640 1 -- 172.21.15.31:0/317851062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efde8010710 con 0x561e731d6390 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 10 client.16296 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 20 client.16296 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e731b59f0 0x561e735bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 4 client.16296 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.652+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e73516da0 con 0x561e731b59f0 2024-09-21T02:55:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.653+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e731b59f0 0x561e735bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.654+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e731b59f0 0x561e735bbf70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04bce4b207dbae3 server_cookie=ec17c30de7b2505b in_seq=0 out_seq=0 2024-09-21T02:55:26.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.654+0000 7efdddffb640 10 client.16296 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efdddffb640 1 -- 172.21.15.31:0/317851062 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e73516da0 con 0x561e731b59f0 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efdddffb640 10 client.16296 handle_command_reply: tid=0 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 client.16296 shutdown 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 msgr2=0x561e735aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 0x561e735aa260 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efde8018670 tx=0x7efde80186a0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 shutdown_connections 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e731b59f0 0x561e735bbf70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731d6390 0x561e735aa260 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 --2- 172.21.15.31:0/317851062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e735a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efdddffb640 0 client.16296 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:26.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 >> 172.21.15.31:0/317851062 conn(0x561e731eb460 msgr2=0x561e735a17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:26.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 shutdown_connections 2024-09-21T02:55:26.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 1 -- 172.21.15.31:0/317851062 wait complete. 2024-09-21T02:55:26.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.655+0000 7efe03270740 20 client.16296 trim_cache size 0 max 16384 2024-09-21T02:55:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 Processor -- start 2024-09-21T02:55:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 -- start start 2024-09-21T02:55:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e7359fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731d6390 0x561e7359bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 0x561e7359c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81af710 con 0x561e731f7070 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b1250 con 0x561e731ce1b0 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efe03270740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81afde0 con 0x561e731d6390 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efdf5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 0x561e7359c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efdf5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 0x561e7359c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50118/0 (socket says 172.21.15.31:50118) 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3294388042 learned_addr learned my addr 172.21.15.31:0/3294388042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e7359fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.673+0000 7efdf659c640 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731d6390 0x561e7359bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3294388042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731d6390 msgr2=0x561e7359bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731d6390 0x561e7359bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3294388042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 msgr2=0x561e7359fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e7359fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3294388042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e735aadc0 con 0x561e731f7070 2024-09-21T02:55:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 0x561e7359c0b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efdec005940 tx=0x7efdec040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde953ec0b67bbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efddffff640 1 -- 172.21.15.31:0/3294388042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec04c070 con 0x561e731f7070 2024-09-21T02:55:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efddffff640 1 -- 172.21.15.31:0/3294388042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec04a030 con 0x561e731f7070 2024-09-21T02:55:26.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.674+0000 7efddffff640 1 -- 172.21.15.31:0/3294388042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec035040 con 0x561e731f7070 2024-09-21T02:55:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 -- 172.21.15.31:0/3294388042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 msgr2=0x561e7359c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 0x561e7359c0b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efdec005940 tx=0x7efdec040040 comp rx=0 tx=0).stop 2024-09-21T02:55:26.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 -- 172.21.15.31:0/3294388042 shutdown_connections 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731d6390 0x561e7359bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731f7070 0x561e7359c0b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 --2- 172.21.15.31:0/3294388042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731ce1b0 0x561e7359fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 -- 172.21.15.31:0/3294388042 >> 172.21.15.31:0/3294388042 conn(0x561e731eb460 msgr2=0x561e7359f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 -- 172.21.15.31:0/3294388042 shutdown_connections 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 -- 172.21.15.31:0/3294388042 wait complete. 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 -- start start 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.675+0000 7efe03270740 1 Processor -- start 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e735bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e735bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b0b80 con 0x561e731ce1b0 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81b04b0 con 0x561e731d6390 2024-09-21T02:55:26.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81ad6f0 con 0x561e731f7070 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efe03270740 10 client.0 _ll_register_callbacks cb 0x7efde801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e735bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e735bf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50128/0 (socket says 172.21.15.31:50128) 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3665364728 learned_addr learned my addr 172.21.15.31:0/3665364728 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf659c640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e735bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3665364728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 msgr2=0x561e735bcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 msgr2=0x561e735bc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e735bc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 -- 172.21.15.31:0/3665364728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e735bd420 con 0x561e731ce1b0 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf659c640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e735bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdf5d9b640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e735bf4d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efdec004170 tx=0x7efdec004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e205283df91915e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.676+0000 7efdddffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:26.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efdddffb640 1 -- 172.21.15.31:0/3665364728 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec04c070 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efdddffb640 1 -- 172.21.15.31:0/3665364728 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7efdec003a00 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdec04a030 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efdddffb640 1 -- 172.21.15.31:0/3665364728 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdec063050 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 10 client.16302 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e736263c0 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efdddffb640 1 -- 172.21.15.31:0/3665364728 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec04a030 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efdddffb640 1 -- 172.21.15.31:0/3665364728 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efdec03a040 con 0x561e731ce1b0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 10 client.16302 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 20 client.16302 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e7318ada0 0x561e73628b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 4 client.16302 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.677+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e73512850 con 0x561e7318ada0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efdf659c640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e7318ada0 0x561e73628b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efdf659c640 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e7318ada0 0x561e73628b00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a11e036aef7fc8 server_cookie=4c8a909e080ffcc8 in_seq=0 out_seq=0 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efdddffb640 10 client.16302 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efdddffb640 1 -- 172.21.15.31:0/3665364728 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x561e73512850 con 0x561e7318ada0 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efdddffb640 10 client.16302 handle_command_reply: tid=0 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 client.16302 shutdown 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 msgr2=0x561e735bf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e735bf4d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efdec004170 tx=0x7efdec004440 comp rx=0 tx=0).stop 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 shutdown_connections 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561e7318ada0 0x561e73628b00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e731f7070 0x561e735bcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e731d6390 0x561e735bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 --2- 172.21.15.31:0/3665364728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e731ce1b0 0x561e735bf4d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 >> 172.21.15.31:0/3665364728 conn(0x561e731eb460 msgr2=0x561e7359f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.678+0000 7efdddffb640 0 client.16302 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 shutdown_connections 2024-09-21T02:55:26.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efe03270740 1 -- 172.21.15.31:0/3665364728 wait complete. 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efe03270740 20 client.16302 trim_cache size 0 max 16384 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "c5ad05d6-d975-460d-b1d3-f1af6af3b302", 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:26.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.686 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:55:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:26.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:26.689 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efde00b6680 msgr2=0x7efde00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efde00b6680 0x7efde00b8b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efdf00e5160 tx=0x7efdf0022f90 comp rx=0 tx=0).stop 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 msgr2=0x7efdf819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 0x7efdf819bd30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7efdf000eaa0 tx=0x7efdf000ef60 comp rx=0 tx=0).stop 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 shutdown_connections 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efde00b6680 0x7efde00b8b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdf8105b10 0x7efdf819bd30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdf81051c0 0x7efdf819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 --2- 172.21.15.31:0/2455545112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdf81031b0 0x7efdf819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 >> 172.21.15.31:0/2455545112 conn(0x7efdf8079e40 msgr2=0x7efdf8106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 shutdown_connections 2024-09-21T02:55:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:26.679+0000 7efdfccfc640 1 -- 172.21.15.31:0/2455545112 wait complete. 2024-09-21T02:55:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:26 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:55:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:26 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:26 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:55:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:26 smithi031 ceph-mon[31209]: pgmap v401: 97 pgs: 97 active+clean; 295 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-09-21T02:55:26.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:26 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:26 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:55:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:26 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:26 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:55:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:26 smithi178 ceph-mon[34759]: pgmap v401: 97 pgs: 97 active+clean; 295 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-09-21T02:55:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:26 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:26 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:55:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:26 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:27.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:26 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:55:27.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:26 smithi165 ceph-mon[34818]: pgmap v401: 97 pgs: 97 active+clean; 295 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-09-21T02:55:27.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:26 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:55:27.480 INFO:tasks.cephfs.filesystem:scrub status for tag:c5ad05d6-d975-460d-b1d3-f1af6af3b302 - {'path': '/', 'tag': 'c5ad05d6-d975-460d-b1d3-f1af6af3b302', 'options': 'recursive,force'} 2024-09-21T02:55:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:28 smithi178 ceph-mon[34759]: pgmap v402: 97 pgs: 97 active+clean; 289 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-21T02:55:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:28 smithi031 ceph-mon[31209]: pgmap v402: 97 pgs: 97 active+clean; 289 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-21T02:55:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:28 smithi165 ceph-mon[34818]: pgmap v402: 97 pgs: 97 active+clean; 289 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 1.41k op/s 2024-09-21T02:55:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:30 smithi178 ceph-mon[34759]: pgmap v403: 97 pgs: 97 active+clean; 277 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-09-21T02:55:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:30 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:55:31.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:30 smithi031 ceph-mon[31209]: pgmap v403: 97 pgs: 97 active+clean; 277 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-09-21T02:55:31.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:30 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:55:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:30 smithi165 ceph-mon[34818]: pgmap v403: 97 pgs: 97 active+clean; 277 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-09-21T02:55:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:30 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:55:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:32 smithi178 ceph-mon[34759]: pgmap v404: 97 pgs: 97 active+clean; 261 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-09-21T02:55:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:32 smithi031 ceph-mon[31209]: pgmap v404: 97 pgs: 97 active+clean; 261 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-09-21T02:55:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:32 smithi165 ceph-mon[34818]: pgmap v404: 97 pgs: 97 active+clean; 261 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 21 MiB/s wr, 1.36k op/s 2024-09-21T02:55:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:34 smithi178 ceph-mon[34759]: pgmap v405: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-21T02:55:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:34 smithi031 ceph-mon[31209]: pgmap v405: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-21T02:55:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:34 smithi165 ceph-mon[34818]: pgmap v405: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-21T02:55:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:36 smithi178 ceph-mon[34759]: pgmap v406: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 14 MiB/s wr, 920 op/s 2024-09-21T02:55:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:36 smithi031 ceph-mon[31209]: pgmap v406: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 14 MiB/s wr, 920 op/s 2024-09-21T02:55:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:36 smithi165 ceph-mon[34818]: pgmap v406: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 14 MiB/s wr, 920 op/s 2024-09-21T02:55:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:38 smithi178 ceph-mon[34759]: pgmap v407: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 106 KiB/s wr, 38 op/s 2024-09-21T02:55:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:38 smithi031 ceph-mon[31209]: pgmap v407: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 106 KiB/s wr, 38 op/s 2024-09-21T02:55:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:38 smithi165 ceph-mon[34818]: pgmap v407: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 106 KiB/s wr, 38 op/s 2024-09-21T02:55:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:40 smithi178 ceph-mon[34759]: pgmap v408: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 14 op/s 2024-09-21T02:55:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:40 smithi031 ceph-mon[31209]: pgmap v408: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 14 op/s 2024-09-21T02:55:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:40 smithi165 ceph-mon[34818]: pgmap v408: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 14 op/s 2024-09-21T02:55:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:42 smithi178 ceph-mon[34759]: pgmap v409: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-21T02:55:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:42 smithi031 ceph-mon[31209]: pgmap v409: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-21T02:55:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:42 smithi165 ceph-mon[34818]: pgmap v409: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-21T02:55:44.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:44 smithi178 ceph-mon[34759]: pgmap v410: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T02:55:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:44 smithi031 ceph-mon[31209]: pgmap v410: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T02:55:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:44 smithi165 ceph-mon[34818]: pgmap v410: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T02:55:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:46 smithi178 ceph-mon[34759]: pgmap v411: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:55:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:46 smithi031 ceph-mon[31209]: pgmap v411: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:55:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:46 smithi165 ceph-mon[34818]: pgmap v411: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T02:55:48.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:48 smithi178 ceph-mon[34759]: pgmap v412: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:55:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:48 smithi031 ceph-mon[31209]: pgmap v412: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:55:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:48 smithi165 ceph-mon[34818]: pgmap v412: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:55:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:50 smithi031 ceph-mon[31209]: pgmap v413: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:55:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:50 smithi165 ceph-mon[34818]: pgmap v413: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:55:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:50 smithi178 ceph-mon[34759]: pgmap v413: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:55:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:52 smithi031 ceph-mon[31209]: pgmap v414: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:55:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:52 smithi165 ceph-mon[34818]: pgmap v414: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:55:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:52 smithi178 ceph-mon[34759]: pgmap v414: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:55:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:54 smithi031 ceph-mon[31209]: pgmap v415: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:55:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:54 smithi165 ceph-mon[34818]: pgmap v415: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:55:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:54 smithi178 ceph-mon[34759]: pgmap v415: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T02:55:56.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:56 smithi165 ceph-mon[34818]: pgmap v416: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:55:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:56 smithi031 ceph-mon[31209]: pgmap v416: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:55:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:56 smithi178 ceph-mon[34759]: pgmap v416: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:55:57.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:55:57.760 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:58.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.233+0000 7f8014607640 1 -- 172.21.15.31:0/1979454654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c102a60 msgr2=0x7f800c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.233+0000 7f8014607640 1 --2- 172.21.15.31:0/1979454654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c102a60 0x7f800c102e70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009ca0 tx=0x7f7ffc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 -- 172.21.15.31:0/1979454654 shutdown_connections 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 --2- 172.21.15.31:0/1979454654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c1083a0 0x7f800c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 --2- 172.21.15.31:0/1979454654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f800c108cf0 0x7f800c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 --2- 172.21.15.31:0/1979454654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c102a60 0x7f800c102e70 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 -- 172.21.15.31:0/1979454654 >> 172.21.15.31:0/1979454654 conn(0x7f800c079730 msgr2=0x7f800c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 -- 172.21.15.31:0/1979454654 shutdown_connections 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 -- 172.21.15.31:0/1979454654 wait complete. 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 Processor -- start 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.234+0000 7f8014607640 1 -- start start 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8014607640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f800c102a60 0x7f800c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8014607640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 0x7f800c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8014607640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c108cf0 0x7f800c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8014607640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f800c112960 con 0x7f800c1083a0 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8014607640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1127e0 con 0x7f800c102a60 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8014607640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f800c112ae0 con 0x7f800c108cf0 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 0x7f800c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 0x7f800c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52262/0 (socket says 172.21.15.31:52262) 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 -- 172.21.15.31:0/3441842314 learned_addr learned my addr 172.21.15.31:0/3441842314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f80123a3640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f800c102a60 0x7f800c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8012ba4640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c108cf0 0x7f800c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 -- 172.21.15.31:0/3441842314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c108cf0 msgr2=0x7f800c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c108cf0 0x7f800c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 -- 172.21.15.31:0/3441842314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f800c102a60 msgr2=0x7f800c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f800c102a60 0x7f800c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 -- 172.21.15.31:0/3441842314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c1a03d0 con 0x7f800c1083a0 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8012ba4640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c108cf0 0x7f800c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.235+0000 7f8011ba2640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 0x7f800c19aa00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f800000e9c0 tx=0x7f800000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.236+0000 7f800b7fe640 1 -- 172.21.15.31:0/3441842314 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000cd80 con 0x7f800c1083a0 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.236+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c1a06c0 con 0x7f800c1083a0 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.236+0000 7f800b7fe640 1 -- 172.21.15.31:0/3441842314 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80000054d0 con 0x7f800c1083a0 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.236+0000 7f800b7fe640 1 -- 172.21.15.31:0/3441842314 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000010430 con 0x7f800c1083a0 2024-09-21T02:55:58.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.236+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1a6e20 con 0x7f800c1083a0 2024-09-21T02:55:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.238+0000 7f801ab7b740 1 Processor -- start 2024-09-21T02:55:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 -- start start 2024-09-21T02:55:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x5611151d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561115148e50 con 0x561114e0a390 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561115138280 con 0x561114e021b0 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f801ab7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56111513c090 con 0x561114e2b070 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f800b7fe640 1 -- 172.21.15.31:0/3441842314 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8000004fd0 con 0x7f800c1083a0 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f80123a3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f80123a3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:46484/0 (socket says 172.21.15.31:46484) 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f80123a3640 1 -- 172.21.15.31:0/1707339657 learned_addr learned my addr 172.21.15.31:0/1707339657 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f8012ba4640 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x5611151d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.239+0000 7f8011ba2640 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8012ba4640 1 -- 172.21.15.31:0/1707339657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 msgr2=0x5611151d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8012ba4640 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8012ba4640 1 -- 172.21.15.31:0/1707339657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 msgr2=0x5611151d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8012ba4640 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8012ba4640 1 -- 172.21.15.31:0/1707339657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561114e96650 con 0x561114e2b070 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8012ba4640 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x5611151d2650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8004009b40 tx=0x7f800400e2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7382e0d05ce16c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f800b7fe640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ff40b6680 0x7f7ff40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f8008ff9640 1 -- 172.21.15.31:0/1707339657 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004016070 con 0x561114e2b070 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.240+0000 7f80123a3640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ff40b6680 0x7f7ff40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.241+0000 7f8008ff9640 1 -- 172.21.15.31:0/1707339657 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800400ed20 con 0x561114e2b070 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.241+0000 7f8008ff9640 1 -- 172.21.15.31:0/1707339657 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004010790 con 0x561114e2b070 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.241+0000 7f800b7fe640 1 -- 172.21.15.31:0/3441842314 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f80000e7c80 con 0x7f800c1083a0 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f80123a3640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ff40b6680 0x7f7ff40b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc005e30 tx=0x7f7ffc0094a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 -- 172.21.15.31:0/1707339657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 msgr2=0x5611151d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x5611151d2650 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8004009b40 tx=0x7f800400e2f0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 -- 172.21.15.31:0/1707339657 shutdown_connections 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 --2- 172.21.15.31:0/1707339657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x5611151d2650 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 -- 172.21.15.31:0/1707339657 >> 172.21.15.31:0/1707339657 conn(0x561114e1f460 msgr2=0x5611151d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 -- 172.21.15.31:0/1707339657 shutdown_connections 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 -- 172.21.15.31:0/1707339657 wait complete. 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 -- start start 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.242+0000 7f801ab7b740 1 Processor -- start 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e021b0 0x5611151e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561114f99700 con 0x561114e0a390 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5611150d3110 con 0x561114e2b070 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5611151e5160 con 0x561114e021b0 2024-09-21T02:55:58.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f801ab7b740 10 client.0 _ll_register_callbacks cb 0x5611150ec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52282/0 (socket says 172.21.15.31:52282) 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 -- 172.21.15.31:0/766708685 learned_addr learned my addr 172.21.15.31:0/766708685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f80123a3640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e021b0 0x5611151e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8012ba4640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 -- 172.21.15.31:0/766708685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e021b0 msgr2=0x5611151e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e021b0 0x5611151e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 -- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 msgr2=0x5611151e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151e2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.243+0000 7f8011ba2640 1 -- 172.21.15.31:0/766708685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561115100c40 con 0x561114e0a390 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f8011ba2640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e25d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f800000c6a0 tx=0x7f80000d12d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e351354644427b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f7ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:58.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/766708685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000a9990 con 0x561114e0a390 2024-09-21T02:55:58.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/766708685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f80000e5b40 con 0x561114e0a390 2024-09-21T02:55:58.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800400ed20 con 0x561114e0a390 2024-09-21T02:55:58.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/766708685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80000ad260 con 0x561114e0a390 2024-09-21T02:55:58.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f801ab7b740 10 client.16314 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:58.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611151f0ed0 con 0x561114e0a390 2024-09-21T02:55:58.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.244+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/766708685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800400ed20 con 0x561114e0a390 2024-09-21T02:55:58.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/766708685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f80000b2200 con 0x561114e0a390 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f801ab7b740 10 client.16314 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f801ab7b740 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f801ab7b740 20 client.16314 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f801ab7b740 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114de99f0 0x5611151f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f801ab7b740 4 client.16314 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56111514ada0 con 0x561114de99f0 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.245+0000 7f80123a3640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114de99f0 0x5611151f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.246+0000 7f80123a3640 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114de99f0 0x5611151f43f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977f68ee7d88c762 server_cookie=96d5d98a882527cc in_seq=0 out_seq=0 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.246+0000 7f7ff1ffb640 10 client.16314 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:58.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.246+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/766708685 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56111514ada0 con 0x561114de99f0 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.246+0000 7f7ff1ffb640 10 client.16314 handle_command_reply: tid=0 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.246+0000 7f801ab7b740 1 client.16314 shutdown 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.246+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 msgr2=0x5611151e25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e25d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f800000c6a0 tx=0x7f80000d12d0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 shutdown_connections 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114de99f0 0x5611151f43f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e25d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f7ff1ffb640 0 client.16314 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 --2- 172.21.15.31:0/766708685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e021b0 0x5611151e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 >> 172.21.15.31:0/766708685 conn(0x561114e1f460 msgr2=0x5611151d57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 shutdown_connections 2024-09-21T02:55:58.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 1 -- 172.21.15.31:0/766708685 wait complete. 2024-09-21T02:55:58.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.247+0000 7f801ab7b740 20 client.16314 trim_cache size 0 max 16384 2024-09-21T02:55:58.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.264+0000 7f801ab7b740 1 Processor -- start 2024-09-21T02:55:58.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.264+0000 7f801ab7b740 1 -- start start 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 0x5611151e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e0a390 0x5611151d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f801ab7b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ad500 con 0x561114e021b0 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f801ab7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1af040 con 0x561114e2b070 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f801ab7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1adbd0 con 0x561114e0a390 2024-09-21T02:55:58.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 0x5611151e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 0x5611151e3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52288/0 (socket says 172.21.15.31:52288) 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 -- 172.21.15.31:0/913804346 learned_addr learned my addr 172.21.15.31:0/913804346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f8011ba2640 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e0a390 0x5611151d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f8012ba4640 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 -- 172.21.15.31:0/913804346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e0a390 msgr2=0x5611151d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e0a390 0x5611151d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 -- 172.21.15.31:0/913804346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 msgr2=0x5611151df280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151df280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.265+0000 7f80123a3640 1 -- 172.21.15.31:0/913804346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611151e0590 con 0x561114e021b0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.266+0000 7f80123a3640 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 0x5611151e3f70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc006010 tx=0x7f7ffc044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c00ce8d822769c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.266+0000 7f7ff3fff640 1 -- 172.21.15.31:0/913804346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0315d0 con 0x561114e021b0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.266+0000 7f7ff3fff640 1 -- 172.21.15.31:0/913804346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc036040 con 0x561114e021b0 2024-09-21T02:55:58.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.266+0000 7f7ff3fff640 1 -- 172.21.15.31:0/913804346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc004610 con 0x561114e021b0 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 -- 172.21.15.31:0/913804346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 msgr2=0x5611151e3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 0x5611151e3f70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc006010 tx=0x7f7ffc044000 comp rx=0 tx=0).stop 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 -- 172.21.15.31:0/913804346 shutdown_connections 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e0a390 0x5611151d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e2b070 0x5611151df280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 --2- 172.21.15.31:0/913804346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e021b0 0x5611151e3f70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 -- 172.21.15.31:0/913804346 >> 172.21.15.31:0/913804346 conn(0x561114e1f460 msgr2=0x5611151d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 -- 172.21.15.31:0/913804346 shutdown_connections 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 -- 172.21.15.31:0/913804346 wait complete. 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.267+0000 7f801ab7b740 1 -- start start 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 Processor -- start 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x56111525dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ae970 con 0x561114e0a390 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ae2a0 con 0x561114e021b0 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ab4e0 con 0x561114e2b070 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f801ab7b740 10 client.0 _ll_register_callbacks cb 0x7f80000eb4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52298/0 (socket says 172.21.15.31:52298) 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 -- 172.21.15.31:0/3230942873 learned_addr learned my addr 172.21.15.31:0/3230942873 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8012ba4640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x56111525dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f80123a3640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 -- 172.21.15.31:0/3230942873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 msgr2=0x56111525dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x56111525dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 -- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 msgr2=0x5611151e6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151e6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f8011ba2640 1 -- 172.21.15.31:0/3230942873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111525e280 con 0x561114e0a390 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.268+0000 7f80123a3640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f8011ba2640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e6e50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f80000058c0 tx=0x7f80000a9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0200167c9fc6882 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f7ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/3230942873 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000ad440 con 0x561114e0a390 2024-09-21T02:55:58.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/3230942873 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f80000e5b40 con 0x561114e0a390 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ffc036040 con 0x561114e0a390 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/3230942873 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80000ad8c0 con 0x561114e0a390 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f801ab7b740 10 client.16320 fetch_fsmap learned FSMap version 27 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56111525f440 con 0x561114e0a390 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.269+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/3230942873 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc036040 con 0x561114e0a390 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/3230942873 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f80000a9440 con 0x561114e0a390 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f801ab7b740 10 client.16320 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f801ab7b740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f801ab7b740 20 client.16320 populate_metadata read hostname 'smithi031' 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f801ab7b740 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114dbeda0 0x561115261a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f801ab7b740 4 client.16320 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:55:58.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561115146850 con 0x561114dbeda0 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.270+0000 7f80123a3640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114dbeda0 0x561115261a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f7ff1ffb640 10 client.16320 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f80123a3640 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114dbeda0 0x561115261a90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3c53a5be408ac3 server_cookie=3feba5f3900a9966 in_seq=0 out_seq=0 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f7ff1ffb640 1 -- 172.21.15.31:0/3230942873 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x561115146850 con 0x561114dbeda0 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f7ff1ffb640 10 client.16320 handle_command_reply: tid=0 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 client.16320 shutdown 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 msgr2=0x5611151e6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e6e50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f80000058c0 tx=0x7f80000a9f90 comp rx=0 tx=0).stop 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 shutdown_connections 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561114dbeda0 0x561115261a90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561114e2b070 0x56111525dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561114e0a390 0x5611151e6e50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f7ff1ffb640 0 client.16320 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 --2- 172.21.15.31:0/3230942873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561114e021b0 0x5611151e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 >> 172.21.15.31:0/3230942873 conn(0x561114e1f460 msgr2=0x5611151d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 shutdown_connections 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 1 -- 172.21.15.31:0/3230942873 wait complete. 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.271+0000 7f801ab7b740 20 client.16320 trim_cache size 0 max 16384 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:55:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:55:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ff40b6680 msgr2=0x7f7ff40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ff40b6680 0x7f7ff40b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc005e30 tx=0x7f7ffc0094a0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 msgr2=0x7f800c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 0x7f800c19aa00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f800000e9c0 tx=0x7f800000ee80 comp rx=0 tx=0).stop 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 shutdown_connections 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7ff40b6680 0x7f7ff40b8b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f800c108cf0 0x7f800c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c1083a0 0x7f800c19aa00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 --2- 172.21.15.31:0/3441842314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f800c102a60 0x7f800c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 >> 172.21.15.31:0/3441842314 conn(0x7f800c079730 msgr2=0x7f800c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 shutdown_connections 2024-09-21T02:55:58.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:58.272+0000 7f8014607640 1 -- 172.21.15.31:0/3441842314 wait complete. 2024-09-21T02:55:59.029 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:55:58 smithi178 ceph-mon[34759]: pgmap v417: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:55:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:55:58 smithi031 ceph-mon[31209]: pgmap v417: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:55:59.132 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:55:59.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:55:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:55:58 smithi165 ceph-mon[34818]: pgmap v417: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:55:59.394 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.891+0000 7f50d225f640 1 -- 172.21.15.31:0/745137654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc103b60 msgr2=0x7f50cc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.891+0000 7f50d225f640 1 --2- 172.21.15.31:0/745137654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc103b60 0x7f50cc103f70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009900 tx=0x7f50bc02a410 comp rx=0 tx=0).stop 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.891+0000 7f50d225f640 1 -- 172.21.15.31:0/745137654 shutdown_connections 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.891+0000 7f50d225f640 1 --2- 172.21.15.31:0/745137654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50cc1094a0 0x7f50cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 --2- 172.21.15.31:0/745137654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50cc109df0 0x7f50cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 --2- 172.21.15.31:0/745137654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc103b60 0x7f50cc103f70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 -- 172.21.15.31:0/745137654 >> 172.21.15.31:0/745137654 conn(0x7f50cc07b850 msgr2=0x7f50cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 -- 172.21.15.31:0/745137654 shutdown_connections 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 -- 172.21.15.31:0/745137654 wait complete. 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 Processor -- start 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.892+0000 7f50d225f640 1 -- start start 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50d225f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50cc103b60 0x7f50cc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50d225f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50cc1094a0 0x7f50cc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:59.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50d225f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 0x7f50cc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50d225f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc113ab0 con 0x7f50cc109df0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50d225f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc113930 con 0x7f50cc1094a0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50d225f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc113c30 con 0x7f50cc103b60 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 0x7f50cc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 0x7f50cc19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51056/0 (socket says 172.21.15.31:51056) 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 -- 172.21.15.31:0/2855881153 learned_addr learned my addr 172.21.15.31:0/2855881153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50caffd640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50cc1094a0 0x7f50cc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cb7fe640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50cc103b60 0x7f50cc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 -- 172.21.15.31:0/2855881153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50cc103b60 msgr2=0x7f50cc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50cc103b60 0x7f50cc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 -- 172.21.15.31:0/2855881153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50cc1094a0 msgr2=0x7f50cc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50cc1094a0 0x7f50cc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.893+0000 7f50cbfff640 1 -- 172.21.15.31:0/2855881153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc1a0450 con 0x7f50cc109df0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.894+0000 7f50cbfff640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 0x7f50cc19fd20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f50c400eb10 tx=0x7f50c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.894+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4004b20 con 0x7f50cc109df0 2024-09-21T02:55:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.894+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc1a0740 con 0x7f50cc109df0 2024-09-21T02:55:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.894+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c4004cc0 con 0x7f50cc109df0 2024-09-21T02:55:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.894+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4010670 con 0x7f50cc109df0 2024-09-21T02:55:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.894+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc1a6ea0 con 0x7f50cc109df0 2024-09-21T02:55:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.895+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc103f70 con 0x7f50cc109df0 2024-09-21T02:55:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.897+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f50c4005000 con 0x7f50cc109df0 2024-09-21T02:55:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.898+0000 7f50c8ff9640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50b40b6680 0x7f50b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.898+0000 7f50cb7fe640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50b40b6680 0x7f50b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.898+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f50c40e7e50 con 0x7f50cc109df0 2024-09-21T02:55:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.899+0000 7f50cb7fe640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50b40b6680 0x7f50b40b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f50bc031440 tx=0x7f50bc009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:55:59.901+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50c4056710 con 0x7f50cc109df0 2024-09-21T02:56:00.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.128+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50cc19b610 con 0x7f50cc109df0 2024-09-21T02:56:00.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.129+0000 7f50c8ff9640 1 -- 172.21.15.31:0/2855881153 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f50cc19b610 con 0x7f50cc109df0 2024-09-21T02:56:00.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:56:00.131 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:56:00.131 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50b40b6680 msgr2=0x7f50b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50b40b6680 0x7f50b40b8b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f50bc031440 tx=0x7f50bc009560 comp rx=0 tx=0).stop 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 msgr2=0x7f50cc19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 0x7f50cc19fd20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f50c400eb10 tx=0x7f50c400c2d0 comp rx=0 tx=0).stop 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 shutdown_connections 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50b40b6680 0x7f50b40b8b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50cc109df0 0x7f50cc19fd20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50cc1094a0 0x7f50cc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:00.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.131+0000 7f50d225f640 1 --2- 172.21.15.31:0/2855881153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50cc103b60 0x7f50cc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:00.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.132+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 >> 172.21.15.31:0/2855881153 conn(0x7f50cc07b850 msgr2=0x7f50cc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:00.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.132+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 shutdown_connections 2024-09-21T02:56:00.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:00.132+0000 7f50d225f640 1 -- 172.21.15.31:0/2855881153 wait complete. 2024-09-21T02:56:00.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:00 smithi031 ceph-mon[31209]: pgmap v418: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:00.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:00 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2855881153' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:56:00.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:56:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:00 smithi165 ceph-mon[34818]: pgmap v418: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:00 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2855881153' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:56:01.218 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:00 smithi178 ceph-mon[34759]: pgmap v418: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:00 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2855881153' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 -- 172.21.15.31:0/2110842019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 msgr2=0x7f14c0110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 --2- 172.21.15.31:0/2110842019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c0110670 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009ca0 tx=0x7f14b802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 -- 172.21.15.31:0/2110842019 shutdown_connections 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 --2- 172.21.15.31:0/2110842019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c0110670 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 --2- 172.21.15.31:0/2110842019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c01062d0 0x7f14c01066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 --2- 172.21.15.31:0/2110842019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c01020e0 0x7f14c01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 -- 172.21.15.31:0/2110842019 >> 172.21.15.31:0/2110842019 conn(0x7f14c0079e40 msgr2=0x7f14c007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 -- 172.21.15.31:0/2110842019 shutdown_connections 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.686+0000 7f14c617b640 1 -- 172.21.15.31:0/2110842019 wait complete. 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.687+0000 7f14c617b640 1 Processor -- start 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.687+0000 7f14c617b640 1 -- start start 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.687+0000 7f14c617b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c01020e0 0x7f14c019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.687+0000 7f14c617b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c01062d0 0x7f14c019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.687+0000 7f14c617b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14c617b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0113ac0 con 0x7f14c0106c20 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14c617b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0113940 con 0x7f14c01020e0 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14c617b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0113c40 con 0x7f14c01062d0 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51062/0 (socket says 172.21.15.31:51062) 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 -- 172.21.15.31:0/744715025 learned_addr learned my addr 172.21.15.31:0/744715025 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c01020e0 0x7f14c019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14beffd640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c01062d0 0x7f14c019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 -- 172.21.15.31:0/744715025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c01062d0 msgr2=0x7f14c019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c01062d0 0x7f14c019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 -- 172.21.15.31:0/744715025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c01020e0 msgr2=0x7f14c019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c01020e0 0x7f14c019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 -- 172.21.15.31:0/744715025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c01a0340 con 0x7f14c0106c20 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bffff640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c019fc10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14b8005cc0 tx=0x7f14b802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.688+0000 7f14bcff9640 1 -- 172.21.15.31:0/744715025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8003bf0 con 0x7f14c0106c20 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.689+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c01a05d0 con 0x7f14c0106c20 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.689+0000 7f14bcff9640 1 -- 172.21.15.31:0/744715025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b8003d90 con 0x7f14c0106c20 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.689+0000 7f14bcff9640 1 -- 172.21.15.31:0/744715025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b802fbc0 con 0x7f14c0106c20 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.689+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c01a6d90 con 0x7f14c0106c20 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.691+0000 7f14cc6ef740 1 Processor -- start 2024-09-21T02:56:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.691+0000 7f14bcff9640 1 -- 172.21.15.31:0/744715025 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f14b802fd60 con 0x7f14c0106c20 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 -- start start 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc7b070 0x555a4e0226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e022be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e023120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555a4df98ea0 con 0x555a4dc5a390 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555a4df882d0 con 0x555a4dc521b0 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a4df8c0e0 con 0x555a4dc7b070 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e023120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e023120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51078/0 (socket says 172.21.15.31:51078) 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 -- 172.21.15.31:0/1041087185 learned_addr learned my addr 172.21.15.31:0/1041087185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e022be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc7b070 0x555a4e0226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 -- 172.21.15.31:0/1041087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc7b070 msgr2=0x555a4e0226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc7b070 0x555a4e0226a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 -- 172.21.15.31:0/1041087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 msgr2=0x555a4e022be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.692+0000 7f14beffd640 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e022be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14beffd640 1 -- 172.21.15.31:0/1041087185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4dce6650 con 0x555a4dc5a390 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14bcff9640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14a80b6680 0x7f14a80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14a80b6680 0x7f14a80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14beffd640 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e023120 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f14b4011700 tx=0x7f14b400a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a36abf16defc38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14bcff9640 1 -- 172.21.15.31:0/744715025 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f14b8108c20 con 0x7f14c0106c20 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14a5ffb640 1 -- 172.21.15.31:0/1041087185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004020 con 0x555a4dc5a390 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14a5ffb640 1 -- 172.21.15.31:0/1041087185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b40054d0 con 0x555a4dc5a390 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.693+0000 7f14a5ffb640 1 -- 172.21.15.31:0/1041087185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004a30 con 0x555a4dc5a390 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 -- 172.21.15.31:0/1041087185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 msgr2=0x555a4e023120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14a80b6680 0x7f14a80b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x555a4e021320 tx=0x7f14b000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e023120 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f14b4011700 tx=0x7f14b400a680 comp rx=0 tx=0).stop 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 -- 172.21.15.31:0/1041087185 shutdown_connections 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e023120 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e022be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/1041087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc7b070 0x555a4e0226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 -- 172.21.15.31:0/1041087185 >> 172.21.15.31:0/1041087185 conn(0x555a4dc6f460 msgr2=0x555a4e020590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 -- 172.21.15.31:0/1041087185 shutdown_connections 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 -- 172.21.15.31:0/1041087185 wait complete. 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.694+0000 7f14cc6ef740 1 -- start start 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 Processor -- start 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc521b0 0x555a4e0320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e032620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc7b070 0x555a4e032b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555a4dde9700 con 0x555a4dc5a390 2024-09-21T02:56:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555a4df23160 con 0x555a4dc7b070 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555a4e035180 con 0x555a4dc521b0 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14cc6ef740 10 client.0 _ll_register_callbacks cb 0x555a4df3c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e032620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14beffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e032620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51082/0 (socket says 172.21.15.31:51082) 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14beffd640 1 -- 172.21.15.31:0/69097386 learned_addr learned my addr 172.21.15.31:0/69097386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.695+0000 7f14bffff640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc7b070 0x555a4e032b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc521b0 0x555a4e0320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14beffd640 1 -- 172.21.15.31:0/69097386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc521b0 msgr2=0x555a4e0320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14beffd640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc521b0 0x555a4e0320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14beffd640 1 -- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc7b070 msgr2=0x555a4e032b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14beffd640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc7b070 0x555a4e032b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14beffd640 1 -- 172.21.15.31:0/69097386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4df50c90 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14bffff640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc7b070 0x555a4e032b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc521b0 0x555a4e0320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14beffd640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e032620 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f14b40044d0 tx=0x7f14b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb28ff9694db028 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f149effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f149effd640 1 -- 172.21.15.31:0/69097386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400f7c0 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b40054d0 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f149effd640 1 -- 172.21.15.31:0/69097386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f14b400f960 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f149effd640 1 -- 172.21.15.31:0/69097386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14b400fb60 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14cc6ef740 10 client.16338 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a4e040e70 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.696+0000 7f149effd640 1 -- 172.21.15.31:0/69097386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b40054d0 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f149effd640 1 -- 172.21.15.31:0/69097386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f14b4025020 con 0x555a4dc5a390 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14cc6ef740 10 client.16338 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14cc6ef740 10 client.16338 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:56:01.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14cc6ef740 20 client.16338 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc399f0 0x555a4e0441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14cc6ef740 4 client.16338 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a4df9adf0 con 0x555a4dc399f0 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.697+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc399f0 0x555a4e0441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.698+0000 7f149effd640 10 client.16338 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.698+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc399f0 0x555a4e0441d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f673917d60df15b6 server_cookie=be712637f34684b9 in_seq=0 out_seq=0 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.698+0000 7f149effd640 1 -- 172.21.15.31:0/69097386 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a4df9adf0 con 0x555a4dc399f0 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.698+0000 7f149effd640 10 client.16338 handle_command_reply: tid=0 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.698+0000 7f14cc6ef740 1 client.16338 shutdown 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 msgr2=0x555a4e032620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e032620 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f14b40044d0 tx=0x7f14b4004a40 comp rx=0 tx=0).stop 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 shutdown_connections 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc7b070 0x555a4e032b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc5a390 0x555a4e032620 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc399f0 0x555a4e0441d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f149effd640 0 client.16338 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/69097386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc521b0 0x555a4e0320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 >> 172.21.15.31:0/69097386 conn(0x555a4dc6f460 msgr2=0x555a4e01ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 shutdown_connections 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 1 -- 172.21.15.31:0/69097386 wait complete. 2024-09-21T02:56:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.699+0000 7f14cc6ef740 20 client.16338 trim_cache size 0 max 16384 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 Processor -- start 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 -- start start 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e024090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e033d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e02f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01ad450 con 0x555a4dc399f0 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01aef90 con 0x555a4dc521b0 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.712+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01adb20 con 0x555a4dc5a390 2024-09-21T02:56:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e024090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e024090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51090/0 (socket says 172.21.15.31:51090) 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 -- 172.21.15.31:0/166929590 learned_addr learned my addr 172.21.15.31:0/166929590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bffff640 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e02f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14beffd640 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e033d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 -- 172.21.15.31:0/166929590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 msgr2=0x555a4e02f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e02f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 -- 172.21.15.31:0/166929590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 msgr2=0x555a4e033d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e033d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 -- 172.21.15.31:0/166929590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4e034c10 con 0x555a4dc399f0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e024090 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f14b000bff0 tx=0x7f14b0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73bb6ce65d24284 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14a57fa640 1 -- 172.21.15.31:0/166929590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0003cf0 con 0x555a4dc399f0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14a57fa640 1 -- 172.21.15.31:0/166929590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b0003e90 con 0x555a4dc399f0 2024-09-21T02:56:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.713+0000 7f14a57fa640 1 -- 172.21.15.31:0/166929590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0030590 con 0x555a4dc399f0 2024-09-21T02:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 -- 172.21.15.31:0/166929590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 msgr2=0x555a4e024090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e024090 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f14b000bff0 tx=0x7f14b0004610 comp rx=0 tx=0).stop 2024-09-21T02:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 -- 172.21.15.31:0/166929590 shutdown_connections 2024-09-21T02:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e02f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e033d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/166929590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e024090 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 -- 172.21.15.31:0/166929590 >> 172.21.15.31:0/166929590 conn(0x555a4dc6f460 msgr2=0x555a4e033600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 -- 172.21.15.31:0/166929590 shutdown_connections 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.714+0000 7f14cc6ef740 1 -- 172.21.15.31:0/166929590 wait complete. 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 -- start start 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 Processor -- start 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e038430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e038970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01ae8c0 con 0x555a4dc399f0 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01ae1f0 con 0x555a4dc521b0 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14c01ab430 con 0x555a4dc5a390 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14cc6ef740 10 client.0 _ll_register_callbacks cb 0x7f14b000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e038430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e038430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51100/0 (socket says 172.21.15.31:51100) 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.715+0000 7f14bf7fe640 1 -- 172.21.15.31:0/219478987 learned_addr learned my addr 172.21.15.31:0/219478987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bffff640 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bf7fe640 1 -- 172.21.15.31:0/219478987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 msgr2=0x555a4e038eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e038eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bf7fe640 1 -- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 msgr2=0x555a4e038970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e038970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bf7fe640 1 -- 172.21.15.31:0/219478987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4e023490 con 0x555a4dc399f0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14bf7fe640 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e038430 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f14b00039c0 tx=0x7f14b0003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf0cfa0edde2519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f149effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f149effd640 1 -- 172.21.15.31:0/219478987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0005180 con 0x555a4dc399f0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b0003e90 con 0x555a4dc399f0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f149effd640 1 -- 172.21.15.31:0/219478987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f14b0005320 con 0x555a4dc399f0 2024-09-21T02:56:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.716+0000 7f149effd640 1 -- 172.21.15.31:0/219478987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14b00055a0 con 0x555a4dc399f0 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 10 client.16344 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a4e0aeb60 con 0x555a4dc399f0 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f149effd640 1 -- 172.21.15.31:0/219478987 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b0003e90 con 0x555a4dc399f0 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f149effd640 1 -- 172.21.15.31:0/219478987 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f14b0040070 con 0x555a4dc399f0 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 10 client.16344 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 10 client.16344 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 20 client.16344 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc7b070 0x555a4e0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 4 client.16344 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a4df968a0 con 0x555a4dc7b070 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.717+0000 7f14beffd640 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc7b070 0x555a4e0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14beffd640 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc7b070 0x555a4e0b1250 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82f88278069b7093 server_cookie=3ebc631e892e479d in_seq=0 out_seq=0 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f149effd640 10 client.16344 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f149effd640 1 -- 172.21.15.31:0/219478987 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a4df968a0 con 0x555a4dc7b070 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f149effd640 10 client.16344 handle_command_reply: tid=0 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 client.16344 shutdown 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 msgr2=0x555a4e038430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e038430 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f14b00039c0 tx=0x7f14b0003cf0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 shutdown_connections 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x555a4dc7b070 0x555a4e0b1250 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555a4dc5a390 0x555a4e038eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555a4dc521b0 0x555a4e038970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 --2- 172.21.15.31:0/219478987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a4dc399f0 0x555a4e038430 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 >> 172.21.15.31:0/219478987 conn(0x555a4dc6f460 msgr2=0x555a4e023df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f149effd640 0 client.16344 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 shutdown_connections 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 1 -- 172.21.15.31:0/219478987 wait complete. 2024-09-21T02:56:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.718+0000 7f14cc6ef740 20 client.16344 trim_cache size 0 max 16384 2024-09-21T02:56:01.725 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14a80b6680 msgr2=0x7f14a80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14a80b6680 0x7f14a80b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x555a4e021320 tx=0x7f14b000b390 comp rx=0 tx=0).stop 2024-09-21T02:56:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 msgr2=0x7f14c019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c019fc10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14b8005cc0 tx=0x7f14b802f380 comp rx=0 tx=0).stop 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 shutdown_connections 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14a80b6680 0x7f14a80b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14c01062d0 0x7f14c019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c0106c20 0x7f14c019fc10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 --2- 172.21.15.31:0/744715025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14c01020e0 0x7f14c019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 >> 172.21.15.31:0/744715025 conn(0x7f14c0079e40 msgr2=0x7f14c01074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 shutdown_connections 2024-09-21T02:56:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:01.719+0000 7f14c617b640 1 -- 172.21.15.31:0/744715025 wait complete. 2024-09-21T02:56:02.574 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:56:02.832 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:02.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:02 smithi031 ceph-mon[31209]: pgmap v419: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:56:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:02 smithi165 ceph-mon[34818]: pgmap v419: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:56:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:02 smithi178 ceph-mon[34759]: pgmap v419: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:56:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 -- 172.21.15.31:0/2638295567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40001094e0 msgr2=0x7f40001098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 --2- 172.21.15.31:0/2638295567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40001094e0 0x7f40001098f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009900 tx=0x7f3fe802a410 comp rx=0 tx=0).stop 2024-09-21T02:56:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 -- 172.21.15.31:0/2638295567 shutdown_connections 2024-09-21T02:56:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 --2- 172.21.15.31:0/2638295567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000109e30 0x7f4000110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 --2- 172.21.15.31:0/2638295567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40001094e0 0x7f40001098f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 --2- 172.21.15.31:0/2638295567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4000102090 0x7f40001024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 -- 172.21.15.31:0/2638295567 >> 172.21.15.31:0/2638295567 conn(0x7f4000079e30 msgr2=0x7f400007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 -- 172.21.15.31:0/2638295567 shutdown_connections 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 -- 172.21.15.31:0/2638295567 wait complete. 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.305+0000 7f400519a640 1 Processor -- start 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 -- start start 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000102090 0x7f400019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40001094e0 0x7f400019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 0x7f400019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4000113af0 con 0x7f4000109e30 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4000113970 con 0x7f4000102090 2024-09-21T02:56:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f400519a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4000113c70 con 0x7f40001094e0 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3fff59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 0x7f400019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3fff59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 0x7f400019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51110/0 (socket says 172.21.15.31:51110) 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3fff59e640 1 -- 172.21.15.31:0/3611953483 learned_addr learned my addr 172.21.15.31:0/3611953483 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000102090 0x7f400019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40001094e0 0x7f400019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3fff59e640 1 -- 172.21.15.31:0/3611953483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40001094e0 msgr2=0x7f400019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.306+0000 7f3fff59e640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40001094e0 0x7f400019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3fff59e640 1 -- 172.21.15.31:0/3611953483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000102090 msgr2=0x7f400019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3fff59e640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000102090 0x7f400019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3fff59e640 1 -- 172.21.15.31:0/3611953483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40001a03b0 con 0x7f4000109e30 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000102090 0x7f400019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3fff59e640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 0x7f400019fd40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c970 tx=0x7f3ff800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3ff7fff640 1 -- 172.21.15.31:0/3611953483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8007bb0 con 0x7f4000109e30 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3ff7fff640 1 -- 172.21.15.31:0/3611953483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff8007d50 con 0x7f4000109e30 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001a06a0 con 0x7f4000109e30 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f3ff7fff640 1 -- 172.21.15.31:0/3611953483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8004e60 con 0x7f4000109e30 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.307+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001a0b60 con 0x7f4000109e30 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.310+0000 7f400b70e740 1 Processor -- start 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.310+0000 7f3ff7fff640 1 -- 172.21.15.31:0/3611953483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3ff8005020 con 0x7f4000109e30 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 -- start start 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x56432596e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 0x56432596ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432596f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5643258e4ea0 con 0x56432559e1b0 2024-09-21T02:56:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5643258d42d0 con 0x5643255a6390 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f400b70e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5643258d80e0 con 0x5643255c7070 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f3ffed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 0x56432596ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f3ffed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 0x56432596ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51124/0 (socket says 172.21.15.31:51124) 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.311+0000 7f3ffed9d640 1 -- 172.21.15.31:0/930066474 learned_addr learned my addr 172.21.15.31:0/930066474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432596f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x56432596e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffed9d640 1 -- 172.21.15.31:0/930066474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 msgr2=0x56432596e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x56432596e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffed9d640 1 -- 172.21.15.31:0/930066474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 msgr2=0x56432596f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432596f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffed9d640 1 -- 172.21.15.31:0/930066474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564325632650 con 0x56432559e1b0 2024-09-21T02:56:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x56432596e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 0x56432596ebe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0007f70 tx=0x7f3ff000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65487da05f3ee062 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ff7fff640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3fec0b66b0 0x7f3fec0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ff57fa640 1 -- 172.21.15.31:0/930066474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0014070 con 0x56432559e1b0 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3fec0b66b0 0x7f3fec0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ff57fa640 1 -- 172.21.15.31:0/930066474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff0004d20 con 0x56432559e1b0 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ff57fa640 1 -- 172.21.15.31:0/930066474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00054e0 con 0x56432559e1b0 2024-09-21T02:56:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.312+0000 7f3ff7fff640 1 -- 172.21.15.31:0/3611953483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3ff80e7460 con 0x7f4000109e30 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3fec0b66b0 0x7f3fec0b8bc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009230 tx=0x7f3fe8009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 -- 172.21.15.31:0/930066474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 msgr2=0x56432596ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 0x56432596ebe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0007f70 tx=0x7f3ff000e7e0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 -- 172.21.15.31:0/930066474 shutdown_connections 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432596f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56432559e1b0 0x56432596ebe0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 --2- 172.21.15.31:0/930066474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x56432596e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 -- 172.21.15.31:0/930066474 >> 172.21.15.31:0/930066474 conn(0x5643255bb460 msgr2=0x56432596c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:03.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 -- 172.21.15.31:0/930066474 shutdown_connections 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.313+0000 7f400b70e740 1 -- 172.21.15.31:0/930066474 wait complete. 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 -- start start 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 Processor -- start 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x564325979d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432597a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255c7070 0x5643259861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564325735700 con 0x5643255a6390 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56432586f160 con 0x5643255c7070 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.314+0000 7f400b70e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56432597ce80 con 0x56432559e1b0 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f400b70e740 10 client.0 _ll_register_callbacks cb 0x5643258889e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:03.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432597a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432597a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51126/0 (socket says 172.21.15.31:51126) 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 -- 172.21.15.31:0/4137555259 learned_addr learned my addr 172.21.15.31:0/4137555259 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x564325979d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3fff59e640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255c7070 0x5643259861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 -- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 msgr2=0x564325979d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x564325979d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 -- 172.21.15.31:0/4137555259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255c7070 msgr2=0x5643259861d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255c7070 0x5643259861d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 -- 172.21.15.31:0/4137555259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432589cc90 con 0x5643255a6390 2024-09-21T02:56:03.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3fff59e640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255c7070 0x5643259861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432597a2d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005ab0 tx=0x7f3fe8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2e281c81e192be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3fe67fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3fe67fc640 1 -- 172.21.15.31:0/4137555259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe804e070 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f3fe67fc640 1 -- 172.21.15.31:0/4137555259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3fe8003a00 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.315+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff0004d20 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f3fe67fc640 1 -- 172.21.15.31:0/4137555259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fe805a050 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 10 client.16356 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564325988b60 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f3fe67fc640 1 -- 172.21.15.31:0/4137555259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff0004d20 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f3fe67fc640 1 -- 172.21.15.31:0/4137555259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3fe803a040 con 0x5643255a6390 2024-09-21T02:56:03.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 10 client.16356 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 10 client.16356 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 20 client.16356 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5643255859f0 0x56432598c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 4 client.16356 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643258e6df0 con 0x5643255859f0 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.316+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5643255859f0 0x56432598c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.317+0000 7f3fe67fc640 10 client.16356 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.317+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5643255859f0 0x56432598c060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e99acfec3f299ff9 server_cookie=5c0e6b6c7d24f7e3 in_seq=0 out_seq=0 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f3fe67fc640 1 -- 172.21.15.31:0/4137555259 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643258e6df0 con 0x5643255859f0 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f3fe67fc640 10 client.16356 handle_command_reply: tid=0 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 client.16356 shutdown 2024-09-21T02:56:03.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 msgr2=0x56432597a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432597a2d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005ab0 tx=0x7f3fe8031400 comp rx=0 tx=0).stop 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 shutdown_connections 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5643255859f0 0x56432598c060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255c7070 0x5643259861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f3fe67fc640 0 client.16356 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432597a2d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 --2- 172.21.15.31:0/4137555259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x564325979d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 >> 172.21.15.31:0/4137555259 conn(0x5643255bb460 msgr2=0x564325971830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 shutdown_connections 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 1 -- 172.21.15.31:0/4137555259 wait complete. 2024-09-21T02:56:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.318+0000 7f400b70e740 20 client.16356 trim_cache size 0 max 16384 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 Processor -- start 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 -- start start 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56432559e1b0 0x56432597bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432596fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x5643259860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40001af650 con 0x5643255a6390 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40001b1190 con 0x56432559e1b0 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f400b70e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40001afd20 con 0x5643255c7070 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f3ffe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432596fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f3ffe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432596fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51138/0 (socket says 172.21.15.31:51138) 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.332+0000 7f3ffe59c640 1 -- 172.21.15.31:0/1141616438 learned_addr learned my addr 172.21.15.31:0/1141616438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56432559e1b0 0x56432597bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3fff59e640 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x5643259860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffe59c640 1 -- 172.21.15.31:0/1141616438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 msgr2=0x5643259860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x5643259860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffe59c640 1 -- 172.21.15.31:0/1141616438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56432559e1b0 msgr2=0x56432597bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56432559e1b0 0x56432597bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffe59c640 1 -- 172.21.15.31:0/1141616438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432597b050 con 0x5643255a6390 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56432559e1b0 0x56432597bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432596fde0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3fe802a450 tx=0x7f3fe804be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531efb7ae1a5fe6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ff4ff9640 1 -- 172.21.15.31:0/1141616438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8063070 con 0x5643255a6390 2024-09-21T02:56:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ff4ff9640 1 -- 172.21.15.31:0/1141616438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe8004380 con 0x5643255a6390 2024-09-21T02:56:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.333+0000 7f3ff4ff9640 1 -- 172.21.15.31:0/1141616438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe804e070 con 0x5643255a6390 2024-09-21T02:56:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 -- 172.21.15.31:0/1141616438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 msgr2=0x56432596fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432596fde0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3fe802a450 tx=0x7f3fe804be80 comp rx=0 tx=0).stop 2024-09-21T02:56:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 -- 172.21.15.31:0/1141616438 shutdown_connections 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5643255c7070 0x5643259860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255a6390 0x56432596fde0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 --2- 172.21.15.31:0/1141616438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56432559e1b0 0x56432597bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 -- 172.21.15.31:0/1141616438 >> 172.21.15.31:0/1141616438 conn(0x5643255bb460 msgr2=0x56432596f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 -- 172.21.15.31:0/1141616438 shutdown_connections 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.334+0000 7f400b70e740 1 -- 172.21.15.31:0/1141616438 wait complete. 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 -- start start 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 Processor -- start 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x56432597eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432597f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 0x5643259ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40001b0ac0 con 0x5643255c7070 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40001b03f0 con 0x5643255a6390 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40001ad630 con 0x56432559e1b0 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f400b70e740 10 client.0 _ll_register_callbacks cb 0x7f3fe8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f3fff59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 0x5643259ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f3fff59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 0x5643259ef9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51146/0 (socket says 172.21.15.31:51146) 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f3fff59e640 1 -- 172.21.15.31:0/2462041976 learned_addr learned my addr 172.21.15.31:0/2462041976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f3ffe59c640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432597f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x56432597eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.335+0000 7f3fff59e640 1 -- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 msgr2=0x56432597eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fff59e640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x56432597eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fff59e640 1 -- 172.21.15.31:0/2462041976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 msgr2=0x56432597f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fff59e640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432597f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fff59e640 1 -- 172.21.15.31:0/2462041976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643259f0060 con 0x5643255c7070 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fff59e640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 0x5643259ef9f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80aa540 tx=0x7f3ff80e5100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b529ecd2d2b953 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fe67fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fe67fc640 1 -- 172.21.15.31:0/2462041976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801d030 con 0x5643255c7070 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fe67fc640 1 -- 172.21.15.31:0/2462041976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3ff80ac730 con 0x5643255c7070 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8004380 con 0x5643255c7070 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fe67fc640 1 -- 172.21.15.31:0/2462041976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ff80b1db0 con 0x5643255c7070 2024-09-21T02:56:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f400b70e740 10 client.16362 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643259f7430 con 0x5643255c7070 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.336+0000 7f3fe67fc640 1 -- 172.21.15.31:0/2462041976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe8004380 con 0x5643255c7070 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f3fe67fc640 1 -- 172.21.15.31:0/2462041976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3ff80b2fd0 con 0x5643255c7070 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f400b70e740 10 client.16362 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f400b70e740 10 client.16362 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f400b70e740 20 client.16362 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f400b70e740 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56432555ada0 0x5643259f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f400b70e740 4 client.16362 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643258e28a0 con 0x56432555ada0 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56432555ada0 0x5643259f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f3ffed9d640 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56432555ada0 0x5643259f9c10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e16587441b9a01a server_cookie=f9f3b82377b13749 in_seq=0 out_seq=0 2024-09-21T02:56:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f3fe67fc640 10 client.16362 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f3fe67fc640 1 -- 172.21.15.31:0/2462041976 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643258e28a0 con 0x56432555ada0 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.337+0000 7f3fe67fc640 10 client.16362 handle_command_reply: tid=0 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 client.16362 shutdown 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 msgr2=0x5643259ef9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 0x5643259ef9f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80aa540 tx=0x7f3ff80e5100 comp rx=0 tx=0).stop 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 shutdown_connections 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x56432555ada0 0x5643259f9c10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643255c7070 0x5643259ef9f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5643255a6390 0x56432597f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f3fe67fc640 0 client.16362 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 --2- 172.21.15.31:0/2462041976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56432559e1b0 0x56432597eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 >> 172.21.15.31:0/2462041976 conn(0x5643255bb460 msgr2=0x56432597b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 shutdown_connections 2024-09-21T02:56:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 1 -- 172.21.15.31:0/2462041976 wait complete. 2024-09-21T02:56:03.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400b70e740 20 client.16362 trim_cache size 0 max 16384 2024-09-21T02:56:03.346 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3fec0b66b0 msgr2=0x7f3fec0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3fec0b66b0 0x7f3fec0b8bc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009230 tx=0x7f3fe8009680 comp rx=0 tx=0).stop 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 msgr2=0x7f400019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 0x7f400019fd40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c970 tx=0x7f3ff800ce30 comp rx=0 tx=0).stop 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 shutdown_connections 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3fec0b66b0 0x7f3fec0b8bc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40001094e0 0x7f400019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4000109e30 0x7f400019fd40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 --2- 172.21.15.31:0/3611953483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4000102090 0x7f400019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.338+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 >> 172.21.15.31:0/3611953483 conn(0x7f4000079e30 msgr2=0x7f40001075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.339+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 shutdown_connections 2024-09-21T02:56:03.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:03.339+0000 7f400519a640 1 -- 172.21.15.31:0/3611953483 wait complete. 2024-09-21T02:56:04.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:56:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:04.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/447403710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd41083c0 msgr2=0x7fdfd41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/447403710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd41083c0 0x7fdfd41087d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009900 tx=0x7fdfc402a410 comp rx=0 tx=0).stop 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/447403710 shutdown_connections 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/447403710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd4108d10 0x7fdfd410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/447403710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd41083c0 0x7fdfd41087d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/447403710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4102a80 0x7fdfd4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/447403710 >> 172.21.15.31:0/447403710 conn(0x7fdfd4079740 msgr2=0x7fdfd4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/447403710 shutdown_connections 2024-09-21T02:56:04.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.774+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/447403710 wait complete. 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 Processor -- start 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 -- start start 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 0x7fdfd419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd41083c0 0x7fdfd419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4108d10 0x7fdfd419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4112980 con 0x7fdfd4102a80 2024-09-21T02:56:04.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4112800 con 0x7fdfd41083c0 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfdb7d2640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4112b00 con 0x7fdfd4108d10 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 0x7fdfd419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 0x7fdfd419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51162/0 (socket says 172.21.15.31:51162) 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 -- 172.21.15.31:0/1385889989 learned_addr learned my addr 172.21.15.31:0/1385889989 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4108d10 0x7fdfd419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd41083c0 0x7fdfd419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 -- 172.21.15.31:0/1385889989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4108d10 msgr2=0x7fdfd419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4108d10 0x7fdfd419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 -- 172.21.15.31:0/1385889989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd41083c0 msgr2=0x7fdfd419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd41083c0 0x7fdfd419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd956e640 1 -- 172.21.15.31:0/1385889989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41a0410 con 0x7fdfd4102a80 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4108d10 0x7fdfd419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:04.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.775+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd41083c0 0x7fdfd419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:04.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.776+0000 7fdfd956e640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 0x7fdfd419a4a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000e9f0 tx=0x7fdfd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:04.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.776+0000 7fdfca7fc640 1 -- 172.21.15.31:0/1385889989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000cd80 con 0x7fdfd4102a80 2024-09-21T02:56:04.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.776+0000 7fdfca7fc640 1 -- 172.21.15.31:0/1385889989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd00054d0 con 0x7fdfd4102a80 2024-09-21T02:56:04.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.776+0000 7fdfca7fc640 1 -- 172.21.15.31:0/1385889989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0010690 con 0x7fdfd4102a80 2024-09-21T02:56:04.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.776+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41a0700 con 0x7fdfd4102a80 2024-09-21T02:56:04.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.776+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41a6e60 con 0x7fdfd4102a80 2024-09-21T02:56:04.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfca7fc640 1 -- 172.21.15.31:0/1385889989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdfd0004e90 con 0x7fdfd4102a80 2024-09-21T02:56:04.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 Processor -- start 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 -- start start 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf45056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 0x555cf4505be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf4506120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555cf447bea0 con 0x555cf41351b0 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555cf446b2d0 con 0x555cf415e070 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555cf446f0e0 con 0x555cf413d390 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfd956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 0x555cf4505be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfd956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 0x555cf4505be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51164/0 (socket says 172.21.15.31:51164) 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.779+0000 7fdfd956e640 1 -- 172.21.15.31:0/2404033176 learned_addr learned my addr 172.21.15.31:0/2404033176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf45056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf4506120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfca7fc640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdfc00b6700 0x7fdfc00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd956e640 1 -- 172.21.15.31:0/2404033176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 msgr2=0x555cf4506120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf4506120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd956e640 1 -- 172.21.15.31:0/2404033176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 msgr2=0x555cf45056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf45056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd956e640 1 -- 172.21.15.31:0/2404033176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf41c9650 con 0x555cf41351b0 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdfc00b6700 0x7fdfc00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 0x555cf4505be0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00e5290 tx=0x7fdfd0076600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd0804863eeecd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfca7fc640 1 -- 172.21.15.31:0/1385889989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdfd00e7bb0 con 0x7fdfd4102a80 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdfc00b6700 0x7fdfc00b8c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc005fd0 tx=0x7fdfcc004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:04.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.780+0000 7fdfbf7fe640 1 -- 172.21.15.31:0/2404033176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd00ad7f0 con 0x555cf41351b0 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfbf7fe640 1 -- 172.21.15.31:0/2404033176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd00ad530 con 0x555cf41351b0 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfbf7fe640 1 -- 172.21.15.31:0/2404033176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0005180 con 0x555cf41351b0 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2404033176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 msgr2=0x555cf4505be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 0x555cf4505be0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00e5290 tx=0x7fdfd0076600 comp rx=0 tx=0).stop 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2404033176 shutdown_connections 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf4506120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf41351b0 0x555cf4505be0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2404033176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf45056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2404033176 >> 172.21.15.31:0/2404033176 conn(0x555cf4152460 msgr2=0x555cf4503590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:04.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2404033176 shutdown_connections 2024-09-21T02:56:04.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.781+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2404033176 wait complete. 2024-09-21T02:56:04.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 -- start start 2024-09-21T02:56:04.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 Processor -- start 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf4510d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf413d390 0x555cf4511280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf4511800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555cf42cc700 con 0x555cf415e070 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555cf4406160 con 0x555cf413d390 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.782+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555cf4513e20 con 0x555cf41351b0 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfe1d46740 10 client.0 _ll_register_callbacks cb 0x555cf441f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf4511800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf4511800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51166/0 (socket says 172.21.15.31:51166) 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/2067974786 learned_addr learned my addr 172.21.15.31:0/2067974786 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf413d390 0x555cf4511280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf4510d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 msgr2=0x555cf4510d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf4510d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/2067974786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf413d390 msgr2=0x555cf4511280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf413d390 0x555cf4511280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/2067974786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf4433c90 con 0x555cf415e070 2024-09-21T02:56:04.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf4511800 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00f960 tx=0x7fdfcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a1f2159fb35fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:04.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:04.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2067974786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc01ed60 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2067974786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdfcc01b410 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.783+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd00ad530 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2067974786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfcc01b610 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 10 client.16374 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555cf451fb50 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2067974786 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd00ad530 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2067974786 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdfcc02e070 con 0x555cf415e070 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 10 client.16374 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 10 client.16374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 20 client.16374 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf411c9f0 0x555cf4523110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 4 client.16374 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf447ddf0 con 0x555cf411c9f0 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.784+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf411c9f0 0x555cf4523110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.785+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf411c9f0 0x555cf4523110 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5b09f6946a3867f server_cookie=4e125599f31df07 in_seq=0 out_seq=0 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.785+0000 7fdfbcff9640 10 client.16374 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2067974786 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555cf447ddf0 con 0x555cf411c9f0 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfbcff9640 10 client.16374 handle_command_reply: tid=0 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 client.16374 shutdown 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 msgr2=0x555cf4511800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf4511800 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00f960 tx=0x7fdfcc004ab0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 shutdown_connections 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf411c9f0 0x555cf4523110 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf4511800 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf413d390 0x555cf4511280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2067974786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf4510d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 >> 172.21.15.31:0/2067974786 conn(0x555cf4152460 msgr2=0x555cf4508830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfbcff9640 0 client.16374 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 shutdown_connections 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2067974786 wait complete. 2024-09-21T02:56:04.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.786+0000 7fdfe1d46740 20 client.16374 trim_cache size 0 max 16384 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.799+0000 7fdfe1d46740 1 Processor -- start 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.799+0000 7fdfe1d46740 1 -- start start 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf41351b0 0x555cf4512d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf450df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf450e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd41af790 con 0x555cf415e070 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd41b12d0 con 0x555cf41351b0 2024-09-21T02:56:04.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd41afe60 con 0x555cf413d390 2024-09-21T02:56:04.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf450e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf450e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51172/0 (socket says 172.21.15.31:51172) 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/4151306998 learned_addr learned my addr 172.21.15.31:0/4151306998 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf450df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd956e640 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf41351b0 0x555cf4512d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/4151306998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 msgr2=0x555cf450df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf450df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/4151306998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf41351b0 msgr2=0x555cf4512d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf41351b0 0x555cf4512d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 -- 172.21.15.31:0/4151306998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf4503780 con 0x555cf415e070 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd956e640 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf41351b0 0x555cf4512d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf450e470 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009890 tx=0x7fdfcc01e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1807ddd5f8cafc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfbeffd640 1 -- 172.21.15.31:0/4151306998 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc034070 con 0x555cf415e070 2024-09-21T02:56:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.800+0000 7fdfbeffd640 1 -- 172.21.15.31:0/4151306998 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc027d70 con 0x555cf415e070 2024-09-21T02:56:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfbeffd640 1 -- 172.21.15.31:0/4151306998 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc01edc0 con 0x555cf415e070 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 -- 172.21.15.31:0/4151306998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 msgr2=0x555cf450e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf450e470 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009890 tx=0x7fdfcc01e820 comp rx=0 tx=0).stop 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 -- 172.21.15.31:0/4151306998 shutdown_connections 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf413d390 0x555cf450df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf415e070 0x555cf450e470 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/4151306998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf41351b0 0x555cf4512d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 -- 172.21.15.31:0/4151306998 >> 172.21.15.31:0/4151306998 conn(0x555cf4152460 msgr2=0x555cf451c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 -- 172.21.15.31:0/4151306998 shutdown_connections 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.801+0000 7fdfe1d46740 1 -- 172.21.15.31:0/4151306998 wait complete. 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 -- start start 2024-09-21T02:56:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 Processor -- start 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf45153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 0x555cf45158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf458cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd41b0c00 con 0x555cf413d390 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd41b0530 con 0x555cf415e070 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd41ad770 con 0x555cf41351b0 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.802+0000 7fdfe1d46740 10 client.0 _ll_register_callbacks cb 0x7fdfcc00f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:04.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 0x555cf45158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 0x555cf45158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51188/0 (socket says 172.21.15.31:51188) 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 -- 172.21.15.31:0/2525036297 learned_addr learned my addr 172.21.15.31:0/2525036297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd956e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf45153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd9d6f640 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf458cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 -- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 msgr2=0x555cf45153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf45153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 -- 172.21.15.31:0/2525036297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 msgr2=0x555cf458cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf458cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 -- 172.21.15.31:0/2525036297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf45161b0 con 0x555cf413d390 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfd8d6d640 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 0x555cf45158e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406b6c0 tx=0x7fdfc40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b726d738515ee8e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2525036297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4035040 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2525036297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdfc4004240 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.803+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc027d70 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2525036297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfc4004440 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 10 client.16380 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555cf458e0b0 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2525036297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc027d70 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2525036297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdfc4004bc0 con 0x555cf413d390 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 10 client.16380 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 10 client.16380 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 20 client.16380 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf40f1da0 0x555cf45908a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 4 client.16380 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cf44798a0 con 0x555cf40f1da0 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.804+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf40f1da0 0x555cf45908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfd956e640 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf40f1da0 0x555cf45908a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0502bd89d9fcbd6 server_cookie=fcdbceb312536fe5 in_seq=0 out_seq=0 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfbcff9640 10 client.16380 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfbcff9640 1 -- 172.21.15.31:0/2525036297 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cf44798a0 con 0x555cf40f1da0 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfbcff9640 10 client.16380 handle_command_reply: tid=0 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 client.16380 shutdown 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 msgr2=0x555cf45158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 0x555cf45158e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406b6c0 tx=0x7fdfc40316d0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 shutdown_connections 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x555cf40f1da0 0x555cf45908a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555cf415e070 0x555cf458cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555cf413d390 0x555cf45158e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 --2- 172.21.15.31:0/2525036297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555cf41351b0 0x555cf45153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfbcff9640 0 client.16380 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 >> 172.21.15.31:0/2525036297 conn(0x555cf4152460 msgr2=0x555cf451c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 shutdown_connections 2024-09-21T02:56:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 1 -- 172.21.15.31:0/2525036297 wait complete. 2024-09-21T02:56:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.805+0000 7fdfe1d46740 20 client.16380 trim_cache size 0 max 16384 2024-09-21T02:56:04.812 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdfc00b6700 msgr2=0x7fdfc00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdfc00b6700 0x7fdfc00b8c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc005fd0 tx=0x7fdfcc004a40 comp rx=0 tx=0).stop 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 msgr2=0x7fdfd419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 0x7fdfd419a4a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000e9f0 tx=0x7fdfd000edf0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 shutdown_connections 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdfc00b6700 0x7fdfc00b8c10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd4108d10 0x7fdfd419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfd41083c0 0x7fdfd419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 --2- 172.21.15.31:0/1385889989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102a80 0x7fdfd419a4a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 >> 172.21.15.31:0/1385889989 conn(0x7fdfd4079740 msgr2=0x7fdfd406bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:04.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 shutdown_connections 2024-09-21T02:56:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:04.806+0000 7fdfdb7d2640 1 -- 172.21.15.31:0/1385889989 wait complete. 2024-09-21T02:56:05.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:04 smithi031 ceph-mon[31209]: pgmap v420: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:56:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:04 smithi165 ceph-mon[34818]: pgmap v420: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:56:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:04 smithi178 ceph-mon[34759]: pgmap v420: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:56:05.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:56:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 -- 172.21.15.31:0/4248925148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c108d20 msgr2=0x7f1b6c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 --2- 172.21.15.31:0/4248925148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c108d20 0x7f1b6c10f550 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009900 tx=0x7f1b6402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 -- 172.21.15.31:0/4248925148 shutdown_connections 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 --2- 172.21.15.31:0/4248925148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c108d20 0x7f1b6c10f550 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 --2- 172.21.15.31:0/4248925148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b6c1083d0 0x7f1b6c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 --2- 172.21.15.31:0/4248925148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 -- 172.21.15.31:0/4248925148 >> 172.21.15.31:0/4248925148 conn(0x7f1b6c079760 msgr2=0x7f1b6c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.405+0000 7f1b745dc640 1 -- 172.21.15.31:0/4248925148 shutdown_connections 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.406+0000 7f1b745dc640 1 -- 172.21.15.31:0/4248925148 wait complete. 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.406+0000 7f1b745dc640 1 Processor -- start 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.406+0000 7f1b745dc640 1 -- start start 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.406+0000 7f1b745dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.406+0000 7f1b745dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b6c1083d0 0x7f1b6c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b745dc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c108d20 0x7f1b6c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b745dc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c113ab0 con 0x7f1b6c102ae0 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b745dc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c113930 con 0x7f1b6c1083d0 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b745dc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c113c30 con 0x7f1b6c108d20 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51206/0 (socket says 172.21.15.31:51206) 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 -- 172.21.15.31:0/746109021 learned_addr learned my addr 172.21.15.31:0/746109021 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b71b77640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b6c1083d0 0x7f1b6c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72b79640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c108d20 0x7f1b6c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 -- 172.21.15.31:0/746109021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c108d20 msgr2=0x7f1b6c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c108d20 0x7f1b6c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 -- 172.21.15.31:0/746109021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b6c1083d0 msgr2=0x7f1b6c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b6c1083d0 0x7f1b6c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 -- 172.21.15.31:0/746109021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c1a02f0 con 0x7f1b6c102ae0 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72b79640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c108d20 0x7f1b6c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b72378640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c19a440 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c920 tx=0x7f1b6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b5f7fe640 1 -- 172.21.15.31:0/746109021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007c70 con 0x7f1b6c102ae0 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c1a05e0 con 0x7f1b6c102ae0 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b5f7fe640 1 -- 172.21.15.31:0/746109021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b60007e10 con 0x7f1b6c102ae0 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b5f7fe640 1 -- 172.21.15.31:0/746109021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60018820 con 0x7f1b6c102ae0 2024-09-21T02:56:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.407+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c1a6e00 con 0x7f1b6c102ae0 2024-09-21T02:56:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.410+0000 7f1b7ab50740 1 Processor -- start 2024-09-21T02:56:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.410+0000 7f1b7ab50740 1 -- start start 2024-09-21T02:56:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.410+0000 7f1b5f7fe640 1 -- 172.21.15.31:0/746109021 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1b6000f450 con 0x7f1b6c102ae0 2024-09-21T02:56:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f6466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f2761b0 0x55b40f646be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f647120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b40f5bcea0 con 0x55b40f27e390 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b40f5ac2d0 con 0x55b40f29f070 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b40f5b00e0 con 0x55b40f2761b0 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f647120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f647120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51212/0 (socket says 172.21.15.31:51212) 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 -- 172.21.15.31:0/3004935944 learned_addr learned my addr 172.21.15.31:0/3004935944 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b72378640 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f2761b0 0x55b40f646be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b72b79640 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f6466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 -- 172.21.15.31:0/3004935944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f2761b0 msgr2=0x55b40f646be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f2761b0 0x55b40f646be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 -- 172.21.15.31:0/3004935944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 msgr2=0x55b40f6466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f6466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 -- 172.21.15.31:0/3004935944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40f30a650 con 0x55b40f27e390 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.411+0000 7f1b71b77640 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f647120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007df0 tx=0x7f1b6800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d7a2d136a75cef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.412+0000 7f1b5cff9640 1 -- 172.21.15.31:0/3004935944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68014070 con 0x55b40f27e390 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.412+0000 7f1b5f7fe640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1b580b66b0 0x7f1b580b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.412+0000 7f1b5cff9640 1 -- 172.21.15.31:0/3004935944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b68003e20 con 0x55b40f27e390 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.412+0000 7f1b5cff9640 1 -- 172.21.15.31:0/3004935944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68005020 con 0x55b40f27e390 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.412+0000 7f1b72b79640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1b580b66b0 0x7f1b580b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.412+0000 7f1b5f7fe640 1 -- 172.21.15.31:0/746109021 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1b600e7900 con 0x7f1b6c102ae0 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 -- 172.21.15.31:0/3004935944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 msgr2=0x55b40f647120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b72b79640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1b580b66b0 0x7f1b580b8bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b64031440 tx=0x7f1b64032b20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f647120 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007df0 tx=0x7f1b6800e720 comp rx=0 tx=0).stop 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 -- 172.21.15.31:0/3004935944 shutdown_connections 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f2761b0 0x55b40f646be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f647120 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/3004935944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f6466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 -- 172.21.15.31:0/3004935944 >> 172.21.15.31:0/3004935944 conn(0x55b40f293460 msgr2=0x55b40f644590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 -- 172.21.15.31:0/3004935944 shutdown_connections 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 -- 172.21.15.31:0/3004935944 wait complete. 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 -- start start 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.413+0000 7f1b7ab50740 1 Processor -- start 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f651c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f6521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f29f070 0x55b40f6526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b40f40d700 con 0x55b40f27e390 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b40f547160 con 0x55b40f2761b0 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b40f654e80 con 0x55b40f29f070 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b7ab50740 10 client.0 _ll_register_callbacks cb 0x55b40f5609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f6521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f6521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51216/0 (socket says 172.21.15.31:51216) 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 -- 172.21.15.31:0/479470691 learned_addr learned my addr 172.21.15.31:0/479470691 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b72378640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f651c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b72b79640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f29f070 0x55b40f6526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 -- 172.21.15.31:0/479470691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f29f070 msgr2=0x55b40f6526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f29f070 0x55b40f6526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 -- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 msgr2=0x55b40f651c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f651c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 -- 172.21.15.31:0/479470691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40f574c90 con 0x55b40f27e390 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b71b77640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f6521a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1b680045a0 tx=0x7f1b68004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc462bbed4c89a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.414+0000 7f1b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b55ffb640 1 -- 172.21.15.31:0/479470691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68027060 con 0x55b40f27e390 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b55ffb640 1 -- 172.21.15.31:0/479470691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1b68003b80 con 0x55b40f27e390 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b68003e20 con 0x55b40f27e390 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b55ffb640 1 -- 172.21.15.31:0/479470691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b68014070 con 0x55b40f27e390 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 10 client.16392 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b40f660a90 con 0x55b40f27e390 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b55ffb640 1 -- 172.21.15.31:0/479470691 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b68003e20 con 0x55b40f27e390 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b55ffb640 1 -- 172.21.15.31:0/479470691 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1b680224a0 con 0x55b40f27e390 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 10 client.16392 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 10 client.16392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 20 client.16392 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f25d9f0 0x55b40f663ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 4 client.16392 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.415+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b40f5bedf0 con 0x55b40f25d9f0 2024-09-21T02:56:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.416+0000 7f1b72378640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f25d9f0 0x55b40f663ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.416+0000 7f1b55ffb640 10 client.16392 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.416+0000 7f1b72378640 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f25d9f0 0x55b40f663ea0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4036ca9099b4dd0 server_cookie=a2cf1482075d27c5 in_seq=0 out_seq=0 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b55ffb640 1 -- 172.21.15.31:0/479470691 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b40f5bedf0 con 0x55b40f25d9f0 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b55ffb640 10 client.16392 handle_command_reply: tid=0 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 client.16392 shutdown 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 msgr2=0x55b40f6521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f6521a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1b680045a0 tx=0x7f1b68004230 comp rx=0 tx=0).stop 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 shutdown_connections 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f25d9f0 0x55b40f663ea0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f29f070 0x55b40f6526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f27e390 0x55b40f6521a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/479470691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f651c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 >> 172.21.15.31:0/479470691 conn(0x55b40f293460 msgr2=0x55b40f649880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b55ffb640 0 client.16392 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 shutdown_connections 2024-09-21T02:56:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 1 -- 172.21.15.31:0/479470691 wait complete. 2024-09-21T02:56:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.417+0000 7f1b7ab50740 20 client.16392 trim_cache size 0 max 16384 2024-09-21T02:56:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 Processor -- start 2024-09-21T02:56:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 -- start start 2024-09-21T02:56:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 0x55b40f653890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f64ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f64f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1af630 con 0x55b40f2761b0 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1b1170 con 0x55b40f29f070 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1afd00 con 0x55b40f27e390 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 0x55b40f653890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 0x55b40f653890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51232/0 (socket says 172.21.15.31:51232) 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 -- 172.21.15.31:0/859158810 learned_addr learned my addr 172.21.15.31:0/859158810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b71b77640 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f64ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72b79640 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f64f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 -- 172.21.15.31:0/859158810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 msgr2=0x55b40f64ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f64ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 -- 172.21.15.31:0/859158810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 msgr2=0x55b40f64f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f64f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72378640 1 -- 172.21.15.31:0/859158810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40f654d60 con 0x55b40f2761b0 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.431+0000 7f1b72b79640 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f64f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b72378640 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 0x55b40f653890 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1b60007590 tx=0x7f1b600e7b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb92e3dce32704c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b57fff640 1 -- 172.21.15.31:0/859158810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b600acee0 con 0x55b40f2761b0 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b57fff640 1 -- 172.21.15.31:0/859158810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b600ad280 con 0x55b40f2761b0 2024-09-21T02:56:06.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b57fff640 1 -- 172.21.15.31:0/859158810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b600a99e0 con 0x55b40f2761b0 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b7ab50740 1 -- 172.21.15.31:0/859158810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 msgr2=0x55b40f653890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 0x55b40f653890 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1b60007590 tx=0x7f1b600e7b00 comp rx=0 tx=0).stop 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b7ab50740 1 -- 172.21.15.31:0/859158810 shutdown_connections 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f64ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f29f070 0x55b40f64f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.432+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/859158810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f2761b0 0x55b40f653890 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- 172.21.15.31:0/859158810 >> 172.21.15.31:0/859158810 conn(0x55b40f293460 msgr2=0x55b40f653180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:06.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- 172.21.15.31:0/859158810 shutdown_connections 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- 172.21.15.31:0/859158810 wait complete. 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- start start 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 Processor -- start 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f6673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f664880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 0x55b40f664dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1b0aa0 con 0x55b40f29f070 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1b03d0 con 0x55b40f2761b0 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1ad610 con 0x55b40f27e390 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.433+0000 7f1b7ab50740 10 client.0 _ll_register_callbacks cb 0x7f1b600ed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 0x55b40f664dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 0x55b40f664dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51248/0 (socket says 172.21.15.31:51248) 2024-09-21T02:56:06.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 -- 172.21.15.31:0/2883234659 learned_addr learned my addr 172.21.15.31:0/2883234659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72378640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f6673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b71b77640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f664880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 -- 172.21.15.31:0/2883234659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 msgr2=0x55b40f664880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f664880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 -- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 msgr2=0x55b40f6673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f6673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 -- 172.21.15.31:0/2883234659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40f665300 con 0x55b40f29f070 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b72b79640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 0x55b40f664dc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b640319f0 tx=0x7f1b6402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92de6ac8451be80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b55ffb640 1 -- 172.21.15.31:0/2883234659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6404d070 con 0x55b40f29f070 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b55ffb640 1 -- 172.21.15.31:0/2883234659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1b640053d0 con 0x55b40f29f070 2024-09-21T02:56:06.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.434+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b600ad280 con 0x55b40f29f070 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b55ffb640 1 -- 172.21.15.31:0/2883234659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b64059050 con 0x55b40f29f070 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 10 client.16398 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b40f6d25e0 con 0x55b40f29f070 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b55ffb640 1 -- 172.21.15.31:0/2883234659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b600ad280 con 0x55b40f29f070 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b55ffb640 1 -- 172.21.15.31:0/2883234659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1b64003710 con 0x55b40f29f070 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 10 client.16398 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 10 client.16398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 20 client.16398 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f232da0 0x55b40f6d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 4 client.16398 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b40f5ba8a0 con 0x55b40f232da0 2024-09-21T02:56:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.435+0000 7f1b72378640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f232da0 0x55b40f6d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b72378640 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f232da0 0x55b40f6d4dd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1bb36122ccca14b server_cookie=4a215ce2802e139c in_seq=0 out_seq=0 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b55ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b55ffb640 1 -- 172.21.15.31:0/2883234659 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b40f5ba8a0 con 0x55b40f232da0 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b55ffb640 10 client.16398 handle_command_reply: tid=0 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 client.16398 shutdown 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 msgr2=0x55b40f664dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 0x55b40f664dc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b640319f0 tx=0x7f1b6402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 shutdown_connections 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55b40f232da0 0x55b40f6d4dd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b40f27e390 0x55b40f664880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b55ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b40f29f070 0x55b40f664dc0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 --2- 172.21.15.31:0/2883234659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b40f2761b0 0x55b40f6673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 >> 172.21.15.31:0/2883234659 conn(0x55b40f293460 msgr2=0x55b40f6476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:06.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 shutdown_connections 2024-09-21T02:56:06.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 1 -- 172.21.15.31:0/2883234659 wait complete. 2024-09-21T02:56:06.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.436+0000 7f1b7ab50740 20 client.16398 trim_cache size 0 max 16384 2024-09-21T02:56:06.444 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1b580b66b0 msgr2=0x7f1b580b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1b580b66b0 0x7f1b580b8bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b64031440 tx=0x7f1b64032b20 comp rx=0 tx=0).stop 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 msgr2=0x7f1b6c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c19a440 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c920 tx=0x7f1b6000cde0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 shutdown_connections 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1b580b66b0 0x7f1b580b8bc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b6c108d20 0x7f1b6c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b6c1083d0 0x7f1b6c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 --2- 172.21.15.31:0/746109021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b6c102ae0 0x7f1b6c19a440 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:06.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 >> 172.21.15.31:0/746109021 conn(0x7f1b6c079760 msgr2=0x7f1b6c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:06.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 shutdown_connections 2024-09-21T02:56:06.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:06.437+0000 7f1b745dc640 1 -- 172.21.15.31:0/746109021 wait complete. 2024-09-21T02:56:07.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:06 smithi031 ceph-mon[31209]: pgmap v421: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:06 smithi165 ceph-mon[34818]: pgmap v421: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:06 smithi178 ceph-mon[34759]: pgmap v421: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:07.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:56:07.503 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:08.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 -- 172.21.15.31:0/3804560003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c81049b0 msgr2=0x7fb7c8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/3804560003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c81049b0 0x7fb7c8104dc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009900 tx=0x7fb7b802a410 comp rx=0 tx=0).stop 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 -- 172.21.15.31:0/3804560003 shutdown_connections 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/3804560003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c810de10 0x7fb7c8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/3804560003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c8105300 0x7fb7c810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/3804560003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c81049b0 0x7fb7c8104dc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 -- 172.21.15.31:0/3804560003 >> 172.21.15.31:0/3804560003 conn(0x7fb7c8079730 msgr2=0x7fb7c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 -- 172.21.15.31:0/3804560003 shutdown_connections 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 -- 172.21.15.31:0/3804560003 wait complete. 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.996+0000 7fb7ce0db640 1 Processor -- start 2024-09-21T02:56:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 -- start start 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c81049b0 0x7fb7c819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c8105300 0x7fb7c819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 0x7fb7c819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8113a70 con 0x7fb7c810de10 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81138f0 con 0x7fb7c81049b0 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7ce0db640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8113bf0 con 0x7fb7c8105300 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 0x7fb7c819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 0x7fb7c819fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51278/0 (socket says 172.21.15.31:51278) 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7c7fff640 1 -- 172.21.15.31:0/1310685766 learned_addr learned my addr 172.21.15.31:0/1310685766 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c8105300 0x7fb7c819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c81049b0 0x7fb7c819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.997+0000 7fb7c7fff640 1 -- 172.21.15.31:0/1310685766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c8105300 msgr2=0x7fb7c819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c7fff640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c8105300 0x7fb7c819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c7fff640 1 -- 172.21.15.31:0/1310685766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c81049b0 msgr2=0x7fb7c819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c7fff640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c81049b0 0x7fb7c819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c7fff640 1 -- 172.21.15.31:0/1310685766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c81a03d0 con 0x7fb7c810de10 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c81049b0 0x7fb7c819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c8105300 0x7fb7c819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c7fff640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 0x7fb7c819fd60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000eaa0 tx=0x7fb7c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c4ff9640 1 -- 172.21.15.31:0/1310685766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000cdb0 con 0x7fb7c810de10 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c4ff9640 1 -- 172.21.15.31:0/1310685766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c0005020 con 0x7fb7c810de10 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7c4ff9640 1 -- 172.21.15.31:0/1310685766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00116d0 con 0x7fb7c810de10 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c81a06c0 con 0x7fb7c810de10 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:07.998+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c81a6ee0 con 0x7fb7c810de10 2024-09-21T02:56:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 Processor -- start 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7c4ff9640 1 -- 172.21.15.31:0/1310685766 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb7c00051d0 con 0x7fb7c810de10 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 -- start start 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 0x55af4c3882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfb81b0 0x55af4c388800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfc0390 0x55af4c388d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55af4c2feea0 con 0x55af4bfb81b0 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af4c2ee2d0 con 0x55af4bfe1070 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7d464f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55af4c2f20e0 con 0x55af4bfc0390 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 0x55af4c3882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.001+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfb81b0 0x55af4c388800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 0x55af4c3882c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:47504/0 (socket says 172.21.15.31:47504) 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/4183697793 learned_addr learned my addr 172.21.15.31:0/4183697793 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfc0390 0x55af4c388d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c4ff9640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb7b00b66b0 0x7fb7b00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c7fff640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb7b00b66b0 0x7fb7b00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/4183697793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfc0390 msgr2=0x55af4c388d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfc0390 0x55af4c388d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/4183697793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfb81b0 msgr2=0x55af4c388800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfb81b0 0x55af4c388800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/4183697793 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4c04c650 con 0x55af4bfe1070 2024-09-21T02:56:08.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c4ff9640 1 -- 172.21.15.31:0/1310685766 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb7c00e78f0 con 0x7fb7c810de10 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 0x55af4c3882c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005e10 tx=0x7fb7bc0083c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be63d48d9ea2eb88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.002+0000 7fb7c7fff640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb7b00b66b0 0x7fb7b00b8bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000efd0 tx=0x7fb7c00a9db0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.003+0000 7fb7adffb640 1 -- 172.21.15.31:0/4183697793 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00cae0 con 0x55af4bfe1070 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.003+0000 7fb7adffb640 1 -- 172.21.15.31:0/4183697793 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc0054d0 con 0x55af4bfe1070 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.003+0000 7fb7adffb640 1 -- 172.21.15.31:0/4183697793 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc004b70 con 0x55af4bfe1070 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.003+0000 7fb7d464f740 1 -- 172.21.15.31:0/4183697793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 msgr2=0x55af4c3882c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.003+0000 7fb7d464f740 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 0x55af4c3882c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005e10 tx=0x7fb7bc0083c0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 -- 172.21.15.31:0/4183697793 shutdown_connections 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfc0390 0x55af4c388d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfb81b0 0x55af4c388800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 --2- 172.21.15.31:0/4183697793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfe1070 0x55af4c3882c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 -- 172.21.15.31:0/4183697793 >> 172.21.15.31:0/4183697793 conn(0x55af4bfd5460 msgr2=0x55af4c3861b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:08.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 -- 172.21.15.31:0/4183697793 shutdown_connections 2024-09-21T02:56:08.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 -- 172.21.15.31:0/4183697793 wait complete. 2024-09-21T02:56:08.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 -- start start 2024-09-21T02:56:08.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.004+0000 7fb7d464f740 1 Processor -- start 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c397c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c3981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c398700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55af4c14f700 con 0x55af4bfc0390 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af4c289160 con 0x55af4bfb81b0 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55af4c39ad40 con 0x55af4bfe1070 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7d464f740 10 client.0 _ll_register_callbacks cb 0x55af4c2a29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:08.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c3981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c397c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c3981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51306/0 (socket says 172.21.15.31:51306) 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3247713824 learned_addr learned my addr 172.21.15.31:0/3247713824 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c398700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3247713824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 msgr2=0x55af4c398700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c398700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 msgr2=0x55af4c397c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c397c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3247713824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4c2b6c90 con 0x55af4bfc0390 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c3981c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005ab0 tx=0x7fb7b8031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1444d4c614c729ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.005+0000 7fb7a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:08.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3247713824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8035040 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3247713824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb7b8004300 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7bc0054d0 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3247713824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7b8005030 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 10 client.16410 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af4c3a6960 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3247713824 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc0054d0 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3247713824 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb7b804c020 con 0x55af4bfc0390 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 10 client.16410 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 20 client.16410 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf9f9f0 0x55af4c3a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 4 client.16410 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.006+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af4c300df0 con 0x55af4bf9f9f0 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.007+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf9f9f0 0x55af4c3a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.007+0000 7fb7a6ffd640 10 client.16410 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.007+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf9f9f0 0x55af4c3a9e10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc123ad531631052 server_cookie=b4afbe582936d3f7 in_seq=0 out_seq=0 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3247713824 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af4c300df0 con 0x55af4bf9f9f0 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7a6ffd640 10 client.16410 handle_command_reply: tid=0 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 client.16410 shutdown 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 msgr2=0x55af4c3981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c3981c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005ab0 tx=0x7fb7b8031870 comp rx=0 tx=0).stop 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 shutdown_connections 2024-09-21T02:56:08.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf9f9f0 0x55af4c3a9e10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c398700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c3981c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3247713824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c397c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 >> 172.21.15.31:0/3247713824 conn(0x55af4bfd5460 msgr2=0x55af4c384750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.008+0000 7fb7a6ffd640 0 client.16410 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.009+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 shutdown_connections 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.009+0000 7fb7d464f740 1 -- 172.21.15.31:0/3247713824 wait complete. 2024-09-21T02:56:08.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.009+0000 7fb7d464f740 20 client.16410 trim_cache size 0 max 16384 2024-09-21T02:56:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 Processor -- start 2024-09-21T02:56:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 -- start start 2024-09-21T02:56:08.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfb81b0 0x55af4c38dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfc0390 0x55af4c394e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 0x55af4c3953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ad520 con 0x55af4bfe1070 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81af060 con 0x55af4bfc0390 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7d464f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81adbf0 con 0x55af4bfb81b0 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 0x55af4c3953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 0x55af4c3953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51308/0 (socket says 172.21.15.31:51308) 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.022+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/673278163 learned_addr learned my addr 172.21.15.31:0/673278163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfc0390 0x55af4c394e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfb81b0 0x55af4c38dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/673278163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfb81b0 msgr2=0x55af4c38dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfb81b0 0x55af4c38dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/673278163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfc0390 msgr2=0x55af4c394e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfc0390 0x55af4c394e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 -- 172.21.15.31:0/673278163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af4c398f40 con 0x55af4bfe1070 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 0x55af4c3953b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005770 tx=0x7fb7bc008900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73645020b0018980 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7ad7fa640 1 -- 172.21.15.31:0/673278163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00cae0 con 0x55af4bfe1070 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7ad7fa640 1 -- 172.21.15.31:0/673278163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc00cc80 con 0x55af4bfe1070 2024-09-21T02:56:08.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.023+0000 7fb7ad7fa640 1 -- 172.21.15.31:0/673278163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc01e6b0 con 0x55af4bfe1070 2024-09-21T02:56:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 -- 172.21.15.31:0/673278163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 msgr2=0x55af4c3953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 0x55af4c3953b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005770 tx=0x7fb7bc008900 comp rx=0 tx=0).stop 2024-09-21T02:56:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 -- 172.21.15.31:0/673278163 shutdown_connections 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfe1070 0x55af4c3953b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfc0390 0x55af4c394e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 --2- 172.21.15.31:0/673278163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfb81b0 0x55af4c38dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 -- 172.21.15.31:0/673278163 >> 172.21.15.31:0/673278163 conn(0x55af4bfd5460 msgr2=0x55af4c38d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 -- 172.21.15.31:0/673278163 shutdown_connections 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 -- 172.21.15.31:0/673278163 wait complete. 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.024+0000 7fb7d464f740 1 -- start start 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 Processor -- start 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c3a0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c39dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c39e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ae990 con 0x55af4bfc0390 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ae2c0 con 0x55af4bfb81b0 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c81ab500 con 0x55af4bfe1070 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7d464f740 10 client.0 _ll_register_callbacks cb 0x7fb7b8032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c39dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c39dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51316/0 (socket says 172.21.15.31:51316) 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3655061383 learned_addr learned my addr 172.21.15.31:0/3655061383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c39e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c3a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3655061383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 msgr2=0x55af4c39e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c39e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 msgr2=0x55af4c3a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c3a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c77fe640 1 -- 172.21.15.31:0/3655061383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4c39ea50 con 0x55af4bfc0390 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.025+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c3a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7c77fe640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c39dd20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8033000 tx=0x7fb7b80098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8e8eaf603bd9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3655061383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8035040 con 0x55af4bfc0390 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3655061383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb7b8004300 con 0x55af4bfc0390 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7bc00cc80 con 0x55af4bfc0390 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3655061383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb7b805d050 con 0x55af4bfc0390 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7d464f740 10 client.16416 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af4c414800 con 0x55af4bfc0390 2024-09-21T02:56:08.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.026+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3655061383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc00cc80 con 0x55af4bfc0390 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3655061383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb7b804c070 con 0x55af4bfc0390 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7d464f740 10 client.16416 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7d464f740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7d464f740 20 client.16416 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf74da0 0x55af4c417040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7d464f740 4 client.16416 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af4c2fc8a0 con 0x55af4bf74da0 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf74da0 0x55af4c417040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7a6ffd640 10 client.16416 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.027+0000 7fb7c6ffd640 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf74da0 0x55af4c417040 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761542ff836fddb9 server_cookie=992941fde2b47612 in_seq=0 out_seq=0 2024-09-21T02:56:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7a6ffd640 1 -- 172.21.15.31:0/3655061383 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af4c2fc8a0 con 0x55af4bf74da0 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7a6ffd640 10 client.16416 handle_command_reply: tid=0 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 client.16416 shutdown 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 msgr2=0x55af4c39dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c39dd20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8033000 tx=0x7fb7b80098a0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 shutdown_connections 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55af4bf74da0 0x55af4c417040 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af4bfe1070 0x55af4c39e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af4bfc0390 0x55af4c39dd20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 --2- 172.21.15.31:0/3655061383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af4bfb81b0 0x55af4c3a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 >> 172.21.15.31:0/3655061383 conn(0x55af4bfd5460 msgr2=0x55af4c396100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7a6ffd640 0 client.16416 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 shutdown_connections 2024-09-21T02:56:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 1 -- 172.21.15.31:0/3655061383 wait complete. 2024-09-21T02:56:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.028+0000 7fb7d464f740 20 client.16416 trim_cache size 0 max 16384 2024-09-21T02:56:08.036 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb7b00b66b0 msgr2=0x7fb7b00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb7b00b66b0 0x7fb7b00b8bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000efd0 tx=0x7fb7c00a9db0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 msgr2=0x7fb7c819fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 0x7fb7c819fd60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000eaa0 tx=0x7fb7c000ef60 comp rx=0 tx=0).stop 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 shutdown_connections 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb7b00b66b0 0x7fb7b00b8bc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7c8105300 0x7fb7c819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7c810de10 0x7fb7c819fd60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 --2- 172.21.15.31:0/1310685766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7c81049b0 0x7fb7c819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 >> 172.21.15.31:0/1310685766 conn(0x7fb7c8079730 msgr2=0x7fb7c80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 shutdown_connections 2024-09-21T02:56:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:08.029+0000 7fb7ce0db640 1 -- 172.21.15.31:0/1310685766 wait complete. 2024-09-21T02:56:09.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:08 smithi031 ceph-mon[31209]: pgmap v422: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:56:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:08 smithi165 ceph-mon[34818]: pgmap v422: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:56:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:08 smithi178 ceph-mon[34759]: pgmap v422: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:56:09.847 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:56:09.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:56:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:10.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 -- 172.21.15.31:0/2940337699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744109e10 msgr2=0x7f0744110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 --2- 172.21.15.31:0/2940337699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744109e10 0x7f0744110640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f073c009ca0 tx=0x7f073c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 -- 172.21.15.31:0/2940337699 shutdown_connections 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 --2- 172.21.15.31:0/2940337699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07441094c0 0x7f07441098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 --2- 172.21.15.31:0/2940337699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744109e10 0x7f0744110640 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 --2- 172.21.15.31:0/2940337699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744103b80 0x7f0744103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 -- 172.21.15.31:0/2940337699 >> 172.21.15.31:0/2940337699 conn(0x7f07440ff610 msgr2=0x7f0744101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 -- 172.21.15.31:0/2940337699 shutdown_connections 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.629+0000 7f074bb28640 1 -- 172.21.15.31:0/2940337699 wait complete. 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 Processor -- start 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 -- start start 2024-09-21T02:56:10.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 0x7f074419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07441094c0 0x7f074419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744109e10 0x7f074419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0744113ad0 con 0x7f0744103b80 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0744113950 con 0x7f0744109e10 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f074bb28640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0744113c50 con 0x7f07441094c0 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.630+0000 7f07498c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 0x7f074419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 0x7f074419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52390/0 (socket says 172.21.15.31:52390) 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 -- 172.21.15.31:0/318432445 learned_addr learned my addr 172.21.15.31:0/318432445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07490c3640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07441094c0 0x7f074419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f074a0c5640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744109e10 0x7f074419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 -- 172.21.15.31:0/318432445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07441094c0 msgr2=0x7f074419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07441094c0 0x7f074419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 -- 172.21.15.31:0/318432445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744109e10 msgr2=0x7f074419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744109e10 0x7f074419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f07498c4640 1 -- 172.21.15.31:0/318432445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07441a0390 con 0x7f0744103b80 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.631+0000 7f074a0c5640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744109e10 0x7f074419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.632+0000 7f07498c4640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 0x7f074419a420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f073400d950 tx=0x7f073400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.632+0000 7f073affd640 1 -- 172.21.15.31:0/318432445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734017070 con 0x7f0744103b80 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.632+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07441a0680 con 0x7f0744103b80 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.632+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07441a6de0 con 0x7f0744103b80 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.632+0000 7f073affd640 1 -- 172.21.15.31:0/318432445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0734009c30 con 0x7f0744103b80 2024-09-21T02:56:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.632+0000 7f073affd640 1 -- 172.21.15.31:0/318432445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073400f620 con 0x7f0744103b80 2024-09-21T02:56:10.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.634+0000 7f075209c740 1 Processor -- start 2024-09-21T02:56:10.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 -- start start 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ff2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c22490 0x558f07ff2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c2a670 0x558f07ff2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558f07f68e60 con 0x558f07c4b350 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f07f58290 con 0x558f07c22490 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f075209c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f07f5c0a0 con 0x558f07c2a670 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f074a0c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ff2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f074a0c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ff2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52404/0 (socket says 172.21.15.31:52404) 2024-09-21T02:56:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f074a0c5640 1 -- 172.21.15.31:0/1565426662 learned_addr learned my addr 172.21.15.31:0/1565426662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.635+0000 7f07490c3640 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c2a670 0x558f07ff2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f074a0c5640 1 -- 172.21.15.31:0/1565426662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c2a670 msgr2=0x558f07ff2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c2a670 0x558f07ff2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f074a0c5640 1 -- 172.21.15.31:0/1565426662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c22490 msgr2=0x558f07ff2970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c22490 0x558f07ff2970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f074a0c5640 1 -- 172.21.15.31:0/1565426662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f07cb6930 con 0x558f07c4b350 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f07490c3640 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c2a670 0x558f07ff2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f073affd640 1 -- 172.21.15.31:0/318432445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0734010460 con 0x7f0744103b80 2024-09-21T02:56:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ff2430 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f073c006010 tx=0x7f073c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d762d371d576c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f072ffff640 1 -- 172.21.15.31:0/1565426662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c031d70 con 0x558f07c4b350 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f072ffff640 1 -- 172.21.15.31:0/1565426662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073c036070 con 0x558f07c4b350 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.636+0000 7f072ffff640 1 -- 172.21.15.31:0/1565426662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c0042a0 con 0x558f07c4b350 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f073affd640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07300b6700 0x7f07300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 -- 172.21.15.31:0/1565426662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 msgr2=0x558f07ff2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ff2430 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f073c006010 tx=0x7f073c009880 comp rx=0 tx=0).stop 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f07498c4640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07300b6700 0x7f07300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 -- 172.21.15.31:0/1565426662 shutdown_connections 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c2a670 0x558f07ff2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c22490 0x558f07ff2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 --2- 172.21.15.31:0/1565426662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ff2430 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 -- 172.21.15.31:0/1565426662 >> 172.21.15.31:0/1565426662 conn(0x558f07c3f740 msgr2=0x558f07ff0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f073affd640 1 -- 172.21.15.31:0/318432445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f07340e7180 con 0x7f0744103b80 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 -- 172.21.15.31:0/1565426662 shutdown_connections 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.637+0000 7f075209c740 1 -- 172.21.15.31:0/1565426662 wait complete. 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.638+0000 7f07498c4640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07300b6700 0x7f07300b8c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x558f07ff10b0 tx=0x7f0734079ca0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.638+0000 7f075209c740 1 -- start start 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.638+0000 7f075209c740 1 Processor -- start 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.638+0000 7f075209c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f07ffdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f075209c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f075209c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f08009f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f075209c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558f07db9890 con 0x558f07c4b350 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f075209c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f07ef2d40 con 0x558f07c2a670 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f075209c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f08000bb0 con 0x558f07c22490 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f075209c740 10 client.0 _ll_register_callbacks cb 0x558f07f56c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f08009f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f08009f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52420/0 (socket says 172.21.15.31:52420) 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 -- 172.21.15.31:0/2102656099 learned_addr learned my addr 172.21.15.31:0/2102656099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f07490c3640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f07ffdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 -- 172.21.15.31:0/2102656099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 msgr2=0x558f07ffdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f07ffdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 -- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 msgr2=0x558f07ffe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffe530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 -- 172.21.15.31:0/2102656099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f0800a4a0 con 0x558f07c4b350 2024-09-21T02:56:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f07490c3640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f074a0c5640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f08009f30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f073c0096a0 tx=0x7f073c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29be2c10618931e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.639+0000 7f072d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f072d7fa640 1 -- 172.21.15.31:0/2102656099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c004d40 con 0x558f07c4b350 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073c036070 con 0x558f07c4b350 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f072d7fa640 1 -- 172.21.15.31:0/2102656099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f073c031450 con 0x558f07c4b350 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f072d7fa640 1 -- 172.21.15.31:0/2102656099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f073c031650 con 0x558f07c4b350 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 10 client.16428 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f0800cdd0 con 0x558f07c4b350 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f072d7fa640 1 -- 172.21.15.31:0/2102656099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073c036070 con 0x558f07c4b350 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f072d7fa640 1 -- 172.21.15.31:0/2102656099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f073c05a070 con 0x558f07c4b350 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 10 client.16428 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 20 client.16428 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07c09cd0 0x558f08010130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 4 client.16428 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.640+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f07f6adb0 con 0x558f07c09cd0 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.641+0000 7f07490c3640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07c09cd0 0x558f08010130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.641+0000 7f07490c3640 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07c09cd0 0x558f08010130 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d2b36a68bb1e44 server_cookie=bf1f2e9133dabbcb in_seq=0 out_seq=0 2024-09-21T02:56:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.641+0000 7f072d7fa640 10 client.16428 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f072d7fa640 1 -- 172.21.15.31:0/2102656099 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f07f6adb0 con 0x558f07c09cd0 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f072d7fa640 10 client.16428 handle_command_reply: tid=0 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 client.16428 shutdown 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 msgr2=0x558f08009f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f08009f30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f073c0096a0 tx=0x7f073c03d040 comp rx=0 tx=0).stop 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 shutdown_connections 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07c09cd0 0x558f08010130 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f08009f30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f072d7fa640 0 client.16428 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 --2- 172.21.15.31:0/2102656099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f07ffdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 >> 172.21.15.31:0/2102656099 conn(0x558f07c3f740 msgr2=0x558f07ff55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 shutdown_connections 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.642+0000 7f075209c740 1 -- 172.21.15.31:0/2102656099 wait complete. 2024-09-21T02:56:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.643+0000 7f075209c740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.660+0000 7f075209c740 1 Processor -- start 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 -- start start 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08009ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ffcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f072800e7a0 con 0x558f07c4b350 2024-09-21T02:56:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f07280102e0 con 0x558f07c2a670 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f075209c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f072800ee70 con 0x558f07c22490 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f074a0c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ffcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f074a0c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ffcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52430/0 (socket says 172.21.15.31:52430) 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f074a0c5640 1 -- 172.21.15.31:0/1897153018 learned_addr learned my addr 172.21.15.31:0/1897153018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08009ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f07490c3640 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f074a0c5640 1 -- 172.21.15.31:0/1897153018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 msgr2=0x558f08009ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.661+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08009ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f074a0c5640 1 -- 172.21.15.31:0/1897153018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 msgr2=0x558f07ffc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f074a0c5640 1 -- 172.21.15.31:0/1897153018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f07ff4720 con 0x558f07c4b350 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ffcbb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f073c02f7b0 tx=0x7f073c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde0105f2b28bc20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f072f7fe640 1 -- 172.21.15.31:0/1897153018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c031c00 con 0x558f07c4b350 2024-09-21T02:56:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f072f7fe640 1 -- 172.21.15.31:0/1897153018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073c036040 con 0x558f07c4b350 2024-09-21T02:56:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.662+0000 7f072f7fe640 1 -- 172.21.15.31:0/1897153018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c0044b0 con 0x558f07c4b350 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 -- 172.21.15.31:0/1897153018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 msgr2=0x558f07ffcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ffcbb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f073c02f7b0 tx=0x7f073c02bfd0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 -- 172.21.15.31:0/1897153018 shutdown_connections 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c4b350 0x558f07ffcbb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c2a670 0x558f07ffc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 --2- 172.21.15.31:0/1897153018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08009ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 -- 172.21.15.31:0/1897153018 >> 172.21.15.31:0/1897153018 conn(0x558f07c3f740 msgr2=0x558f07ff8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:10.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 -- 172.21.15.31:0/1897153018 shutdown_connections 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 -- 172.21.15.31:0/1897153018 wait complete. 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.663+0000 7f075209c740 1 -- start start 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 Processor -- start 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08013b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 0x558f08010fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c4b350 0x558f080116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f072800fc10 con 0x558f07c2a670 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f072800f540 con 0x558f07c4b350 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f072800c780 con 0x558f07c22490 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.664+0000 7f075209c740 10 client.0 _ll_register_callbacks cb 0x7f073c009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 0x558f08010fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 0x558f08010fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52436/0 (socket says 172.21.15.31:52436) 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 -- 172.21.15.31:0/1949827173 learned_addr learned my addr 172.21.15.31:0/1949827173 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08013b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c4b350 0x558f080116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 -- 172.21.15.31:0/1949827173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 msgr2=0x558f08013b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08013b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 -- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c4b350 msgr2=0x558f080116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c4b350 0x558f080116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 -- 172.21.15.31:0/1949827173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f080798b0 con 0x558f07c2a670 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f07490c3640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 0x558f08010fd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f074406a800 tx=0x7f074001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d482d1c65f4896 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f072d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f072d7fa640 1 -- 172.21.15.31:0/1949827173 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740004030 con 0x558f07c2a670 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.665+0000 7f072d7fa640 1 -- 172.21.15.31:0/1949827173 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f07400041d0 con 0x558f07c2a670 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073c036040 con 0x558f07c2a670 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f072d7fa640 1 -- 172.21.15.31:0/1949827173 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0740031050 con 0x558f07c2a670 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 10 client.16434 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f0807aa40 con 0x558f07c2a670 2024-09-21T02:56:10.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f072d7fa640 1 -- 172.21.15.31:0/1949827173 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073c036040 con 0x558f07c2a670 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f072d7fa640 1 -- 172.21.15.31:0/1949827173 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0740005290 con 0x558f07c2a670 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 10 client.16434 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 20 client.16434 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07bdf080 0x558f0807d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 4 client.16434 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.666+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558f07f66860 con 0x558f07bdf080 2024-09-21T02:56:10.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.667+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07bdf080 0x558f0807d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.667+0000 7f074a0c5640 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07bdf080 0x558f0807d280 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76c548bf497e466 server_cookie=a20312506744ee7f in_seq=0 out_seq=0 2024-09-21T02:56:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.667+0000 7f072d7fa640 10 client.16434 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.669+0000 7f072d7fa640 1 -- 172.21.15.31:0/1949827173 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558f07f66860 con 0x558f07bdf080 2024-09-21T02:56:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.669+0000 7f072d7fa640 10 client.16434 handle_command_reply: tid=0 2024-09-21T02:56:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.669+0000 7f075209c740 1 client.16434 shutdown 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.669+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 msgr2=0x558f08010fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.669+0000 7f075209c740 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 0x558f08010fd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f074406a800 tx=0x7f074001ea70 comp rx=0 tx=0).stop 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 shutdown_connections 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558f07bdf080 0x558f0807d280 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f07c4b350 0x558f080116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f07c2a670 0x558f08010fd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 --2- 172.21.15.31:0/1949827173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f07c22490 0x558f08013b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f072d7fa640 0 client.16434 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 >> 172.21.15.31:0/1949827173 conn(0x558f07c3f740 msgr2=0x558f07ffc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 shutdown_connections 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 1 -- 172.21.15.31:0/1949827173 wait complete. 2024-09-21T02:56:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.670+0000 7f075209c740 20 client.16434 trim_cache size 0 max 16384 2024-09-21T02:56:10.674 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:56:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:56:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "1f3c55b2-f33d-43f4-a2d0-5017d66838d8", 2024-09-21T02:56:10.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:56:10.674 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07300b6700 msgr2=0x7f07300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07300b6700 0x7f07300b8c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x558f07ff10b0 tx=0x7f0734079ca0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 msgr2=0x7f074419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 0x7f074419a420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f073400d950 tx=0x7f073400de10 comp rx=0 tx=0).stop 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 shutdown_connections 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f07300b6700 0x7f07300b8c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07441094c0 0x7f074419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0744109e10 0x7f074419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 --2- 172.21.15.31:0/318432445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0744103b80 0x7f074419a420 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 >> 172.21.15.31:0/318432445 conn(0x7f07440ff610 msgr2=0x7f0744108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 shutdown_connections 2024-09-21T02:56:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:10.671+0000 7f074bb28640 1 -- 172.21.15.31:0/318432445 wait complete. 2024-09-21T02:56:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:10 smithi031 ceph-mon[31209]: pgmap v423: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:10 smithi165 ceph-mon[34818]: pgmap v423: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:11.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:10 smithi178 ceph-mon[34759]: pgmap v423: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:11.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:56:11.736 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:11.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:11 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:56:11.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:11 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:11.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:11 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:56:11.996 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:11 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:11 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:56:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:11 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:11 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:56:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:11 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:11 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:56:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:11 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:11 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:56:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:11 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 -- 172.21.15.31:0/2533123104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109570 msgr2=0x7f97d4109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/2533123104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109570 0x7f97d4109980 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009900 tx=0x7f97c802a410 comp rx=0 tx=0).stop 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 -- 172.21.15.31:0/2533123104 shutdown_connections 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/2533123104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109ec0 0x7f97d41106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/2533123104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109570 0x7f97d4109980 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/2533123104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d4103c30 0x7f97d4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 -- 172.21.15.31:0/2533123104 >> 172.21.15.31:0/2533123104 conn(0x7f97d407b8d0 msgr2=0x7f97d407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 -- 172.21.15.31:0/2533123104 shutdown_connections 2024-09-21T02:56:12.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.246+0000 7f97d8d2a640 1 -- 172.21.15.31:0/2533123104 wait complete. 2024-09-21T02:56:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 Processor -- start 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 -- start start 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d4103c30 0x7f97d419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109570 0x7f97d419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 0x7f97d419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4113ae0 con 0x7f97d4109ec0 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4113960 con 0x7f97d4109570 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d8d2a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4113c60 con 0x7f97d4103c30 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 0x7f97d419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.247+0000 7f97d259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d4103c30 0x7f97d419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 0x7f97d419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52446/0 (socket says 172.21.15.31:52446) 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 -- 172.21.15.31:0/1768501818 learned_addr learned my addr 172.21.15.31:0/1768501818 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109570 0x7f97d419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 -- 172.21.15.31:0/1768501818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d4103c30 msgr2=0x7f97d419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d4103c30 0x7f97d419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 -- 172.21.15.31:0/1768501818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109570 msgr2=0x7f97d419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109570 0x7f97d419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 -- 172.21.15.31:0/1768501818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d41a0440 con 0x7f97d4109ec0 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109570 0x7f97d419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 0x7f97d419fd10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00eb10 tx=0x7f97cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.248+0000 7f97c37fe640 1 -- 172.21.15.31:0/1768501818 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc004b20 con 0x7f97d4109ec0 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.249+0000 7f97c37fe640 1 -- 172.21.15.31:0/1768501818 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97cc004cc0 con 0x7f97d4109ec0 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.249+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d41a0730 con 0x7f97d4109ec0 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.249+0000 7f97c37fe640 1 -- 172.21.15.31:0/1768501818 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc010670 con 0x7f97d4109ec0 2024-09-21T02:56:12.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.249+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d41a6e90 con 0x7f97d4109ec0 2024-09-21T02:56:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.252+0000 7f97df29e740 1 Processor -- start 2024-09-21T02:56:12.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.252+0000 7f97c37fe640 1 -- 172.21.15.31:0/1768501818 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f97cc010830 con 0x7f97d4109ec0 2024-09-21T02:56:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.252+0000 7f97df29e740 1 -- start start 2024-09-21T02:56:12.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97df29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75dff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97df29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75dff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97df29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75dffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97df29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd75d75e50 con 0x55dd75a2f1b0 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97df29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dd75d65280 con 0x55dd75a37390 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97df29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd75d69090 con 0x55dd75a58070 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75dff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75dff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60738/0 (socket says 172.21.15.31:60738) 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75dff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 -- 172.21.15.31:0/1482288292 learned_addr learned my addr 172.21.15.31:0/1482288292 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75dffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 -- 172.21.15.31:0/1482288292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 msgr2=0x55dd75dffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75dffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 -- 172.21.15.31:0/1482288292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 msgr2=0x55dd75dff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.253+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75dff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97c37fe640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97bc0b6680 0x7f97bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97d1d9b640 1 -- 172.21.15.31:0/1482288292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd75ac3650 con 0x55dd75a58070 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97d259c640 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75dff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97bc0b6680 0x7f97bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97c37fe640 1 -- 172.21.15.31:0/1768501818 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f97cc0e7cc0 con 0x7f97d4109ec0 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75dff270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97c80057c0 tx=0x7f97c8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cac9041f7b4ce53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97c0ff9640 1 -- 172.21.15.31:0/1482288292 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8035070 con 0x55dd75a58070 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97c0ff9640 1 -- 172.21.15.31:0/1482288292 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97c8003aa0 con 0x55dd75a58070 2024-09-21T02:56:12.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.254+0000 7f97c0ff9640 1 -- 172.21.15.31:0/1482288292 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c803d960 con 0x55dd75a58070 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 -- 172.21.15.31:0/1482288292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 msgr2=0x55dd75dff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75dff270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97c80057c0 tx=0x7f97c8000bc0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97bc0b6680 0x7f97bc0b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97c80096c0 tx=0x7f97c8049000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 -- 172.21.15.31:0/1482288292 shutdown_connections 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75dffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75dff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 --2- 172.21.15.31:0/1482288292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75dff270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 -- 172.21.15.31:0/1482288292 >> 172.21.15.31:0/1482288292 conn(0x55dd75a4c460 msgr2=0x55dd75dfd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 -- 172.21.15.31:0/1482288292 shutdown_connections 2024-09-21T02:56:12.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.255+0000 7f97df29e740 1 -- 172.21.15.31:0/1482288292 wait complete. 2024-09-21T02:56:12.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 -- start start 2024-09-21T02:56:12.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 Processor -- start 2024-09-21T02:56:12.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e0a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e16d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd75bc6700 con 0x55dd75a2f1b0 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dd75d00110 con 0x55dd75a37390 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.256+0000 7f97df29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd75e0da50 con 0x55dd75a58070 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e0a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e0a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52476/0 (socket says 172.21.15.31:52476) 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d259c640 1 -- 172.21.15.31:0/372262513 learned_addr learned my addr 172.21.15.31:0/372262513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97df29e740 10 client.0 _ll_register_callbacks cb 0x55dd75d19990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e16d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 -- 172.21.15.31:0/372262513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 msgr2=0x55dd75e16d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e16d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 -- 172.21.15.31:0/372262513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 msgr2=0x55dd75e0a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e0a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 -- 172.21.15.31:0/372262513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd75d2dc40 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d259c640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e0a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0ae70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97c8031440 tx=0x7f97c803d660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0c77cdbcebd9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.257+0000 7f97b9ffb640 1 -- 172.21.15.31:0/372262513 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8035030 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c8003aa0 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97b9ffb640 1 -- 172.21.15.31:0/372262513 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f97c803d760 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97b9ffb640 1 -- 172.21.15.31:0/372262513 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97c803d960 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 10 client.25988 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd75e19730 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97b9ffb640 1 -- 172.21.15.31:0/372262513 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97c8003aa0 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97b9ffb640 1 -- 172.21.15.31:0/372262513 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f97c8065020 con 0x55dd75a37390 2024-09-21T02:56:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 10 client.25988 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 20 client.25988 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.258+0000 7f97df29e740 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd75a169f0 0x55dd75e1ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.259+0000 7f97df29e740 4 client.25988 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.259+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd75d77da0 con 0x55dd75a169f0 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.259+0000 7f97d259c640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd75a169f0 0x55dd75e1ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.259+0000 7f97d259c640 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd75a169f0 0x55dd75e1ca90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0af5ec6d058671 server_cookie=b961c2191c97f338 in_seq=0 out_seq=0 2024-09-21T02:56:12.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.259+0000 7f97b9ffb640 10 client.25988 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97b9ffb640 1 -- 172.21.15.31:0/372262513 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd75d77da0 con 0x55dd75a169f0 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97b9ffb640 10 client.25988 handle_command_reply: tid=0 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 client.25988 shutdown 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 msgr2=0x55dd75e0ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0ae70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97c8031440 tx=0x7f97c803d660 comp rx=0 tx=0).stop 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 shutdown_connections 2024-09-21T02:56:12.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd75a169f0 0x55dd75e1ca90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e16d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0ae70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 --2- 172.21.15.31:0/372262513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e0a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 >> 172.21.15.31:0/372262513 conn(0x55dd75a4c460 msgr2=0x55dd75e02400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97b9ffb640 0 client.25988 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 shutdown_connections 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.260+0000 7f97df29e740 1 -- 172.21.15.31:0/372262513 wait complete. 2024-09-21T02:56:12.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.261+0000 7f97df29e740 20 client.25988 trim_cache size 0 max 16384 2024-09-21T02:56:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 Processor -- start 2024-09-21T02:56:12.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 -- start start 2024-09-21T02:56:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a2f1b0 0x55dd75e00c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 0x55dd75dfc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41ad550 con 0x55dd75a58070 2024-09-21T02:56:12.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.278+0000 7f97df29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41af090 con 0x55dd75a37390 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97df29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41adc20 con 0x55dd75a2f1b0 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 0x55dd75dfc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 0x55dd75dfc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52480/0 (socket says 172.21.15.31:52480) 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 -- 172.21.15.31:0/3838112689 learned_addr learned my addr 172.21.15.31:0/3838112689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d259c640 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a2f1b0 0x55dd75e00c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 -- 172.21.15.31:0/3838112689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a2f1b0 msgr2=0x55dd75e00c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a2f1b0 0x55dd75e00c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 -- 172.21.15.31:0/3838112689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 msgr2=0x55dd75e0c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 -- 172.21.15.31:0/3838112689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd75a16c20 con 0x55dd75a58070 2024-09-21T02:56:12.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 0x55dd75dfc710 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0aada0 tx=0x7f97cc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c290db985e538d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97bbfff640 1 -- 172.21.15.31:0/3838112689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0aee60 con 0x55dd75a58070 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.279+0000 7f97bbfff640 1 -- 172.21.15.31:0/3838112689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97cc0ad640 con 0x55dd75a58070 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.280+0000 7f97bbfff640 1 -- 172.21.15.31:0/3838112689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0b2200 con 0x55dd75a58070 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 -- 172.21.15.31:0/3838112689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 msgr2=0x55dd75dfc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 0x55dd75dfc710 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0aada0 tx=0x7f97cc00cee0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 -- 172.21.15.31:0/3838112689 shutdown_connections 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a58070 0x55dd75dfc710 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e0c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 --2- 172.21.15.31:0/3838112689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a2f1b0 0x55dd75e00c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 -- 172.21.15.31:0/3838112689 >> 172.21.15.31:0/3838112689 conn(0x55dd75a4c460 msgr2=0x55dd75e0bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 -- 172.21.15.31:0/3838112689 shutdown_connections 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 -- 172.21.15.31:0/3838112689 wait complete. 2024-09-21T02:56:12.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.281+0000 7f97df29e740 1 -- start start 2024-09-21T02:56:12.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 Processor -- start 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e1d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e1d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41ae9c0 con 0x55dd75a2f1b0 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41ae2f0 con 0x55dd75a37390 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41ab530 con 0x55dd75a58070 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97df29e740 10 client.0 _ll_register_callbacks cb 0x7f97c40137d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e1d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e1d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52488/0 (socket says 172.21.15.31:52488) 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97d259c640 1 -- 172.21.15.31:0/3037005421 learned_addr learned my addr 172.21.15.31:0/3037005421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:12.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e1d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.282+0000 7f97d2d9d640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d259c640 1 -- 172.21.15.31:0/3037005421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 msgr2=0x55dd75e7f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d259c640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e7f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d259c640 1 -- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 msgr2=0x55dd75e1d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d259c640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e1d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d259c640 1 -- 172.21.15.31:0/3037005421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd75e7f840 con 0x55dd75a2f1b0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e1d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97d259c640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e1d450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009b40 tx=0x7f97c4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884f2f54ec1e84ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97b9ffb640 1 -- 172.21.15.31:0/3037005421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4003da0 con 0x55dd75a2f1b0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97cc0ad640 con 0x55dd75a2f1b0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97b9ffb640 1 -- 172.21.15.31:0/3037005421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f97c4029430 con 0x55dd75a2f1b0 2024-09-21T02:56:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97b9ffb640 1 -- 172.21.15.31:0/3037005421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97c4037050 con 0x55dd75a2f1b0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97df29e740 10 client.16452 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.283+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd75e86c20 con 0x55dd75a2f1b0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97b9ffb640 1 -- 172.21.15.31:0/3037005421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97cc0ad640 con 0x55dd75a2f1b0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97b9ffb640 1 -- 172.21.15.31:0/3037005421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f97c4028070 con 0x55dd75a2f1b0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97df29e740 10 client.16452 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97df29e740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97df29e740 20 client.16452 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97df29e740 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd759ebda0 0x55dd75e89380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97df29e740 4 client.16452 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd75d73850 con 0x55dd759ebda0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.284+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd759ebda0 0x55dd75e89380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97d1d9b640 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd759ebda0 0x55dd75e89380 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b5ccdf0464b2f3 server_cookie=b17332ac3e3dae93 in_seq=0 out_seq=0 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97b9ffb640 10 client.16452 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97b9ffb640 1 -- 172.21.15.31:0/3037005421 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55dd75d73850 con 0x55dd759ebda0 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97b9ffb640 10 client.16452 handle_command_reply: tid=0 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 client.16452 shutdown 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 msgr2=0x55dd75e1d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e1d450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009b40 tx=0x7f97c4004410 comp rx=0 tx=0).stop 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 shutdown_connections 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55dd759ebda0 0x55dd75e89380 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd75a58070 0x55dd75e7f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd75a37390 0x55dd75e1d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 --2- 172.21.15.31:0/3037005421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd75a2f1b0 0x55dd75e1d450 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 >> 172.21.15.31:0/3037005421 conn(0x55dd75a4c460 msgr2=0x55dd75e0c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97b9ffb640 0 client.16452 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 shutdown_connections 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 1 -- 172.21.15.31:0/3037005421 wait complete. 2024-09-21T02:56:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.285+0000 7f97df29e740 20 client.16452 trim_cache size 0 max 16384 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "1f3c55b2-f33d-43f4-a2d0-5017d66838d8", 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:56:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97bc0b6680 msgr2=0x7f97bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97bc0b6680 0x7f97bc0b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f97c80096c0 tx=0x7f97c8049000 comp rx=0 tx=0).stop 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 msgr2=0x7f97d419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 0x7f97d419fd10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00eb10 tx=0x7f97cc00c2d0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 shutdown_connections 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f97bc0b6680 0x7f97bc0b8b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97d4109ec0 0x7f97d419fd10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97d4109570 0x7f97d419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 --2- 172.21.15.31:0/1768501818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d4103c30 0x7f97d419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 >> 172.21.15.31:0/1768501818 conn(0x7f97d407b8d0 msgr2=0x7f97d4107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 shutdown_connections 2024-09-21T02:56:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:12.286+0000 7f97d8d2a640 1 -- 172.21.15.31:0/1768501818 wait complete. 2024-09-21T02:56:13.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:12 smithi031 ceph-mon[31209]: pgmap v424: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:13.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:13.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:13.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:13.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:56:13.084 INFO:tasks.cephfs.filesystem:scrub status for tag:1f3c55b2-f33d-43f4-a2d0-5017d66838d8 - {'path': '/', 'tag': '1f3c55b2-f33d-43f4-a2d0-5017d66838d8', 'options': 'recursive,force'} 2024-09-21T02:56:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:12 smithi165 ceph-mon[34818]: pgmap v424: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:56:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:12 smithi178 ceph-mon[34759]: pgmap v424: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:56:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:56:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:14 smithi031 ceph-mon[31209]: pgmap v425: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:56:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:14 smithi165 ceph-mon[34818]: pgmap v425: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:56:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:14 smithi178 ceph-mon[34759]: pgmap v425: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:56:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:15 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:56:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:15 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:56:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:15 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:56:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:16 smithi031 ceph-mon[31209]: pgmap v426: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:56:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:16 smithi165 ceph-mon[34818]: pgmap v426: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:56:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:16 smithi178 ceph-mon[34759]: pgmap v426: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-21T02:56:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:18 smithi031 ceph-mon[31209]: pgmap v427: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:56:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:18 smithi165 ceph-mon[34818]: pgmap v427: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:56:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:18 smithi178 ceph-mon[34759]: pgmap v427: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:56:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:20 smithi031 ceph-mon[31209]: pgmap v428: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:56:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:20 smithi165 ceph-mon[34818]: pgmap v428: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:56:21.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:20 smithi178 ceph-mon[34759]: pgmap v428: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:56:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:22 smithi031 ceph-mon[31209]: pgmap v429: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:56:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:22 smithi165 ceph-mon[34818]: pgmap v429: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:56:23.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:22 smithi178 ceph-mon[34759]: pgmap v429: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:56:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:24 smithi031 ceph-mon[31209]: pgmap v430: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:56:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:24 smithi165 ceph-mon[34818]: pgmap v430: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:56:25.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:24 smithi178 ceph-mon[34759]: pgmap v430: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:56:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:26 smithi031 ceph-mon[31209]: pgmap v431: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:26 smithi165 ceph-mon[34818]: pgmap v431: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:27.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:26 smithi178 ceph-mon[34759]: pgmap v431: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:28 smithi031 ceph-mon[31209]: pgmap v432: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:28 smithi165 ceph-mon[34818]: pgmap v432: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:29.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:28 smithi178 ceph-mon[34759]: pgmap v432: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:30 smithi031 ceph-mon[31209]: pgmap v433: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:30 smithi165 ceph-mon[34818]: pgmap v433: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:31.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:30 smithi178 ceph-mon[34759]: pgmap v433: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:32 smithi031 ceph-mon[31209]: pgmap v434: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:32 smithi165 ceph-mon[34818]: pgmap v434: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:33.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:32 smithi178 ceph-mon[34759]: pgmap v434: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:34 smithi031 ceph-mon[31209]: pgmap v435: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:34 smithi165 ceph-mon[34818]: pgmap v435: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:34 smithi178 ceph-mon[34759]: pgmap v435: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:36 smithi031 ceph-mon[31209]: pgmap v436: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:36 smithi165 ceph-mon[34818]: pgmap v436: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:37.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:36 smithi178 ceph-mon[34759]: pgmap v436: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:38 smithi031 ceph-mon[31209]: pgmap v437: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:38 smithi165 ceph-mon[34818]: pgmap v437: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:39.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:38 smithi178 ceph-mon[34759]: pgmap v437: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:40 smithi031 ceph-mon[31209]: pgmap v438: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:40 smithi165 ceph-mon[34818]: pgmap v438: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:41.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:40 smithi178 ceph-mon[34759]: pgmap v438: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:42 smithi031 ceph-mon[31209]: pgmap v439: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:43.084 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:56:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:42 smithi165 ceph-mon[34818]: pgmap v439: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:42 smithi178 ceph-mon[34759]: pgmap v439: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:43.359 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 -- 172.21.15.31:0/153340981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 msgr2=0x7f6c64110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/153340981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c64110680 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009ca0 tx=0x7f6c5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 -- 172.21.15.31:0/153340981 shutdown_connections 2024-09-21T02:56:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/153340981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c64110680 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/153340981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c64109500 0x7f6c64109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/153340981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64101fc0 0x7f6c641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 -- 172.21.15.31:0/153340981 >> 172.21.15.31:0/153340981 conn(0x7f6c64079f90 msgr2=0x7f6c6407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 -- 172.21.15.31:0/153340981 shutdown_connections 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.864+0000 7f6c6ae61640 1 -- 172.21.15.31:0/153340981 wait complete. 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 Processor -- start 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 -- start start 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c64101fc0 0x7f6c6419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64109500 0x7f6c6419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c6419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64113b10 con 0x7f6c64109e50 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64113990 con 0x7f6c64101fc0 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.865+0000 7f6c6ae61640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64113c90 con 0x7f6c64109500 2024-09-21T02:56:43.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c6419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c6419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33970/0 (socket says 172.21.15.31:33970) 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 -- 172.21.15.31:0/3957805549 learned_addr learned my addr 172.21.15.31:0/3957805549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c63fff640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64109500 0x7f6c6419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c64101fc0 0x7f6c6419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 -- 172.21.15.31:0/3957805549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64109500 msgr2=0x7f6c6419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64109500 0x7f6c6419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 -- 172.21.15.31:0/3957805549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c64101fc0 msgr2=0x7f6c6419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c64101fc0 0x7f6c6419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 -- 172.21.15.31:0/3957805549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c641a0330 con 0x7f6c64109e50 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c63fff640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64109500 0x7f6c6419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c693fe640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c6419fcc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c005cc0 tx=0x7f6c5c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c61ffb640 1 -- 172.21.15.31:0/3957805549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c003bf0 con 0x7f6c64109e50 2024-09-21T02:56:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c61ffb640 1 -- 172.21.15.31:0/3957805549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c5c003d90 con 0x7f6c64109e50 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c641a05c0 con 0x7f6c64109e50 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.866+0000 7f6c61ffb640 1 -- 172.21.15.31:0/3957805549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c02fbc0 con 0x7f6c64109e50 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.867+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a0b00 con 0x7f6c64109e50 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.869+0000 7f6c713d5740 1 Processor -- start 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c61ffb640 1 -- 172.21.15.31:0/3957805549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6c5c02fd60 con 0x7f6c64109e50 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 -- start start 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64213650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64213b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b642140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558b64189e50 con 0x558b63e6c070 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558b64179280 con 0x558b63e431b0 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c713d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558b6417d090 con 0x558b63e4b390 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64213650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64213650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33976/0 (socket says 172.21.15.31:33976) 2024-09-21T02:56:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 -- 172.21.15.31:0/1904988242 learned_addr learned my addr 172.21.15.31:0/1904988242 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64213b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c61ffb640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6c4c0b66b0 0x7f6c4c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b642140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 -- 172.21.15.31:0/1904988242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 msgr2=0x558b642140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b642140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 -- 172.21.15.31:0/1904988242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 msgr2=0x558b64213b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64213b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.870+0000 7f6c63fff640 1 -- 172.21.15.31:0/1904988242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b63ed7650 con 0x558b63e6c070 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.871+0000 7f6c693fe640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6c4c0b66b0 0x7f6c4c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.871+0000 7f6c61ffb640 1 -- 172.21.15.31:0/3957805549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6c5c108ba0 con 0x7f6c64109e50 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.871+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64213650 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800bf70 tx=0x7f6c5800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3afeefb58283aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.871+0000 7f6c4affd640 1 -- 172.21.15.31:0/1904988242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004030 con 0x558b63e6c070 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.871+0000 7f6c4affd640 1 -- 172.21.15.31:0/1904988242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c58019070 con 0x558b63e6c070 2024-09-21T02:56:43.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.871+0000 7f6c4affd640 1 -- 172.21.15.31:0/1904988242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800f3f0 con 0x558b63e6c070 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 -- 172.21.15.31:0/1904988242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 msgr2=0x558b64213650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c693fe640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6c4c0b66b0 0x7f6c4c0b8bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0caf80 tx=0x7f6c5c0095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64213650 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800bf70 tx=0x7f6c5800b6d0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 -- 172.21.15.31:0/1904988242 shutdown_connections 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b642140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64213b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1904988242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64213650 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 -- 172.21.15.31:0/1904988242 >> 172.21.15.31:0/1904988242 conn(0x558b63e60460 msgr2=0x558b64211540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 -- 172.21.15.31:0/1904988242 shutdown_connections 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 -- 172.21.15.31:0/1904988242 wait complete. 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 -- start start 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.872+0000 7f6c713d5740 1 Processor -- start 2024-09-21T02:56:43.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 0x558b64223090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e4b390 0x558b642235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b64223b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558b63fda700 con 0x558b63e431b0 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558b64114110 con 0x558b63e4b390 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558b64226160 con 0x558b63e6c070 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c713d5740 10 client.0 _ll_register_callbacks cb 0x558b6412d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 0x558b64223090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 0x558b64223090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33980/0 (socket says 172.21.15.31:33980) 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 -- 172.21.15.31:0/566150164 learned_addr learned my addr 172.21.15.31:0/566150164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e4b390 0x558b642235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b64223b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 -- 172.21.15.31:0/566150164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 msgr2=0x558b64223b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b64223b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 -- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e4b390 msgr2=0x558b642235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.873+0000 7f6c63fff640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e4b390 0x558b642235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c63fff640 1 -- 172.21.15.31:0/566150164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b64141c40 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e4b390 0x558b642235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c63fff640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 0x558b64223090 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006010 tx=0x7f6c5800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb30321945945512 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c43fff640 1 -- 172.21.15.31:0/566150164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58028020 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c43fff640 1 -- 172.21.15.31:0/566150164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6c58005230 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c58019070 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c43fff640 1 -- 172.21.15.31:0/566150164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c58005430 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c713d5740 10 client.16464 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b64231e40 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.874+0000 7f6c43fff640 1 -- 172.21.15.31:0/566150164 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c58019070 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c43fff640 1 -- 172.21.15.31:0/566150164 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c58004ab0 con 0x558b63e431b0 2024-09-21T02:56:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c713d5740 10 client.16464 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c713d5740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c713d5740 20 client.16464 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c713d5740 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63e2a9f0 0x558b642352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c713d5740 4 client.16464 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b6418bda0 con 0x558b63e2a9f0 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.875+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63e2a9f0 0x558b642352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.876+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63e2a9f0 0x558b642352b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20891ad79062864 server_cookie=82f515efc514379a in_seq=0 out_seq=0 2024-09-21T02:56:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.876+0000 7f6c43fff640 10 client.16464 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.876+0000 7f6c43fff640 1 -- 172.21.15.31:0/566150164 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b6418bda0 con 0x558b63e2a9f0 2024-09-21T02:56:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.876+0000 7f6c43fff640 10 client.16464 handle_command_reply: tid=0 2024-09-21T02:56:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.876+0000 7f6c713d5740 1 client.16464 shutdown 2024-09-21T02:56:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 msgr2=0x558b64223090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 0x558b64223090 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006010 tx=0x7f6c5800b630 comp rx=0 tx=0).stop 2024-09-21T02:56:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 shutdown_connections 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63e2a9f0 0x558b642352b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b64223b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e4b390 0x558b642235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 --2- 172.21.15.31:0/566150164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e431b0 0x558b64223090 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 >> 172.21.15.31:0/566150164 conn(0x558b63e60460 msgr2=0x558b642167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c43fff640 0 client.16464 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 shutdown_connections 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 1 -- 172.21.15.31:0/566150164 wait complete. 2024-09-21T02:56:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.877+0000 7f6c713d5740 20 client.16464 trim_cache size 0 max 16384 2024-09-21T02:56:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.894+0000 7f6c713d5740 1 Processor -- start 2024-09-21T02:56:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.894+0000 7f6c713d5740 1 -- start start 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64223550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 0x558b64220280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b642207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c713d5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ad440 con 0x558b63e4b390 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c713d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641aef80 con 0x558b63e431b0 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c713d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641adb10 con 0x558b63e6c070 2024-09-21T02:56:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 0x558b64220280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 0x558b64220280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33984/0 (socket says 172.21.15.31:33984) 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 -- 172.21.15.31:0/3845614182 learned_addr learned my addr 172.21.15.31:0/3845614182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c63fff640 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b642207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 -- 172.21.15.31:0/3845614182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 msgr2=0x558b642207c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b642207c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 -- 172.21.15.31:0/3845614182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 msgr2=0x558b64223550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c63fff640 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64223550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64223550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 -- 172.21.15.31:0/3845614182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b63e2ac20 con 0x558b63e4b390 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.895+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 0x558b64220280 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6c54013490 tx=0x7f6c5400e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1545f7adf3f7f9c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/3845614182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54004020 con 0x558b63e4b390 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/3845614182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c540041c0 con 0x558b63e4b390 2024-09-21T02:56:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/3845614182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54010750 con 0x558b63e4b390 2024-09-21T02:56:43.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c713d5740 1 -- 172.21.15.31:0/3845614182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 msgr2=0x558b64220280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c713d5740 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 0x558b64220280 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6c54013490 tx=0x7f6c5400e3b0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c713d5740 1 -- 172.21.15.31:0/3845614182 shutdown_connections 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c713d5740 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e6c070 0x558b642207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c713d5740 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e4b390 0x558b64220280 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.896+0000 7f6c713d5740 1 --2- 172.21.15.31:0/3845614182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64223550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 -- 172.21.15.31:0/3845614182 >> 172.21.15.31:0/3845614182 conn(0x558b63e60460 msgr2=0x558b64222e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 -- 172.21.15.31:0/3845614182 shutdown_connections 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 -- 172.21.15.31:0/3845614182 wait complete. 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 -- start start 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 Processor -- start 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64236410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b64236950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.897+0000 7f6c713d5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64236e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c713d5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ae8b0 con 0x558b63e6c070 2024-09-21T02:56:43.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c713d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ae1e0 con 0x558b63e431b0 2024-09-21T02:56:43.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c713d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641ab420 con 0x558b63e4b390 2024-09-21T02:56:43.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c713d5740 10 client.0 _ll_register_callbacks cb 0x7f6c5800bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64236e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64236e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33986/0 (socket says 172.21.15.31:33986) 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 -- 172.21.15.31:0/1538114999 learned_addr learned my addr 172.21.15.31:0/1538114999 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b64236950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64236410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 -- 172.21.15.31:0/1538114999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 msgr2=0x558b64236950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b64236950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 -- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 msgr2=0x558b64236410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64236410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 -- 172.21.15.31:0/1538114999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6429eb60 con 0x558b63e6c070 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c63fff640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64236e90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800bd70 tx=0x7f6c5800bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80b974301c3bbee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c43fff640 1 -- 172.21.15.31:0/1538114999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58028040 con 0x558b63e6c070 2024-09-21T02:56:43.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.898+0000 7f6c43fff640 1 -- 172.21.15.31:0/1538114999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6c5800b6a0 con 0x558b63e6c070 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c540041c0 con 0x558b63e6c070 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c43fff640 1 -- 172.21.15.31:0/1538114999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c58003e50 con 0x558b63e6c070 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 10 client.16470 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b6429fdf0 con 0x558b63e6c070 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c43fff640 1 -- 172.21.15.31:0/1538114999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c540041c0 con 0x558b63e6c070 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c43fff640 1 -- 172.21.15.31:0/1538114999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6c5803e020 con 0x558b63e6c070 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 10 client.16470 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 20 client.16470 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63dffda0 0x558b642a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 4 client.16470 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.899+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b64187850 con 0x558b63dffda0 2024-09-21T02:56:43.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63dffda0 0x558b642a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c68bfd640 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63dffda0 0x558b642a24a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c538b21705ac814 server_cookie=e0279aa78f9a22fb in_seq=0 out_seq=0 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c43fff640 10 client.16470 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c43fff640 1 -- 172.21.15.31:0/1538114999 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x558b64187850 con 0x558b63dffda0 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c43fff640 10 client.16470 handle_command_reply: tid=0 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c713d5740 1 client.16470 shutdown 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 msgr2=0x558b64236e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64236e90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800bd70 tx=0x7f6c5800bbd0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 shutdown_connections 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x558b63dffda0 0x558b642a24a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.900+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558b63e4b390 0x558b64236950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c43fff640 0 client.16470 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b63e6c070 0x558b64236e90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c713d5740 1 --2- 172.21.15.31:0/1538114999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b63e431b0 0x558b64236410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 >> 172.21.15.31:0/1538114999 conn(0x558b63e60460 msgr2=0x558b64214660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 shutdown_connections 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c713d5740 1 -- 172.21.15.31:0/1538114999 wait complete. 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c713d5740 20 client.16470 trim_cache size 0 max 16384 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.907 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:56:43.908 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.910 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:43.911 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6c4c0b66b0 msgr2=0x7f6c4c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6c4c0b66b0 0x7f6c4c0b8bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0caf80 tx=0x7f6c5c0095c0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 msgr2=0x7f6c6419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c6419fcc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c005cc0 tx=0x7f6c5c02f380 comp rx=0 tx=0).stop 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 shutdown_connections 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6c4c0b66b0 0x7f6c4c0b8bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c64109500 0x7f6c6419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c64109e50 0x7f6c6419fcc0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 --2- 172.21.15.31:0/3957805549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c64101fc0 0x7f6c6419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.901+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 >> 172.21.15.31:0/3957805549 conn(0x7f6c64079f90 msgr2=0x7f6c641074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.902+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 shutdown_connections 2024-09-21T02:56:43.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:43.902+0000 7f6c6ae61640 1 -- 172.21.15.31:0/3957805549 wait complete. 2024-09-21T02:56:44.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:56:44.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:56:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:44 smithi031 ceph-mon[31209]: pgmap v440: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:44 smithi165 ceph-mon[34818]: pgmap v440: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:44 smithi178 ceph-mon[34759]: pgmap v440: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 -- 172.21.15.31:0/2483247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 msgr2=0x7f34d8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 --2- 172.21.15.31:0/2483247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d8102c90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc02a410 comp rx=0 tx=0).stop 2024-09-21T02:56:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 -- 172.21.15.31:0/2483247059 shutdown_connections 2024-09-21T02:56:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 --2- 172.21.15.31:0/2483247059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d8108b10 0x7f34d810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 --2- 172.21.15.31:0/2483247059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d81081c0 0x7f34d81085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 --2- 172.21.15.31:0/2483247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d8102c90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.426+0000 7f34ddda0640 1 -- 172.21.15.31:0/2483247059 >> 172.21.15.31:0/2483247059 conn(0x7f34d80fe2d0 msgr2=0x7f34d8100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.427+0000 7f34ddda0640 1 -- 172.21.15.31:0/2483247059 shutdown_connections 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.427+0000 7f34ddda0640 1 -- 172.21.15.31:0/2483247059 wait complete. 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.427+0000 7f34ddda0640 1 Processor -- start 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 -- start start 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d81081c0 0x7f34d819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d8108b10 0x7f34d819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81127c0 con 0x7f34d8102880 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34d8112640 con 0x7f34d8108b10 2024-09-21T02:56:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34ddda0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34d8112940 con 0x7f34d81081c0 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33998/0 (socket says 172.21.15.31:33998) 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 -- 172.21.15.31:0/3279399045 learned_addr learned my addr 172.21.15.31:0/3279399045 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34d7fff640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d81081c0 0x7f34d819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dd59f640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d8108b10 0x7f34d819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 -- 172.21.15.31:0/3279399045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d81081c0 msgr2=0x7f34d819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d81081c0 0x7f34d819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 -- 172.21.15.31:0/3279399045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d8108b10 msgr2=0x7f34d819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d8108b10 0x7f34d819fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.428+0000 7f34dcd9e640 1 -- 172.21.15.31:0/3279399045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d81a0210 con 0x7f34d8102880 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.429+0000 7f34dcd9e640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d819a2a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f34cc005940 tx=0x7f34cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.429+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc03f030 con 0x7f34d8102880 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.429+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d81a04a0 con 0x7f34d8102880 2024-09-21T02:56:45.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.429+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34cc003cb0 con 0x7f34d8102880 2024-09-21T02:56:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.429+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc035070 con 0x7f34d8102880 2024-09-21T02:56:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.429+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d81a6c60 con 0x7f34d8102880 2024-09-21T02:56:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.430+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d8102c90 con 0x7f34d8102880 2024-09-21T02:56:45.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.433+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f34cc042030 con 0x7f34d8102880 2024-09-21T02:56:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.434+0000 7f34d5ffb640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34c80b6680 0x7f34c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.434+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f34cc108270 con 0x7f34d8102880 2024-09-21T02:56:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.434+0000 7f34d7fff640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34c80b6680 0x7f34c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.434+0000 7f34d7fff640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34c80b6680 0x7f34c80b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34d8104830 tx=0x7f34c400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.437+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34cc0ce780 con 0x7f34d8102880 2024-09-21T02:56:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.667+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34d81a6ec0 con 0x7f34d8102880 2024-09-21T02:56:45.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.668+0000 7f34d5ffb640 1 -- 172.21.15.31:0/3279399045 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f34cc0d2b60 con 0x7f34d8102880 2024-09-21T02:56:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:56:45.670 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:56:45.670 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:56:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34c80b6680 msgr2=0x7f34c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34c80b6680 0x7f34c80b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34d8104830 tx=0x7f34c400b390 comp rx=0 tx=0).stop 2024-09-21T02:56:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 msgr2=0x7f34d819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d819a2a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f34cc005940 tx=0x7f34cc0042f0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 shutdown_connections 2024-09-21T02:56:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f34c80b6680 0x7f34c80b8b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34d81081c0 0x7f34d819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d8108b10 0x7f34d819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 --2- 172.21.15.31:0/3279399045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34d8102880 0x7f34d819a2a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 >> 172.21.15.31:0/3279399045 conn(0x7f34d80fe2d0 msgr2=0x7f34d81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 shutdown_connections 2024-09-21T02:56:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:45.671+0000 7f34ddda0640 1 -- 172.21.15.31:0/3279399045 wait complete. 2024-09-21T02:56:46.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:45 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3279399045' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:56:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:45 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3279399045' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:56:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:45 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3279399045' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:56:46.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:56:46.762 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:46 smithi031 ceph-mon[31209]: pgmap v441: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:46 smithi165 ceph-mon[34818]: pgmap v441: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:46 smithi178 ceph-mon[34759]: pgmap v441: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:47.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 -- 172.21.15.31:0/376837455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac108420 msgr2=0x7fa1ac108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/376837455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac108420 0x7fa1ac108830 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c02a410 comp rx=0 tx=0).stop 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 -- 172.21.15.31:0/376837455 shutdown_connections 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/376837455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ac108d70 0x7fa1ac10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/376837455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac108420 0x7fa1ac108830 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/376837455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 -- 172.21.15.31:0/376837455 >> 172.21.15.31:0/376837455 conn(0x7fa1ac079770 msgr2=0x7fa1ac079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 -- 172.21.15.31:0/376837455 shutdown_connections 2024-09-21T02:56:47.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 -- 172.21.15.31:0/376837455 wait complete. 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.266+0000 7fa1b2d10640 1 Processor -- start 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 -- start start 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac108420 0x7fa1ac19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ac108d70 0x7fa1ac19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac112a30 con 0x7fa1ac102ae0 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1128b0 con 0x7fa1ac108420 2024-09-21T02:56:47.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b2d10640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac112bb0 con 0x7fa1ac108d70 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34010/0 (socket says 172.21.15.31:34010) 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 -- 172.21.15.31:0/3961733697 learned_addr learned my addr 172.21.15.31:0/3961733697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ac108d70 0x7fa1ac19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac108420 0x7fa1ac19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 -- 172.21.15.31:0/3961733697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ac108d70 msgr2=0x7fa1ac19be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ac108d70 0x7fa1ac19be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 -- 172.21.15.31:0/3961733697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac108420 msgr2=0x7fa1ac19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac108420 0x7fa1ac19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1b0aac640 1 -- 172.21.15.31:0/3961733697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac1a6650 con 0x7fa1ac102ae0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.267+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac108420 0x7fa1ac19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.268+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac19b3c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800c890 tx=0x7fa1a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.268+0000 7fa1a1ffb640 1 -- 172.21.15.31:0/3961733697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8004220 con 0x7fa1ac102ae0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.268+0000 7fa1a1ffb640 1 -- 172.21.15.31:0/3961733697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a80043c0 con 0x7fa1ac102ae0 2024-09-21T02:56:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.268+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac1a6940 con 0x7fa1ac102ae0 2024-09-21T02:56:47.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.268+0000 7fa1a1ffb640 1 -- 172.21.15.31:0/3961733697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8004a00 con 0x7fa1ac102ae0 2024-09-21T02:56:47.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.268+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac1a6ed0 con 0x7fa1ac102ae0 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 Processor -- start 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1a1ffb640 1 -- 172.21.15.31:0/3961733697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa1a80079b0 con 0x7fa1ac102ae0 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 -- start start 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9f3070 0x55ef5dd9a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dd9abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 0x55ef5dd9b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ef5dd10ea0 con 0x55ef5d9d2390 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ef5dd002d0 con 0x55ef5d9f3070 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b9284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef5dd040e0 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 0x55ef5dd9b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1b0aac640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dd9abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 0x55ef5dd9b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34018/0 (socket says 172.21.15.31:34018) 2024-09-21T02:56:47.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.271+0000 7fa1a3fff640 1 -- 172.21.15.31:0/2295810552 learned_addr learned my addr 172.21.15.31:0/2295810552 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:47.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9f3070 0x55ef5dd9a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a3fff640 1 -- 172.21.15.31:0/2295810552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9ca1b0 msgr2=0x55ef5dd9abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dd9abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a3fff640 1 -- 172.21.15.31:0/2295810552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9f3070 msgr2=0x55ef5dd9a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9f3070 0x55ef5dd9a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a3fff640 1 -- 172.21.15.31:0/2295810552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef5da5e650 con 0x55ef5d9d2390 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a1ffb640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa1940b6730 0x7fa1940b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 0x55ef5dd9b120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa19c000c00 tx=0x7fa19c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812e8e632a5421ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa1940b6730 0x7fa1940b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa1a1ffb640 1 -- 172.21.15.31:0/3961733697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa1a80e7470 con 0x7fa1ac102ae0 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa192ffd640 1 -- 172.21.15.31:0/2295810552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c03f070 con 0x55ef5d9d2390 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa192ffd640 1 -- 172.21.15.31:0/2295810552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0316a0 con 0x55ef5d9d2390 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.272+0000 7fa192ffd640 1 -- 172.21.15.31:0/2295810552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c035070 con 0x55ef5d9d2390 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 -- 172.21.15.31:0/2295810552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 msgr2=0x55ef5dd9b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa1940b6730 0x7fa1940b8c40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40092f0 tx=0x7fa1a4009280 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 0x55ef5dd9b120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa19c000c00 tx=0x7fa19c032ce0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 -- 172.21.15.31:0/2295810552 shutdown_connections 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dd9abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9d2390 0x55ef5dd9b120 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2295810552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9f3070 0x55ef5dd9a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 -- 172.21.15.31:0/2295810552 >> 172.21.15.31:0/2295810552 conn(0x55ef5d9e7460 msgr2=0x55ef5dd98590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 -- 172.21.15.31:0/2295810552 shutdown_connections 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.273+0000 7fa1b9284740 1 -- 172.21.15.31:0/2295810552 wait complete. 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 -- start start 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 Processor -- start 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dda5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 0x55ef5dda66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ef5db61700 con 0x55ef5d9f3070 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ef5dc9b160 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef5dda8e20 con 0x55ef5d9d2390 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.274+0000 7fa1b9284740 10 client.0 _ll_register_callbacks cb 0x55ef5dcb49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 0x55ef5dda66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 0x55ef5dda66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34028/0 (socket says 172.21.15.31:34028) 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 -- 172.21.15.31:0/615605973 learned_addr learned my addr 172.21.15.31:0/615605973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dda5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 -- 172.21.15.31:0/615605973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 msgr2=0x55ef5dda61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 -- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 msgr2=0x55ef5dda5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dda5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 -- 172.21.15.31:0/615605973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef5dcc8c90 con 0x55ef5d9f3070 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 0x55ef5dda66e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4012870 tx=0x7fa1a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d972882c76cd91e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa18bfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:47.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa18bfff640 1 -- 172.21.15.31:0/615605973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400ead0 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa18bfff640 1 -- 172.21.15.31:0/615605973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa1a400ec70 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.275+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19c0316a0 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa18bfff640 1 -- 172.21.15.31:0/615605973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1a401f420 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 10 client.16488 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef5ddb4a90 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa18bfff640 1 -- 172.21.15.31:0/615605973 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0316a0 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa18bfff640 1 -- 172.21.15.31:0/615605973 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa1a400f040 con 0x55ef5d9f3070 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 10 client.16488 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 10 client.16488 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 20 client.16488 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9b19f0 0x55ef5ddb7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 4 client.16488 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef5dd12df0 con 0x55ef5d9b19f0 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.276+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9b19f0 0x55ef5ddb7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.277+0000 7fa18bfff640 10 client.16488 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.277+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9b19f0 0x55ef5ddb7fb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eb5216be8761d215 server_cookie=8b90d680f5a45efe in_seq=0 out_seq=0 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa18bfff640 1 -- 172.21.15.31:0/615605973 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef5dd12df0 con 0x55ef5d9b19f0 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa18bfff640 10 client.16488 handle_command_reply: tid=0 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 client.16488 shutdown 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 msgr2=0x55ef5dda66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 0x55ef5dda66e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4012870 tx=0x7fa1a4004a40 comp rx=0 tx=0).stop 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 shutdown_connections 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9b19f0 0x55ef5ddb7fb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa18bfff640 0 client.16488 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9f3070 0x55ef5dda66e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 --2- 172.21.15.31:0/615605973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5dda5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 >> 172.21.15.31:0/615605973 conn(0x55ef5d9e7460 msgr2=0x55ef5dd9d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 shutdown_connections 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 1 -- 172.21.15.31:0/615605973 wait complete. 2024-09-21T02:56:47.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.278+0000 7fa1b9284740 20 client.16488 trim_cache size 0 max 16384 2024-09-21T02:56:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.291+0000 7fa1b9284740 1 Processor -- start 2024-09-21T02:56:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.291+0000 7fa1b9284740 1 -- start start 2024-09-21T02:56:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 0x55ef5dda7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddb2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b9284740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1af7b0 con 0x55ef5d9b19f0 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b9284740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1b12f0 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b9284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1afe80 con 0x55ef5d9d2390 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 0x55ef5dda7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 0x55ef5dda7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34030/0 (socket says 172.21.15.31:34030) 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 -- 172.21.15.31:0/2662390297 learned_addr learned my addr 172.21.15.31:0/2662390297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddb2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 -- 172.21.15.31:0/2662390297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 msgr2=0x55ef5dda2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 -- 172.21.15.31:0/2662390297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 msgr2=0x55ef5ddb2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddb2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 -- 172.21.15.31:0/2662390297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef5dd98ad0 con 0x55ef5d9b19f0 2024-09-21T02:56:47.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddb2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:47.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:47.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 0x55ef5dda7890 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80e4c40 tx=0x7fa1a80ec040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba34028a706fbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1927fc640 1 -- 172.21.15.31:0/2662390297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a801d070 con 0x55ef5d9b19f0 2024-09-21T02:56:47.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.292+0000 7fa1927fc640 1 -- 172.21.15.31:0/2662390297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a8005290 con 0x55ef5d9b19f0 2024-09-21T02:56:47.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1927fc640 1 -- 172.21.15.31:0/2662390297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80b1dc0 con 0x55ef5d9b19f0 2024-09-21T02:56:47.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 -- 172.21.15.31:0/2662390297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 msgr2=0x55ef5dda7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 0x55ef5dda7890 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80e4c40 tx=0x7fa1a80ec040 comp rx=0 tx=0).stop 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 -- 172.21.15.31:0/2662390297 shutdown_connections 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9d2390 0x55ef5dda2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddb2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 --2- 172.21.15.31:0/2662390297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9b19f0 0x55ef5dda7890 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 -- 172.21.15.31:0/2662390297 >> 172.21.15.31:0/2662390297 conn(0x55ef5d9e7460 msgr2=0x55ef5dda7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 -- 172.21.15.31:0/2662390297 shutdown_connections 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.293+0000 7fa1b9284740 1 -- 172.21.15.31:0/2662390297 wait complete. 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 -- start start 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 Processor -- start 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9b19f0 0x55ef5ddaa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddaa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9d2390 0x55ef5ddaab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1b0c20 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1b0550 con 0x55ef5d9d2390 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1ad790 con 0x55ef5d9b19f0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1b9284740 10 client.0 _ll_register_callbacks cb 0x7fa1a4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddaa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.294+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddaa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34034/0 (socket says 172.21.15.31:34034) 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 -- 172.21.15.31:0/1422502973 learned_addr learned my addr 172.21.15.31:0/1422502973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9b19f0 0x55ef5ddaa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1b12ad640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9d2390 0x55ef5ddaab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 -- 172.21.15.31:0/1422502973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9b19f0 msgr2=0x55ef5ddaa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9b19f0 0x55ef5ddaa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 -- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9d2390 msgr2=0x55ef5ddaab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9d2390 0x55ef5ddaab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 -- 172.21.15.31:0/1422502973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef5de25e70 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9b19f0 0x55ef5ddaa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1a3fff640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddaa630 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa19c033000 tx=0x7fa19c0055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50b0959dafd3d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa18bfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa18bfff640 1 -- 172.21.15.31:0/1422502973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c03f040 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa18bfff640 1 -- 172.21.15.31:0/1422502973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa19c0318b0 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.295+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef5dd0e8a0 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa18bfff640 1 -- 172.21.15.31:0/1422502973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa19c035040 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 10 client.16494 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef5de27060 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa18bfff640 1 -- 172.21.15.31:0/1422502973 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ef5dd0e8a0 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa18bfff640 1 -- 172.21.15.31:0/1422502973 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa19c053020 con 0x55ef5d9ca1b0 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 10 client.16494 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 10 client.16494 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:56:47.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 20 client.16494 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9f3070 0x55ef5de29400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 4 client.16494 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef5dd0bcd0 con 0x55ef5d9f3070 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.296+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9f3070 0x55ef5de29400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa18bfff640 10 client.16494 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b0aac640 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9f3070 0x55ef5de29400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eac47737e5cfe123 server_cookie=901342ba00b8d718 in_seq=0 out_seq=0 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa18bfff640 1 -- 172.21.15.31:0/1422502973 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef5dd0bcd0 con 0x55ef5d9f3070 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa18bfff640 10 client.16494 handle_command_reply: tid=0 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 client.16494 shutdown 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 msgr2=0x55ef5ddaa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddaa630 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa19c033000 tx=0x7fa19c0055b0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 shutdown_connections 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ef5d9d2390 0x55ef5ddaab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ef5d9ca1b0 0x55ef5ddaa630 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55ef5d9f3070 0x55ef5de29400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 --2- 172.21.15.31:0/1422502973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef5d9b19f0 0x55ef5ddaa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa18bfff640 0 client.16494 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 >> 172.21.15.31:0/1422502973 conn(0x55ef5d9e7460 msgr2=0x55ef5dda70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 shutdown_connections 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 1 -- 172.21.15.31:0/1422502973 wait complete. 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.297+0000 7fa1b9284740 20 client.16494 trim_cache size 0 max 16384 2024-09-21T02:56:47.304 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa1940b6730 msgr2=0x7fa1940b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa1940b6730 0x7fa1940b8c40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40092f0 tx=0x7fa1a4009280 comp rx=0 tx=0).stop 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 msgr2=0x7fa1ac19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac19b3c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800c890 tx=0x7fa1a800cd50 comp rx=0 tx=0).stop 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 shutdown_connections 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa1940b6730 0x7fa1940b8c40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ac108d70 0x7fa1ac19be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1ac108420 0x7fa1ac19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 --2- 172.21.15.31:0/3961733697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac102ae0 0x7fa1ac19b3c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 >> 172.21.15.31:0/3961733697 conn(0x7fa1ac079770 msgr2=0x7fa1ac106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 shutdown_connections 2024-09-21T02:56:47.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:47.298+0000 7fa1b2d10640 1 -- 172.21.15.31:0/3961733697 wait complete. 2024-09-21T02:56:48.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:56:48.387 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.874+0000 7f612aa5e640 1 -- 172.21.15.31:0/3895150337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61241094a0 msgr2=0x7f61241098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.874+0000 7f612aa5e640 1 --2- 172.21.15.31:0/3895150337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61241094a0 0x7f61241098b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6118009900 tx=0x7f611802a410 comp rx=0 tx=0).stop 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 -- 172.21.15.31:0/3895150337 shutdown_connections 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 --2- 172.21.15.31:0/3895150337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6124109df0 0x7f6124110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 --2- 172.21.15.31:0/3895150337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61241094a0 0x7f61241098b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 --2- 172.21.15.31:0/3895150337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 0x7f6124103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 -- 172.21.15.31:0/3895150337 >> 172.21.15.31:0/3895150337 conn(0x7f612407b850 msgr2=0x7f612407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 -- 172.21.15.31:0/3895150337 shutdown_connections 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 -- 172.21.15.31:0/3895150337 wait complete. 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 Processor -- start 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.875+0000 7f612aa5e640 1 -- start start 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f612aa5e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 0x7f612419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f612aa5e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61241094a0 0x7f612419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f612aa5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 0x7f612419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f612aa5e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113ab0 con 0x7f6124109df0 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f612aa5e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113930 con 0x7f61241094a0 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f612aa5e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6124113c30 con 0x7f6124103b60 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 0x7f612419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 0x7f612419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34042/0 (socket says 172.21.15.31:34042) 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 -- 172.21.15.31:0/2930361633 learned_addr learned my addr 172.21.15.31:0/2930361633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f61237fe640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61241094a0 0x7f612419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6123fff640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 0x7f612419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 -- 172.21.15.31:0/2930361633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 msgr2=0x7f612419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 0x7f612419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 -- 172.21.15.31:0/2930361633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61241094a0 msgr2=0x7f612419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61241094a0 0x7f612419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 -- 172.21.15.31:0/2930361633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61241a0410 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6123fff640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 0x7f612419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.876+0000 7f6128ffb640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 0x7f612419fce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f611c00eaa0 tx=0x7f611c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.877+0000 7f61217fa640 1 -- 172.21.15.31:0/2930361633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00cdb0 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.877+0000 7f61217fa640 1 -- 172.21.15.31:0/2930361633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c005020 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.877+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61241a0700 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.877+0000 7f61217fa640 1 -- 172.21.15.31:0/2930361633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0116d0 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.877+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a6e60 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.879+0000 7f6130fd2740 1 Processor -- start 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f61217fa640 1 -- 172.21.15.31:0/2930361633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f611c0051d0 con 0x7f6124109df0 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 -- start start 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 0x55d8084b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d80842fea0 con 0x55d8080e91b0 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d80841f2d0 con 0x55d808112070 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6130fd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d8084230e0 con 0x55d8080f1390 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 0x55d8084b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 0x55d8084b9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34048/0 (socket says 172.21.15.31:34048) 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f6123fff640 1 -- 172.21.15.31:0/1283457839 learned_addr learned my addr 172.21.15.31:0/1283457839 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f61237fe640 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f61217fa640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f610c0b6700 0x7f610c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.880+0000 7f61237fe640 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6128ffb640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f610c0b6700 0x7f610c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6123fff640 1 -- 172.21.15.31:0/1283457839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 msgr2=0x55d8084b9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6123fff640 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084b9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6123fff640 1 -- 172.21.15.31:0/1283457839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 msgr2=0x55d8084b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6123fff640 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084b92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6123fff640 1 -- 172.21.15.31:0/1283457839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d80817d650 con 0x55d8080e91b0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f61217fa640 1 -- 172.21.15.31:0/2930361633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f611c0e7850 con 0x7f6124109df0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6123fff640 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 0x55d8084b9800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f611400bd40 tx=0x7f611400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73c0c4591a5d552 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f6128ffb640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f610c0b6700 0x7f610c0b8c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f611c020000 tx=0x7f611c077110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f610a7fc640 1 -- 172.21.15.31:0/1283457839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61140149e0 con 0x55d8080e91b0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f610a7fc640 1 -- 172.21.15.31:0/1283457839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611400be30 con 0x55d8080e91b0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.881+0000 7f610a7fc640 1 -- 172.21.15.31:0/1283457839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611401ce50 con 0x55d8080e91b0 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 -- 172.21.15.31:0/1283457839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 msgr2=0x55d8084b9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 0x55d8084b9800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f611400bd40 tx=0x7f611400f730 comp rx=0 tx=0).stop 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 -- 172.21.15.31:0/1283457839 shutdown_connections 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084b9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080e91b0 0x55d8084b9800 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1283457839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 -- 172.21.15.31:0/1283457839 >> 172.21.15.31:0/1283457839 conn(0x55d808106460 msgr2=0x55d8084b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:48.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 -- 172.21.15.31:0/1283457839 shutdown_connections 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 -- 172.21.15.31:0/1283457839 wait complete. 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.882+0000 7f6130fd2740 1 -- start start 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 Processor -- start 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 0x55d8084c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d808280700 con 0x55d808112070 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d8083ba160 con 0x55d8080e91b0 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d8084c7aa0 con 0x55d8080f1390 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6130fd2740 10 client.0 _ll_register_callbacks cb 0x55d8083d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f61237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 0x55d8084c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f61237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 0x55d8084c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34054/0 (socket says 172.21.15.31:34054) 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f61237fe640 1 -- 172.21.15.31:0/1332988091 learned_addr learned my addr 172.21.15.31:0/1332988091 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.883+0000 7f6123fff640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 -- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 msgr2=0x55d8084c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 -- 172.21.15.31:0/1332988091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 msgr2=0x55d8084c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 -- 172.21.15.31:0/1332988091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8083e7c90 con 0x55d808112070 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61237fe640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 0x55d8084c5370 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f61180096c0 tx=0x7f611802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2347546c32d252b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61037fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61037fe640 1 -- 172.21.15.31:0/1332988091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118035040 con 0x55d808112070 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61037fe640 1 -- 172.21.15.31:0/1332988091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6118004240 con 0x55d808112070 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611400be30 con 0x55d808112070 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61037fe640 1 -- 172.21.15.31:0/1332988091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6118004440 con 0x55d808112070 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f6130fd2740 10 client.16506 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8084d3730 con 0x55d808112070 2024-09-21T02:56:48.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.884+0000 7f61037fe640 1 -- 172.21.15.31:0/1332988091 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611400be30 con 0x55d808112070 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f61037fe640 1 -- 172.21.15.31:0/1332988091 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6118049060 con 0x55d808112070 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6130fd2740 10 client.16506 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6130fd2740 10 client.16506 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6130fd2740 20 client.16506 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080d09f0 0x55d8084d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6130fd2740 4 client.16506 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d808431df0 con 0x55d8080d09f0 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.885+0000 7f6123fff640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080d09f0 0x55d8084d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.886+0000 7f61037fe640 10 client.16506 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.886+0000 7f6123fff640 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080d09f0 0x55d8084d6c60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9271231439aa032f server_cookie=e17f1cd2ba81c5c9 in_seq=0 out_seq=0 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.886+0000 7f61037fe640 1 -- 172.21.15.31:0/1332988091 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d808431df0 con 0x55d8080d09f0 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.886+0000 7f61037fe640 10 client.16506 handle_command_reply: tid=0 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.886+0000 7f6130fd2740 1 client.16506 shutdown 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 msgr2=0x55d8084c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 0x55d8084c5370 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f61180096c0 tx=0x7f611802a410 comp rx=0 tx=0).stop 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 shutdown_connections 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080d09f0 0x55d8084d6c60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080f1390 0x55d8084c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d808112070 0x55d8084c5370 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f61037fe640 0 client.16506 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1332988091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 >> 172.21.15.31:0/1332988091 conn(0x55d808106460 msgr2=0x55d8084bc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 shutdown_connections 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 1 -- 172.21.15.31:0/1332988091 wait complete. 2024-09-21T02:56:48.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.887+0000 7f6130fd2740 20 client.16506 trim_cache size 0 max 16384 2024-09-21T02:56:48.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 Processor -- start 2024-09-21T02:56:48.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 -- start start 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080e91b0 0x55d8084baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084c2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ad4c0 con 0x55d8080f1390 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61241af000 con 0x55d808112070 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.900+0000 7f6130fd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61241adb90 con 0x55d8080e91b0 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34068/0 (socket says 172.21.15.31:34068) 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 -- 172.21.15.31:0/3325996491 learned_addr learned my addr 172.21.15.31:0/3325996491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f61237fe640 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080e91b0 0x55d8084baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f61237fe640 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 -- 172.21.15.31:0/3325996491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080e91b0 msgr2=0x55d8084baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080e91b0 0x55d8084baa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 -- 172.21.15.31:0/3325996491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 msgr2=0x55d8084c2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084c2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 -- 172.21.15.31:0/3325996491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8084b1440 con 0x55d8080f1390 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f61237fe640 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6123fff640 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c1ae0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f611401cac0 tx=0x7f611400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e67203e2f76362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6109ffb640 1 -- 172.21.15.31:0/3325996491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611400be30 con 0x55d8080f1390 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6109ffb640 1 -- 172.21.15.31:0/3325996491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6114034070 con 0x55d8080f1390 2024-09-21T02:56:48.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.901+0000 7f6109ffb640 1 -- 172.21.15.31:0/3325996491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114014ab0 con 0x55d8080f1390 2024-09-21T02:56:48.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 -- 172.21.15.31:0/3325996491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 msgr2=0x55d8084c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c1ae0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f611401cac0 tx=0x7f611400f730 comp rx=0 tx=0).stop 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 -- 172.21.15.31:0/3325996491 shutdown_connections 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d808112070 0x55d8084c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c1ae0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 --2- 172.21.15.31:0/3325996491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8080e91b0 0x55d8084baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 -- 172.21.15.31:0/3325996491 >> 172.21.15.31:0/3325996491 conn(0x55d808106460 msgr2=0x55d8084ba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 -- 172.21.15.31:0/3325996491 shutdown_connections 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.902+0000 7f6130fd2740 1 -- 172.21.15.31:0/3325996491 wait complete. 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 -- start start 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 Processor -- start 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d808112070 0x55d8084c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ae930 con 0x55d8080f1390 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ae260 con 0x55d8080e91b0 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ab4a0 con 0x55d808112070 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6130fd2740 10 client.0 _ll_register_callbacks cb 0x7f6118005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34070/0 (socket says 172.21.15.31:34070) 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.903+0000 7f6123fff640 1 -- 172.21.15.31:0/1959294671 learned_addr learned my addr 172.21.15.31:0/1959294671 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61237fe640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d808112070 0x55d8084c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61237fe640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6123fff640 1 -- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d808112070 msgr2=0x55d8084c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6123fff640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d808112070 0x55d8084c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6123fff640 1 -- 172.21.15.31:0/1959294671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 msgr2=0x55d8084cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6123fff640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6123fff640 1 -- 172.21.15.31:0/1959294671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d808540620 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61237fe640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6123fff640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c8d10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6114010010 tx=0x7f611400fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb4e47013330be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61037fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61037fe640 1 -- 172.21.15.31:0/1959294671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611403c070 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61037fe640 1 -- 172.21.15.31:0/1959294671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f611400be30 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6114034070 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f61037fe640 1 -- 172.21.15.31:0/1959294671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6114048050 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6130fd2740 10 client.16512 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.904+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d808541720 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f61037fe640 1 -- 172.21.15.31:0/1959294671 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6114034070 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f61037fe640 1 -- 172.21.15.31:0/1959294671 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6114005040 con 0x55d8080f1390 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f6130fd2740 10 client.16512 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f6130fd2740 10 client.16512 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f6130fd2740 20 client.16512 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:48.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080a5da0 0x55d808543c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:48.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f6130fd2740 4 client.16512 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:48.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d80842d8a0 con 0x55d8080a5da0 2024-09-21T02:56:48.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.905+0000 7f61237fe640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080a5da0 0x55d808543c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f61037fe640 10 client.16512 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f61237fe640 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080a5da0 0x55d808543c60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28c2eb5406c4af64 server_cookie=54eb7cf50d04db86 in_seq=0 out_seq=0 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f61037fe640 1 -- 172.21.15.31:0/1959294671 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d80842d8a0 con 0x55d8080a5da0 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f61037fe640 10 client.16512 handle_command_reply: tid=0 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 client.16512 shutdown 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 msgr2=0x55d8084c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c8d10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6114010010 tx=0x7f611400fdd0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 shutdown_connections 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55d8080a5da0 0x55d808543c60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d808112070 0x55d8084c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d8080f1390 0x55d8084c8d10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f61037fe640 0 client.16512 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:56:48.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 --2- 172.21.15.31:0/1959294671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8080e91b0 0x55d8084cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 >> 172.21.15.31:0/1959294671 conn(0x55d808106460 msgr2=0x55d8084cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 shutdown_connections 2024-09-21T02:56:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.906+0000 7f6130fd2740 1 -- 172.21.15.31:0/1959294671 wait complete. 2024-09-21T02:56:48.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f6130fd2740 20 client.16512 trim_cache size 0 max 16384 2024-09-21T02:56:48.912 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f610c0b6700 msgr2=0x7f610c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f610c0b6700 0x7f610c0b8c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f611c020000 tx=0x7f611c077110 comp rx=0 tx=0).stop 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 msgr2=0x7f612419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 0x7f612419fce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f611c00eaa0 tx=0x7f611c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 shutdown_connections 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f610c0b6700 0x7f610c0b8c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6124109df0 0x7f612419fce0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61241094a0 0x7f612419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 --2- 172.21.15.31:0/2930361633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6124103b60 0x7f612419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 >> 172.21.15.31:0/2930361633 conn(0x7f612407b850 msgr2=0x7f6124107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 shutdown_connections 2024-09-21T02:56:48.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:48.907+0000 7f612aa5e640 1 -- 172.21.15.31:0/2930361633 wait complete. 2024-09-21T02:56:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:48 smithi031 ceph-mon[31209]: pgmap v442: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:48 smithi165 ceph-mon[34818]: pgmap v442: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:48 smithi178 ceph-mon[34759]: pgmap v442: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:56:49.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:56:49.964 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 -- 172.21.15.31:0/554535796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc103bc0 msgr2=0x7f61dc103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 --2- 172.21.15.31:0/554535796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc103bc0 0x7f61dc103fd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc02a410 comp rx=0 tx=0).stop 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 -- 172.21.15.31:0/554535796 shutdown_connections 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 --2- 172.21.15.31:0/554535796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc1094b0 0x7f61dc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 --2- 172.21.15.31:0/554535796 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc109e00 0x7f61dc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 --2- 172.21.15.31:0/554535796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc103bc0 0x7f61dc103fd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 -- 172.21.15.31:0/554535796 >> 172.21.15.31:0/554535796 conn(0x7f61dc07b910 msgr2=0x7f61dc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 -- 172.21.15.31:0/554535796 shutdown_connections 2024-09-21T02:56:50.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.447+0000 7f61e1d99640 1 -- 172.21.15.31:0/554535796 wait complete. 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 Processor -- start 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 -- start start 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc103bc0 0x7f61dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 0x7f61dc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc109e00 0x7f61dc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc113ac0 con 0x7f61dc1094b0 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc113940 con 0x7f61dc103bc0 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61e1d99640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc113c40 con 0x7f61dc109e00 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 0x7f61dc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 0x7f61dc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35034/0 (socket says 172.21.15.31:35034) 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.448+0000 7f61d2ffd640 1 -- 172.21.15.31:0/1156298082 learned_addr learned my addr 172.21.15.31:0/1156298082 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61dbfff640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc109e00 0x7f61dc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61db7fe640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc103bc0 0x7f61dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d2ffd640 1 -- 172.21.15.31:0/1156298082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc109e00 msgr2=0x7f61dc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc109e00 0x7f61dc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d2ffd640 1 -- 172.21.15.31:0/1156298082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc103bc0 msgr2=0x7f61dc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc103bc0 0x7f61dc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d2ffd640 1 -- 172.21.15.31:0/1156298082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a86c0 con 0x7f61dc1094b0 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61dbfff640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc109e00 0x7f61dc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61db7fe640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc103bc0 0x7f61dc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 0x7f61dc075c50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c8c0 tx=0x7f61c400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d97fa640 1 -- 172.21.15.31:0/1156298082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4007bb0 con 0x7f61dc1094b0 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a89b0 con 0x7f61dc1094b0 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d97fa640 1 -- 172.21.15.31:0/1156298082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c4007d50 con 0x7f61dc1094b0 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61d97fa640 1 -- 172.21.15.31:0/1156298082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40187c0 con 0x7f61dc1094b0 2024-09-21T02:56:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.449+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1a8f40 con 0x7f61dc1094b0 2024-09-21T02:56:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 Processor -- start 2024-09-21T02:56:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 -- start start 2024-09-21T02:56:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61d97fa640 1 -- 172.21.15.31:0/1156298082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f61c400fa20 con 0x7f61dc1094b0 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c89496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8949be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c894a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0c88bfea0 con 0x55e0c8581390 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0c88af2d0 con 0x55e0c85791b0 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61e830d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0c88b30e0 con 0x55e0c85a2070 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.452+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c894a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61db7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8949be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61db7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8949be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:52594/0 (socket says 172.21.15.31:52594) 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c894a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35042/0 (socket says 172.21.15.31:35042) 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2938415505 learned_addr learned my addr 172.21.15.31:0/2938415505 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61dbfff640 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c89496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2938415505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 msgr2=0x55e0c89496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c89496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2938415505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 msgr2=0x55e0c8949be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8949be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2938415505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c860d650 con 0x55e0c8581390 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61db7fe640 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8949be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c894a120 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f61c40e4fb0 tx=0x7f61c40edcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaf9b235d297498 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.453+0000 7f61d97fa640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61c80b6680 0x7f61c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.454+0000 7f61dbfff640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61c80b6680 0x7f61c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.454+0000 7f61d97fa640 1 -- 172.21.15.31:0/1156298082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f61c40e78d0 con 0x7f61dc1094b0 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.454+0000 7f61d1ffb640 1 -- 172.21.15.31:0/2938415505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40f6070 con 0x55e0c8581390 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.454+0000 7f61d1ffb640 1 -- 172.21.15.31:0/2938415505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c40ad250 con 0x55e0c8581390 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.454+0000 7f61d1ffb640 1 -- 172.21.15.31:0/2938415505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40ad510 con 0x55e0c8581390 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 -- 172.21.15.31:0/2938415505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 msgr2=0x55e0c894a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61dbfff640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61c80b6680 0x7f61c80b8b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61d400af80 tx=0x7f61d400a730 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c894a120 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f61c40e4fb0 tx=0x7f61c40edcd0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 -- 172.21.15.31:0/2938415505 shutdown_connections 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c894a120 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8949be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 --2- 172.21.15.31:0/2938415505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c89496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 -- 172.21.15.31:0/2938415505 >> 172.21.15.31:0/2938415505 conn(0x55e0c8596460 msgr2=0x55e0c8947590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 -- 172.21.15.31:0/2938415505 shutdown_connections 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 -- 172.21.15.31:0/2938415505 wait complete. 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.455+0000 7f61e830d740 1 -- start start 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 Processor -- start 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8954d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8955250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8961170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0c8710700 con 0x55e0c8581390 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0c884a160 con 0x55e0c85791b0 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0c8957e80 con 0x55e0c85a2070 2024-09-21T02:56:50.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61e830d740 10 client.0 _ll_register_callbacks cb 0x55e0c88639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8955250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8955250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35050/0 (socket says 172.21.15.31:35050) 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2810457197 learned_addr learned my addr 172.21.15.31:0/2810457197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61db7fe640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8954d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61dbfff640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8961170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 msgr2=0x55e0c8961170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8961170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2810457197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 msgr2=0x55e0c8954d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.456+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8954d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61d2ffd640 1 -- 172.21.15.31:0/2810457197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c8877c90 con 0x55e0c8581390 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8955250 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ed1b0 tx=0x7f61c400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5ee24d613bc3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61baffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61baffd640 1 -- 172.21.15.31:0/2810457197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40f6070 con 0x55e0c8581390 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61baffd640 1 -- 172.21.15.31:0/2810457197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f61c40b6cd0 con 0x55e0c8581390 2024-09-21T02:56:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c40ad250 con 0x55e0c8581390 2024-09-21T02:56:50.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61baffd640 1 -- 172.21.15.31:0/2810457197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61c40aceb0 con 0x55e0c8581390 2024-09-21T02:56:50.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61e830d740 10 client.16524 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:50.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0c8963b50 con 0x55e0c8581390 2024-09-21T02:56:50.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61baffd640 1 -- 172.21.15.31:0/2810457197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c40ad250 con 0x55e0c8581390 2024-09-21T02:56:50.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.457+0000 7f61baffd640 1 -- 172.21.15.31:0/2810457197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61c40a9090 con 0x55e0c8581390 2024-09-21T02:56:50.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61e830d740 10 client.16524 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:50.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61e830d740 10 client.16524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:56:50.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61e830d740 20 client.16524 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61e830d740 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c85609f0 0x55e0c8966eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61e830d740 4 client.16524 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c88c1df0 con 0x55e0c85609f0 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61db7fe640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c85609f0 0x55e0c8966eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61baffd640 10 client.16524 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.458+0000 7f61db7fe640 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c85609f0 0x55e0c8966eb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a73d3b18acfb4a7 server_cookie=6a85398f76f8c77d in_seq=0 out_seq=0 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61baffd640 1 -- 172.21.15.31:0/2810457197 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0c88c1df0 con 0x55e0c85609f0 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61baffd640 10 client.16524 handle_command_reply: tid=0 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 client.16524 shutdown 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 msgr2=0x55e0c8955250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8955250 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ed1b0 tx=0x7f61c400fd10 comp rx=0 tx=0).stop 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 shutdown_connections 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c85609f0 0x55e0c8966eb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8961170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61baffd640 0 client.16524 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:50.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8955250 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 --2- 172.21.15.31:0/2810457197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8954d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.459+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 >> 172.21.15.31:0/2810457197 conn(0x55e0c8596460 msgr2=0x55e0c894c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.460+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 shutdown_connections 2024-09-21T02:56:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.460+0000 7f61e830d740 1 -- 172.21.15.31:0/2810457197 wait complete. 2024-09-21T02:56:50.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.460+0000 7f61e830d740 20 client.16524 trim_cache size 0 max 16384 2024-09-21T02:56:50.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 Processor -- start 2024-09-21T02:56:50.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 -- start start 2024-09-21T02:56:50.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8956940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8946bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8947100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b18a0 con 0x55e0c8581390 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b33e0 con 0x55e0c85791b0 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61e830d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b1f70 con 0x55e0c85a2070 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8946bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8946bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35056/0 (socket says 172.21.15.31:35056) 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.473+0000 7f61d2ffd640 1 -- 172.21.15.31:0/3361451258 learned_addr learned my addr 172.21.15.31:0/3361451258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8947100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61db7fe640 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8956940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d2ffd640 1 -- 172.21.15.31:0/3361451258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 msgr2=0x55e0c8947100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8947100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d2ffd640 1 -- 172.21.15.31:0/3361451258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 msgr2=0x55e0c8956940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8956940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d2ffd640 1 -- 172.21.15.31:0/3361451258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c896d1a0 con 0x55e0c8581390 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61db7fe640 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8956940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8946bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f61c40a9d10 tx=0x7f61c40edcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a465c8199ad399b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d17fa640 1 -- 172.21.15.31:0/3361451258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40f6070 con 0x55e0c8581390 2024-09-21T02:56:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d17fa640 1 -- 172.21.15.31:0/3361451258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c40ad250 con 0x55e0c8581390 2024-09-21T02:56:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.474+0000 7f61d17fa640 1 -- 172.21.15.31:0/3361451258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c405c070 con 0x55e0c8581390 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 -- 172.21.15.31:0/3361451258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 msgr2=0x55e0c8946bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8946bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f61c40a9d10 tx=0x7f61c40edcd0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 -- 172.21.15.31:0/3361451258 shutdown_connections 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c85a2070 0x55e0c8947100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c8581390 0x55e0c8946bc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 --2- 172.21.15.31:0/3361451258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c8956940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 -- 172.21.15.31:0/3361451258 >> 172.21.15.31:0/3361451258 conn(0x55e0c8596460 msgr2=0x55e0c8956230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 -- 172.21.15.31:0/3361451258 shutdown_connections 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 -- 172.21.15.31:0/3361451258 wait complete. 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.475+0000 7f61e830d740 1 -- start start 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 Processor -- start 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c896a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c8581390 0x55e0c8967e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 0x55e0c8968520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b2d10 con 0x55e0c85a2070 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b2640 con 0x55e0c85791b0 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1af880 con 0x55e0c8581390 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61e830d740 10 client.0 _ll_register_callbacks cb 0x7f61cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 0x55e0c8968520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 0x55e0c8968520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35066/0 (socket says 172.21.15.31:35066) 2024-09-21T02:56:50.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61dbfff640 1 -- 172.21.15.31:0/3290656241 learned_addr learned my addr 172.21.15.31:0/3290656241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c8581390 0x55e0c8967e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61db7fe640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c896a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.476+0000 7f61dbfff640 1 -- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c8581390 msgr2=0x55e0c8967e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c8581390 0x55e0c8967e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61dbfff640 1 -- 172.21.15.31:0/3290656241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 msgr2=0x55e0c896a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c896a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61dbfff640 1 -- 172.21.15.31:0/3290656241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c89d0630 con 0x55e0c85a2070 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61d2ffd640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c8581390 0x55e0c8967e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61db7fe640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c896a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 0x55e0c8968520 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f61d400fed0 tx=0x7f61d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b308ca8b40860c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61baffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61baffd640 1 -- 172.21.15.31:0/3290656241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d402b060 con 0x55e0c85a2070 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61baffd640 1 -- 172.21.15.31:0/3290656241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f61d401a650 con 0x55e0c85a2070 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c40ad250 con 0x55e0c85a2070 2024-09-21T02:56:50.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61baffd640 1 -- 172.21.15.31:0/3290656241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61d401a850 con 0x55e0c85a2070 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61e830d740 10 client.16530 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e0c89d1720 con 0x55e0c85a2070 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.477+0000 7f61baffd640 1 -- 172.21.15.31:0/3290656241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c40ad250 con 0x55e0c85a2070 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61baffd640 1 -- 172.21.15.31:0/3290656241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61d4025070 con 0x55e0c85a2070 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61e830d740 10 client.16530 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61e830d740 10 client.16530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61e830d740 20 client.16530 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61e830d740 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c8535da0 0x55e0c89d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61e830d740 4 client.16530 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0c88bd8a0 con 0x55e0c8535da0 2024-09-21T02:56:50.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61db7fe640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c8535da0 0x55e0c89d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:50.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61baffd640 10 client.16530 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:50.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.478+0000 7f61db7fe640 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c8535da0 0x55e0c89d3f40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b317a9fb60d59a18 server_cookie=576aa3fcd6bfc1c in_seq=0 out_seq=0 2024-09-21T02:56:50.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61baffd640 1 -- 172.21.15.31:0/3290656241 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0c88bd8a0 con 0x55e0c8535da0 2024-09-21T02:56:50.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61baffd640 10 client.16530 handle_command_reply: tid=0 2024-09-21T02:56:50.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 client.16530 shutdown 2024-09-21T02:56:50.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 msgr2=0x55e0c8968520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 0x55e0c8968520 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f61d400fed0 tx=0x7f61d4004a40 comp rx=0 tx=0).stop 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 shutdown_connections 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55e0c8535da0 0x55e0c89d3f40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0c8581390 0x55e0c8967e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0c85a2070 0x55e0c8968520 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 --2- 172.21.15.31:0/3290656241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c85791b0 0x55e0c896a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 >> 172.21.15.31:0/3290656241 conn(0x55e0c8596460 msgr2=0x55e0c894af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61baffd640 0 client.16530 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 shutdown_connections 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 1 -- 172.21.15.31:0/3290656241 wait complete. 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.479+0000 7f61e830d740 20 client.16530 trim_cache size 0 max 16384 2024-09-21T02:56:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61c80b6680 msgr2=0x7f61c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61c80b6680 0x7f61c80b8b90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61d400af80 tx=0x7f61d400a730 comp rx=0 tx=0).stop 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 msgr2=0x7f61dc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 0x7f61dc075c50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c8c0 tx=0x7f61c400cd80 comp rx=0 tx=0).stop 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 shutdown_connections 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61c80b6680 0x7f61c80b8b90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61dc109e00 0x7f61dc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc1094b0 0x7f61dc075c50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 --2- 172.21.15.31:0/1156298082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61dc103bc0 0x7f61dc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 >> 172.21.15.31:0/1156298082 conn(0x7f61dc07b910 msgr2=0x7f61dc1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 shutdown_connections 2024-09-21T02:56:50.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:50.480+0000 7f61e1d99640 1 -- 172.21.15.31:0/1156298082 wait complete. 2024-09-21T02:56:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:50 smithi031 ceph-mon[31209]: pgmap v443: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:50 smithi165 ceph-mon[34818]: pgmap v443: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:50 smithi178 ceph-mon[34759]: pgmap v443: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:51.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:56:51.604 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:52.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 -- 172.21.15.31:0/2795029940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 msgr2=0x7fba341098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 --2- 172.21.15.31:0/2795029940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba341098c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fba24009900 tx=0x7fba2402a410 comp rx=0 tx=0).stop 2024-09-21T02:56:52.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 -- 172.21.15.31:0/2795029940 shutdown_connections 2024-09-21T02:56:52.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 --2- 172.21.15.31:0/2795029940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba34109e00 0x7fba34110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 --2- 172.21.15.31:0/2795029940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba341098c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 --2- 172.21.15.31:0/2795029940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba34101f70 0x7fba34102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 -- 172.21.15.31:0/2795029940 >> 172.21.15.31:0/2795029940 conn(0x7fba34077c60 msgr2=0x7fba340780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:52.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 -- 172.21.15.31:0/2795029940 shutdown_connections 2024-09-21T02:56:52.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 -- 172.21.15.31:0/2795029940 wait complete. 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.107+0000 7fba3c303640 1 Processor -- start 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 -- start start 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba34101f70 0x7fba3419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba3419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba34109e00 0x7fba3419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba34113ac0 con 0x7fba341094b0 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fba34113940 con 0x7fba34101f70 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3c303640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba34113c40 con 0x7fba34109e00 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba3419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba3419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35084/0 (socket says 172.21.15.31:35084) 2024-09-21T02:56:52.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 -- 172.21.15.31:0/2199849097 learned_addr learned my addr 172.21.15.31:0/2199849097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba34109e00 0x7fba3419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3a09f640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba34101f70 0x7fba3419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 -- 172.21.15.31:0/2199849097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba34109e00 msgr2=0x7fba3419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba34109e00 0x7fba3419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 -- 172.21.15.31:0/2199849097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba34101f70 msgr2=0x7fba3419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba34101f70 0x7fba3419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.108+0000 7fba3989e640 1 -- 172.21.15.31:0/2199849097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba341a0360 con 0x7fba341094b0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.109+0000 7fba3989e640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba3419aa50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fba2402a930 tx=0x7fba24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.109+0000 7fba2b7fe640 1 -- 172.21.15.31:0/2199849097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2403f030 con 0x7fba341094b0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.109+0000 7fba2b7fe640 1 -- 172.21.15.31:0/2199849097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba24003aa0 con 0x7fba341094b0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.109+0000 7fba2b7fe640 1 -- 172.21.15.31:0/2199849097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24035070 con 0x7fba341094b0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.109+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba341a05f0 con 0x7fba341094b0 2024-09-21T02:56:52.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.109+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba341a0ad0 con 0x7fba341094b0 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.111+0000 7fba42877740 1 Processor -- start 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 -- start start 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba2b7fe640 1 -- 172.21.15.31:0/2199849097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fba24042030 con 0x7fba341094b0 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517ca9070 0x55c5180502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c801b0 0x55c518050800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518050d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c517fc6ea0 con 0x55c517c88390 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c517fb62d0 con 0x55c517c801b0 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba42877740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c517fba0e0 con 0x55c517ca9070 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba3989e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518050d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba3989e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518050d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35094/0 (socket says 172.21.15.31:35094) 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba3989e640 1 -- 172.21.15.31:0/2905007903 learned_addr learned my addr 172.21.15.31:0/2905007903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.112+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517ca9070 0x55c5180502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3a09f640 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c801b0 0x55c518050800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3989e640 1 -- 172.21.15.31:0/2905007903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517ca9070 msgr2=0x55c5180502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3989e640 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517ca9070 0x55c5180502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3989e640 1 -- 172.21.15.31:0/2905007903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c801b0 msgr2=0x55c518050800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3989e640 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c801b0 0x55c518050800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3989e640 1 -- 172.21.15.31:0/2905007903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c517d14650 con 0x55c517c88390 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba2b7fe640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fba200b66b0 0x7fba200b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fba200b66b0 0x7fba200b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba2b7fe640 1 -- 172.21.15.31:0/2199849097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fba24107bd0 con 0x7fba341094b0 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba3989e640 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518050d40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fba24105280 tx=0x7fba241101a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662bbf446b870859 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba28d9a640 1 -- 172.21.15.31:0/2905007903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24049070 con 0x55c517c88390 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba28d9a640 1 -- 172.21.15.31:0/2905007903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba240c91a0 con 0x55c517c88390 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.113+0000 7fba28d9a640 1 -- 172.21.15.31:0/2905007903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240c98e0 con 0x55c517c88390 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 -- 172.21.15.31:0/2905007903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 msgr2=0x55c518050d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fba200b66b0 0x7fba200b8bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fba2c007f80 tx=0x7fba2c007520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518050d40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fba24105280 tx=0x7fba241101a0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 -- 172.21.15.31:0/2905007903 shutdown_connections 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518050d40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c801b0 0x55c518050800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 --2- 172.21.15.31:0/2905007903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517ca9070 0x55c5180502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 -- 172.21.15.31:0/2905007903 >> 172.21.15.31:0/2905007903 conn(0x55c517c9d460 msgr2=0x55c51804e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 -- 172.21.15.31:0/2905007903 shutdown_connections 2024-09-21T02:56:52.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.114+0000 7fba42877740 1 -- 172.21.15.31:0/2905007903 wait complete. 2024-09-21T02:56:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 -- start start 2024-09-21T02:56:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 Processor -- start 2024-09-21T02:56:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518060200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517ca9070 0x55c518060740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c517e17700 con 0x55c517c88390 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c517f51160 con 0x55c517ca9070 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c518062b90 con 0x55c517c801b0 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba42877740 10 client.0 _ll_register_callbacks cb 0x55c517f6a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba3989e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518060200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba3989e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518060200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35100/0 (socket says 172.21.15.31:35100) 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.115+0000 7fba3989e640 1 -- 172.21.15.31:0/3473668749 learned_addr learned my addr 172.21.15.31:0/3473668749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3a09f640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517ca9070 0x55c518060740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3989e640 1 -- 172.21.15.31:0/3473668749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 msgr2=0x55c51805fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3989e640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3989e640 1 -- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517ca9070 msgr2=0x55c518060740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3989e640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517ca9070 0x55c518060740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3989e640 1 -- 172.21.15.31:0/3473668749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c517f7ec90 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517ca9070 0x55c518060740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba3989e640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518060200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fba2403d040 tx=0x7fba24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3007e2f5eb93af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba1dffb640 1 -- 172.21.15.31:0/3473668749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24049040 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba1dffb640 1 -- 172.21.15.31:0/3473668749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fba240c9cb0 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba240c91a0 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba1dffb640 1 -- 172.21.15.31:0/3473668749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba240cd1b0 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba42877740 10 client.16542 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.116+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c51806e7a0 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba1dffb640 1 -- 172.21.15.31:0/3473668749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba240b6070 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba1dffb640 1 -- 172.21.15.31:0/3473668749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fba2410b040 con 0x55c517c88390 2024-09-21T02:56:52.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba42877740 10 client.16542 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba42877740 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba42877740 20 client.16542 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba42877740 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c679f0 0x55c518071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba42877740 4 client.16542 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c517fc8df0 con 0x55c517c679f0 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.117+0000 7fba3a09f640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c679f0 0x55c518071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.118+0000 7fba1dffb640 10 client.16542 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:52.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.118+0000 7fba3a09f640 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c679f0 0x55c518071b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2edd8421e6a64b33 server_cookie=500c2926dc9432fb in_seq=0 out_seq=0 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.118+0000 7fba1dffb640 1 -- 172.21.15.31:0/3473668749 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c517fc8df0 con 0x55c517c679f0 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.118+0000 7fba1dffb640 10 client.16542 handle_command_reply: tid=0 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.118+0000 7fba42877740 1 client.16542 shutdown 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 msgr2=0x55c518060200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518060200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fba2403d040 tx=0x7fba24000bc0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 shutdown_connections 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c679f0 0x55c518071b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517ca9070 0x55c518060740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517c88390 0x55c518060200 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 --2- 172.21.15.31:0/3473668749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 >> 172.21.15.31:0/3473668749 conn(0x55c517c9d460 msgr2=0x55c518053410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:52.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba1dffb640 0 client.16542 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:52.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 shutdown_connections 2024-09-21T02:56:52.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 1 -- 172.21.15.31:0/3473668749 wait complete. 2024-09-21T02:56:52.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.119+0000 7fba42877740 20 client.16542 trim_cache size 0 max 16384 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 Processor -- start 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 -- start start 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c51805d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c5180611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba341ad400 con 0x55c517ca9070 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fba341aef40 con 0x55c517c88390 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.132+0000 7fba42877740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba341adad0 con 0x55c517c801b0 2024-09-21T02:56:52.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c51805d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3a8a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c5180611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3a09f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c51805d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:52666/0 (socket says 172.21.15.31:52666) 2024-09-21T02:56:52.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 -- 172.21.15.31:0/1061575279 learned_addr learned my addr 172.21.15.31:0/1061575279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:52.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 -- 172.21.15.31:0/1061575279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 msgr2=0x55c51805ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 -- 172.21.15.31:0/1061575279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 msgr2=0x55c5180611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c5180611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 -- 172.21.15.31:0/1061575279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c518054370 con 0x55c517c88390 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c5180611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba3989e640 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c51805d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba240ca010 tx=0x7fba2403b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0bcdc0dbb364d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.133+0000 7fba1ffff640 1 -- 172.21.15.31:0/1061575279 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24049070 con 0x55c517c88390 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba1ffff640 1 -- 172.21.15.31:0/1061575279 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba240b6070 con 0x55c517c88390 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba1ffff640 1 -- 172.21.15.31:0/1061575279 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240cd390 con 0x55c517c88390 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 -- 172.21.15.31:0/1061575279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 msgr2=0x55c51805d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c51805d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba240ca010 tx=0x7fba2403b6b0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 -- 172.21.15.31:0/1061575279 shutdown_connections 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c5180611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c51805d3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 --2- 172.21.15.31:0/1061575279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c51805ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 -- 172.21.15.31:0/1061575279 >> 172.21.15.31:0/1061575279 conn(0x55c517c9d460 msgr2=0x55c51805e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 -- 172.21.15.31:0/1061575279 shutdown_connections 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.134+0000 7fba42877740 1 -- 172.21.15.31:0/1061575279 wait complete. 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 -- start start 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 Processor -- start 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c5180d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c5180d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c518063630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba341ae870 con 0x55c517ca9070 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fba341ae1a0 con 0x55c517c88390 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fba341ab3e0 con 0x55c517c801b0 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba42877740 10 client.0 _ll_register_callbacks cb 0x7fba3000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:52.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba3a8a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c518063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba3a8a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c518063630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35130/0 (socket says 172.21.15.31:35130) 2024-09-21T02:56:52.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba3a8a0640 1 -- 172.21.15.31:0/979813745 learned_addr learned my addr 172.21.15.31:0/979813745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:52.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba3a09f640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c5180d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.135+0000 7fba3989e640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c5180d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba3a8a0640 1 -- 172.21.15.31:0/979813745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 msgr2=0x55c5180d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c5180d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba3a8a0640 1 -- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 msgr2=0x55c5180d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c5180d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba3a8a0640 1 -- 172.21.15.31:0/979813745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c518063da0 con 0x55c517ca9070 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba3a8a0640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c518063630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00fec0 tx=0x7fba2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1251f4953bd98829 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba1dffb640 1 -- 172.21.15.31:0/979813745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c01d0d0 con 0x55c517ca9070 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba1dffb640 1 -- 172.21.15.31:0/979813745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fba2c003970 con 0x55c517ca9070 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba240b6070 con 0x55c517ca9070 2024-09-21T02:56:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba1dffb640 1 -- 172.21.15.31:0/979813745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba2c01d270 con 0x55c517ca9070 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba42877740 10 client.16548 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5180dbf00 con 0x55c517ca9070 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.136+0000 7fba1dffb640 1 -- 172.21.15.31:0/979813745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba240b6070 con 0x55c517ca9070 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba1dffb640 1 -- 172.21.15.31:0/979813745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fba2c030020 con 0x55c517ca9070 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba42877740 10 client.16548 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba42877740 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba42877740 20 client.16548 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba42877740 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c3cda0 0x55c5180de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba42877740 4 client.16548 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c51804e2a0 con 0x55c517c3cda0 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.137+0000 7fba3a09f640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c3cda0 0x55c5180de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:52.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba1dffb640 10 client.16548 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba3a09f640 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c3cda0 0x55c5180de770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9f64f494bc853b0 server_cookie=8ec16c16be3b1ffa in_seq=0 out_seq=0 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba1dffb640 1 -- 172.21.15.31:0/979813745 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c51804e2a0 con 0x55c517c3cda0 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba1dffb640 10 client.16548 handle_command_reply: tid=0 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 client.16548 shutdown 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 msgr2=0x55c518063630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c518063630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00fec0 tx=0x7fba2c015000 comp rx=0 tx=0).stop 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 shutdown_connections 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55c517c3cda0 0x55c5180de770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c517ca9070 0x55c518063630 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c517c88390 0x55c5180d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 --2- 172.21.15.31:0/979813745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c517c801b0 0x55c5180d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 >> 172.21.15.31:0/979813745 conn(0x55c517c9d460 msgr2=0x55c5180512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba1dffb640 0 client.16548 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 shutdown_connections 2024-09-21T02:56:52.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 1 -- 172.21.15.31:0/979813745 wait complete. 2024-09-21T02:56:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.138+0000 7fba42877740 20 client.16548 trim_cache size 0 max 16384 2024-09-21T02:56:52.144 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fba200b66b0 msgr2=0x7fba200b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fba200b66b0 0x7fba200b8bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fba2c007f80 tx=0x7fba2c007520 comp rx=0 tx=0).stop 2024-09-21T02:56:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 msgr2=0x7fba3419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:52.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba3419aa50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fba2402a930 tx=0x7fba24004270 comp rx=0 tx=0).stop 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 shutdown_connections 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fba200b66b0 0x7fba200b8bc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba34109e00 0x7fba3419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba341094b0 0x7fba3419aa50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 --2- 172.21.15.31:0/2199849097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba34101f70 0x7fba3419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 >> 172.21.15.31:0/2199849097 conn(0x7fba34077c60 msgr2=0x7fba34105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 shutdown_connections 2024-09-21T02:56:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:52.139+0000 7fba3c303640 1 -- 172.21.15.31:0/2199849097 wait complete. 2024-09-21T02:56:53.028 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:52 smithi031 ceph-mon[31209]: pgmap v444: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:53.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:56:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:52 smithi165 ceph-mon[34818]: pgmap v444: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:52 smithi178 ceph-mon[34759]: pgmap v444: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:56:53.325 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 -- 172.21.15.31:0/3688565308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584108d30 msgr2=0x7f158410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 --2- 172.21.15.31:0/3688565308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584108d30 0x7f158410f560 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f157c009ca0 tx=0x7f157c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 -- 172.21.15.31:0/3688565308 shutdown_connections 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 --2- 172.21.15.31:0/3688565308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584108d30 0x7f158410f560 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 --2- 172.21.15.31:0/3688565308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f15841083e0 0x7f15841087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 --2- 172.21.15.31:0/3688565308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1584102aa0 0x7f1584102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 -- 172.21.15.31:0/3688565308 >> 172.21.15.31:0/3688565308 conn(0x7f15840797e0 msgr2=0x7f1584079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 -- 172.21.15.31:0/3688565308 shutdown_connections 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.802+0000 7f158b632640 1 -- 172.21.15.31:0/3688565308 wait complete. 2024-09-21T02:56:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.803+0000 7f158b632640 1 Processor -- start 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 -- start start 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 0x7f1584199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15841083e0 0x7f15841998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1584108d30 0x7f158419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15841129a0 con 0x7f1584102aa0 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1584112820 con 0x7f1584108d30 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f158b632640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1584112b20 con 0x7f15841083e0 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f15893ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 0x7f1584199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f15893ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 0x7f1584199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35146/0 (socket says 172.21.15.31:35146) 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f15893ce640 1 -- 172.21.15.31:0/4222684477 learned_addr learned my addr 172.21.15.31:0/4222684477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f1588bcd640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15841083e0 0x7f15841998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f1589bcf640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1584108d30 0x7f158419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f15893ce640 1 -- 172.21.15.31:0/4222684477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15841083e0 msgr2=0x7f15841998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.804+0000 7f15893ce640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15841083e0 0x7f15841998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f15893ce640 1 -- 172.21.15.31:0/4222684477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1584108d30 msgr2=0x7f158419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f15893ce640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1584108d30 0x7f158419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f15893ce640 1 -- 172.21.15.31:0/4222684477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158419f280 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f1589bcf640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1584108d30 0x7f158419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f15893ce640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 0x7f1584199310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f158000eb00 tx=0x7f1580009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f157a7fc640 1 -- 172.21.15.31:0/4222684477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158001c070 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f157a7fc640 1 -- 172.21.15.31:0/4222684477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158000bc30 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158419f570 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f157a7fc640 1 -- 172.21.15.31:0/4222684477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580016610 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.805+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158406b820 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.808+0000 7f157a7fc640 1 -- 172.21.15.31:0/4222684477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f158001f070 con 0x7f1584102aa0 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.808+0000 7f1591ba6740 1 Processor -- start 2024-09-21T02:56:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.808+0000 7f1591ba6740 1 -- start start 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2eca070 0x561ab32716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea11b0 0x561ab3271be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 0x561ab3272120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1591ba6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561ab31e7ea0 con 0x561ab2ea9390 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1591ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ab31d72d0 con 0x561ab2eca070 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1591ba6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ab31db0e0 con 0x561ab2ea11b0 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 0x561ab3272120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 0x561ab3272120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35160/0 (socket says 172.21.15.31:35160) 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 -- 172.21.15.31:0/1227635109 learned_addr learned my addr 172.21.15.31:0/1227635109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1589bcf640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2eca070 0x561ab32716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f15893ce640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea11b0 0x561ab3271be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 -- 172.21.15.31:0/1227635109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea11b0 msgr2=0x561ab3271be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea11b0 0x561ab3271be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 -- 172.21.15.31:0/1227635109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2eca070 msgr2=0x561ab32716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2eca070 0x561ab32716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.809+0000 7f1588bcd640 1 -- 172.21.15.31:0/1227635109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab2f35650 con 0x561ab2ea9390 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f1589bcf640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2eca070 0x561ab32716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f15893ce640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea11b0 0x561ab3271be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f1588bcd640 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 0x561ab3272120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1574009c60 tx=0x7f1574011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02fc4a95f755e01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f157a7fc640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f15700baa90 0x7f15700bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f156f7fe640 1 -- 172.21.15.31:0/1227635109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574004230 con 0x561ab2ea9390 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f1589bcf640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f15700baa90 0x7f15700bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f157a7fc640 1 -- 172.21.15.31:0/4222684477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f15800e7210 con 0x7f1584102aa0 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f156f7fe640 1 -- 172.21.15.31:0/1227635109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15740043d0 con 0x561ab2ea9390 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.810+0000 7f156f7fe640 1 -- 172.21.15.31:0/1227635109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574004cd0 con 0x561ab2ea9390 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 -- 172.21.15.31:0/1227635109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 msgr2=0x561ab3272120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1589bcf640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f15700baa90 0x7f15700bcfa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f157c004af0 tx=0x7f157c005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 0x561ab3272120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1574009c60 tx=0x7f1574011810 comp rx=0 tx=0).stop 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 -- 172.21.15.31:0/1227635109 shutdown_connections 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea11b0 0x561ab3271be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea9390 0x561ab3272120 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1227635109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2eca070 0x561ab32716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 -- 172.21.15.31:0/1227635109 >> 172.21.15.31:0/1227635109 conn(0x561ab2ebe460 msgr2=0x561ab326f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 -- 172.21.15.31:0/1227635109 shutdown_connections 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 -- 172.21.15.31:0/1227635109 wait complete. 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 -- start start 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.812+0000 7f1591ba6740 1 Processor -- start 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab327ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab327d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab3289170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561ab3038700 con 0x561ab2eca070 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ab3172160 con 0x561ab2ea11b0 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ab327fe80 con 0x561ab2ea9390 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1591ba6740 10 client.0 _ll_register_callbacks cb 0x561ab318b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1589bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab3289170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1589bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab3289170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35164/0 (socket says 172.21.15.31:35164) 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1589bcf640 1 -- 172.21.15.31:0/524941356 learned_addr learned my addr 172.21.15.31:0/524941356 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f1588bcd640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab327d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.813+0000 7f15893ce640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab327ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1589bcf640 1 -- 172.21.15.31:0/524941356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 msgr2=0x561ab327d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1589bcf640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab327d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1589bcf640 1 -- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 msgr2=0x561ab327ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1589bcf640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab327ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1589bcf640 1 -- 172.21.15.31:0/524941356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab319fc90 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f15893ce640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab327ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1589bcf640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab3289170 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f157c035000 tx=0x7f157c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4017c526b0f3f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f156cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f156cff9640 1 -- 172.21.15.31:0/524941356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c003b80 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f156cff9640 1 -- 172.21.15.31:0/524941356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f157c003d20 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.814+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15740043d0 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f156cff9640 1 -- 172.21.15.31:0/524941356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f157c004040 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 10 client.16560 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ab328bb30 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f156cff9640 1 -- 172.21.15.31:0/524941356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15740043d0 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f156cff9640 1 -- 172.21.15.31:0/524941356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f157c05a070 con 0x561ab2eca070 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 10 client.16560 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 20 client.16560 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e889f0 0x561ab328f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 4 client.16560 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.815+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab31e9df0 con 0x561ab2e889f0 2024-09-21T02:56:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.816+0000 7f15893ce640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e889f0 0x561ab328f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.816+0000 7f15893ce640 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e889f0 0x561ab328f070 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea63062f6ccddaca server_cookie=6a37518902b861cc in_seq=0 out_seq=0 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.816+0000 7f156cff9640 10 client.16560 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f156cff9640 1 -- 172.21.15.31:0/524941356 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ab31e9df0 con 0x561ab2e889f0 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f156cff9640 10 client.16560 handle_command_reply: tid=0 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 client.16560 shutdown 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 msgr2=0x561ab3289170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab3289170 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f157c035000 tx=0x7f157c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 shutdown_connections 2024-09-21T02:56:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e889f0 0x561ab328f070 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab327d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab3289170 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 --2- 172.21.15.31:0/524941356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab327ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f156cff9640 0 client.16560 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 >> 172.21.15.31:0/524941356 conn(0x561ab2ebe460 msgr2=0x561ab3274880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 shutdown_connections 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 1 -- 172.21.15.31:0/524941356 wait complete. 2024-09-21T02:56:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.817+0000 7f1591ba6740 20 client.16560 trim_cache size 0 max 16384 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 Processor -- start 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 -- start start 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab3272de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab326ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab326f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ad480 con 0x561ab2eca070 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f15841aefc0 con 0x561ab2ea11b0 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1591ba6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15841adb50 con 0x561ab2ea9390 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.831+0000 7f1589bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab326f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab326f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35174/0 (socket says 172.21.15.31:35174) 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 -- 172.21.15.31:0/1933262937 learned_addr learned my addr 172.21.15.31:0/1933262937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f15893ce640 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab3272de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1588bcd640 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab326ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 -- 172.21.15.31:0/1933262937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 msgr2=0x561ab326ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab326ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 -- 172.21.15.31:0/1933262937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 msgr2=0x561ab3272de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab3272de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 -- 172.21.15.31:0/1933262937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab3273c60 con 0x561ab2eca070 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f1589bcf640 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab326f100 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f157c02ade0 tx=0x7f157c0305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe000c3a72494f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f156effd640 1 -- 172.21.15.31:0/1933262937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c05f030 con 0x561ab2eca070 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f156effd640 1 -- 172.21.15.31:0/1933262937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157c036040 con 0x561ab2eca070 2024-09-21T02:56:53.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.832+0000 7f156effd640 1 -- 172.21.15.31:0/1933262937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0041e0 con 0x561ab2eca070 2024-09-21T02:56:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 -- 172.21.15.31:0/1933262937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 msgr2=0x561ab326f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab326f100 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f157c02ade0 tx=0x7f157c0305c0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 -- 172.21.15.31:0/1933262937 shutdown_connections 2024-09-21T02:56:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2ea9390 0x561ab326ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2eca070 0x561ab326f100 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 --2- 172.21.15.31:0/1933262937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea11b0 0x561ab3272de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 -- 172.21.15.31:0/1933262937 >> 172.21.15.31:0/1933262937 conn(0x561ab2ebe460 msgr2=0x561ab32726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.833+0000 7f1591ba6740 1 -- 172.21.15.31:0/1933262937 shutdown_connections 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.834+0000 7f1591ba6740 1 -- 172.21.15.31:0/1933262937 wait complete. 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.834+0000 7f1591ba6740 1 -- start start 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.834+0000 7f1591ba6740 1 Processor -- start 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 0x561ab3281300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea9390 0x561ab3281840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2eca070 0x561ab32f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ae8f0 con 0x561ab2ea11b0 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ae220 con 0x561ab2ea9390 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15841ab460 con 0x561ab2eca070 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1591ba6740 10 client.0 _ll_register_callbacks cb 0x7f1580005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 0x561ab3281300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 0x561ab3281300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35182/0 (socket says 172.21.15.31:35182) 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 -- 172.21.15.31:0/4007138544 learned_addr learned my addr 172.21.15.31:0/4007138544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1588bcd640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea9390 0x561ab3281840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f1589bcf640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2eca070 0x561ab32f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 -- 172.21.15.31:0/4007138544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2eca070 msgr2=0x561ab32f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2eca070 0x561ab32f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 -- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea9390 msgr2=0x561ab3281840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea9390 0x561ab3281840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 -- 172.21.15.31:0/4007138544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab3282030 con 0x561ab2ea11b0 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f15893ce640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 0x561ab3281300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f15800ad010 tx=0x7f1580017c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dec704863ce5146 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.835+0000 7f156cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f156cff9640 1 -- 172.21.15.31:0/4007138544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15800ac9d0 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f156cff9640 1 -- 172.21.15.31:0/4007138544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f15800abd50 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157c036040 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f156cff9640 1 -- 172.21.15.31:0/4007138544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15800ac7f0 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 10 client.16566 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ab32f9f80 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f156cff9640 1 -- 172.21.15.31:0/4007138544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157c036040 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f156cff9640 1 -- 172.21.15.31:0/4007138544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15800abf50 con 0x561ab2ea11b0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 10 client.16566 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 20 client.16566 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e5dda0 0x561ab32fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 4 client.16566 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.836+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ab31e58a0 con 0x561ab2e5dda0 2024-09-21T02:56:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.837+0000 7f1588bcd640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e5dda0 0x561ab32fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:53.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.837+0000 7f1588bcd640 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e5dda0 0x561ab32fc6c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cde3ab530494f3c server_cookie=6fa0e33518c78711 in_seq=0 out_seq=0 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.837+0000 7f156cff9640 10 client.16566 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f156cff9640 1 -- 172.21.15.31:0/4007138544 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ab31e58a0 con 0x561ab2e5dda0 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f156cff9640 10 client.16566 handle_command_reply: tid=0 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 client.16566 shutdown 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 msgr2=0x561ab3281300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 0x561ab3281300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f15800ad010 tx=0x7f1580017c90 comp rx=0 tx=0).stop 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 shutdown_connections 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ab2e5dda0 0x561ab32fc6c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ab2eca070 0x561ab32f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ab2ea9390 0x561ab3281840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 --2- 172.21.15.31:0/4007138544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ab2ea11b0 0x561ab3281300 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f156cff9640 0 client.16566 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:53.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 >> 172.21.15.31:0/4007138544 conn(0x561ab2ebe460 msgr2=0x561ab32726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:53.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 shutdown_connections 2024-09-21T02:56:53.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 1 -- 172.21.15.31:0/4007138544 wait complete. 2024-09-21T02:56:53.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.838+0000 7f1591ba6740 20 client.16566 trim_cache size 0 max 16384 2024-09-21T02:56:53.844 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:56:53.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f15700baa90 msgr2=0x7f15700bcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f15700baa90 0x7f15700bcfa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f157c004af0 tx=0x7f157c005c30 comp rx=0 tx=0).stop 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 msgr2=0x7f1584199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 0x7f1584199310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f158000eb00 tx=0x7f1580009f20 comp rx=0 tx=0).stop 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 shutdown_connections 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f15700baa90 0x7f15700bcfa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15841083e0 0x7f15841998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1584108d30 0x7f158419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 --2- 172.21.15.31:0/4222684477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1584102aa0 0x7f1584199310 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 >> 172.21.15.31:0/4222684477 conn(0x7f15840797e0 msgr2=0x7f15841043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 shutdown_connections 2024-09-21T02:56:53.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:53.839+0000 7f158b632640 1 -- 172.21.15.31:0/4222684477 wait complete. 2024-09-21T02:56:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:54 smithi031 ceph-mon[31209]: pgmap v445: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:54 smithi165 ceph-mon[34818]: pgmap v445: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:54 smithi178 ceph-mon[34759]: pgmap v445: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:56:55.657 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:56:55.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:56:55.928 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/250537279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0108cf0 msgr2=0x7fb4a010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/250537279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0108cf0 0x7fb4a010f520 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb498009900 tx=0x7fb49802a410 comp rx=0 tx=0).stop 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/250537279 shutdown_connections 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/250537279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a01083a0 0x7fb4a01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/250537279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0108cf0 0x7fb4a010f520 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/250537279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a0102a60 0x7fb4a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/250537279 >> 172.21.15.31:0/250537279 conn(0x7fb4a0079730 msgr2=0x7fb4a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/250537279 shutdown_connections 2024-09-21T02:56:56.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.449+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/250537279 wait complete. 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.450+0000 7fb4a5ddb640 1 Processor -- start 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.450+0000 7fb4a5ddb640 1 -- start start 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb4a5ddb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 0x7fb4a019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb4a5ddb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a01083a0 0x7fb4a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb4a5ddb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0108cf0 0x7fb4a019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb4a5ddb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112960 con 0x7fb4a0102a60 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb4a5ddb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01127e0 con 0x7fb4a01083a0 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb4a5ddb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112ae0 con 0x7fb4a0108cf0 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 0x7fb4a019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 0x7fb4a019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35196/0 (socket says 172.21.15.31:35196) 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 -- 172.21.15.31:0/3527532497 learned_addr learned my addr 172.21.15.31:0/3527532497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49ffff640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0108cf0 0x7fb4a019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49effd640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a01083a0 0x7fb4a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 -- 172.21.15.31:0/3527532497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0108cf0 msgr2=0x7fb4a019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0108cf0 0x7fb4a019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 -- 172.21.15.31:0/3527532497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a01083a0 msgr2=0x7fb4a019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a01083a0 0x7fb4a019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 -- 172.21.15.31:0/3527532497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01a03f0 con 0x7fb4a0102a60 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49ffff640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0108cf0 0x7fb4a019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49effd640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a01083a0 0x7fb4a019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 0x7fb4a019a480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb49000cce0 tx=0x7fb49000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.451+0000 7fb49cff9640 1 -- 172.21.15.31:0/3527532497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490004b20 con 0x7fb4a0102a60 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.452+0000 7fb49cff9640 1 -- 172.21.15.31:0/3527532497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb490004cc0 con 0x7fb4a0102a60 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.452+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a01a06e0 con 0x7fb4a0102a60 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.452+0000 7fb49cff9640 1 -- 172.21.15.31:0/3527532497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49000f6b0 con 0x7fb4a0102a60 2024-09-21T02:56:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.452+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01a6e40 con 0x7fb4a0102a60 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.454+0000 7fb4ac34f740 1 Processor -- start 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.454+0000 7fb4ac34f740 1 -- start start 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a051729e60 con 0x55a0513e3490 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a051719290 con 0x55a05140c350 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a05171d0a0 con 0x55a0513eb670 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49cff9640 1 -- 172.21.15.31:0/3527532497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb49000f8f0 con 0x7fb4a0102a60 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54092/0 (socket says 172.21.15.31:54092) 2024-09-21T02:56:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 -- 172.21.15.31:0/3802295527 learned_addr learned my addr 172.21.15.31:0/3802295527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 -- 172.21.15.31:0/3802295527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 msgr2=0x55a0517b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 -- 172.21.15.31:0/3802295527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 msgr2=0x55a0517b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49effd640 1 -- 172.21.15.31:0/3802295527 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a051477930 con 0x55a0513eb670 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.455+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb49effd640 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517b3eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb48800cbc0 tx=0x7fb48800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5f617b1674c102 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb495ffb640 1 -- 172.21.15.31:0/3802295527 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488005220 con 0x55a0513eb670 2024-09-21T02:56:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb495ffb640 1 -- 172.21.15.31:0/3802295527 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb488004c50 con 0x55a0513eb670 2024-09-21T02:56:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb49cff9640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb48c0b6680 0x7fb48c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb495ffb640 1 -- 172.21.15.31:0/3802295527 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4880106b0 con 0x55a0513eb670 2024-09-21T02:56:56.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb49ffff640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb48c0b6680 0x7fb48c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.456+0000 7fb49cff9640 1 -- 172.21.15.31:0/3527532497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb4900e78c0 con 0x7fb4a0102a60 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 -- 172.21.15.31:0/3802295527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 msgr2=0x55a0517b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517b3eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb48800cbc0 tx=0x7fb48800e720 comp rx=0 tx=0).stop 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb49ffff640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb48c0b6680 0x7fb48c0b8b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb498005940 tx=0x7fb498009660 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 -- 172.21.15.31:0/3802295527 shutdown_connections 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517b3eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/3802295527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 -- 172.21.15.31:0/3802295527 >> 172.21.15.31:0/3802295527 conn(0x55a051400740 msgr2=0x55a0517b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 -- 172.21.15.31:0/3802295527 shutdown_connections 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 -- 172.21.15.31:0/3802295527 wait complete. 2024-09-21T02:56:56.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.457+0000 7fb4ac34f740 1 -- start start 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 Processor -- start 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a05157a890 con 0x55a0513e3490 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0516b3d40 con 0x55a05140c350 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0517c5f40 con 0x55a0513eb670 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb4ac34f740 10 client.0 _ll_register_callbacks cb 0x55a051717c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517c3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35222/0 (socket says 172.21.15.31:35222) 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 -- 172.21.15.31:0/32357046 learned_addr learned my addr 172.21.15.31:0/32357046 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49effd640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49ffff640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 -- 172.21.15.31:0/32357046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 msgr2=0x55a0517c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 -- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 msgr2=0x55a0517cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.458+0000 7fb49f7fe640 1 -- 172.21.15.31:0/32357046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0517cf7e0 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517c3390 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb490005c40 tx=0x7fb4900acd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6c46b5f57d490b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb47effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb47effd640 1 -- 172.21.15.31:0/32357046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49001d040 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb47effd640 1 -- 172.21.15.31:0/32357046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb49000fbe0 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb488004c50 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb47effd640 1 -- 172.21.15.31:0/32357046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4900a9d80 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 10 client.16578 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0517d2100 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb47effd640 1 -- 172.21.15.31:0/32357046 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb488004c50 con 0x55a0513e3490 2024-09-21T02:56:56.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb47effd640 1 -- 172.21.15.31:0/32357046 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb4900b6690 con 0x55a0513e3490 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 10 client.16578 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 20 client.16578 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513cacd0 0x55a0517d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.459+0000 7fb4ac34f740 4 client.16578 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.460+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05172bdb0 con 0x55a0513cacd0 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.460+0000 7fb49effd640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513cacd0 0x55a0517d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.460+0000 7fb47effd640 10 client.16578 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.460+0000 7fb49effd640 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513cacd0 0x55a0517d55f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d9a8387bffa1456 server_cookie=90f3d376abb637d in_seq=0 out_seq=0 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb47effd640 1 -- 172.21.15.31:0/32357046 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a05172bdb0 con 0x55a0513cacd0 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb47effd640 10 client.16578 handle_command_reply: tid=0 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 client.16578 shutdown 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 msgr2=0x55a0517c3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517c3390 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb490005c40 tx=0x7fb4900acd60 comp rx=0 tx=0).stop 2024-09-21T02:56:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 shutdown_connections 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513cacd0 0x55a0517d55f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a05140c350 0x55a0517cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/32357046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513e3490 0x55a0517c3390 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 >> 172.21.15.31:0/32357046 conn(0x55a051400740 msgr2=0x55a0517b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb47effd640 0 client.16578 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 shutdown_connections 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.461+0000 7fb4ac34f740 1 -- 172.21.15.31:0/32357046 wait complete. 2024-09-21T02:56:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.462+0000 7fb4ac34f740 20 client.16578 trim_cache size 0 max 16384 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 Processor -- start 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 -- start start 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 0x55a0517c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a05140c350 0x55a0517b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01ad5a0 con 0x55a0513eb670 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01af0e0 con 0x55a0513e3490 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01adc70 con 0x55a05140c350 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb49effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 0x55a0517c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb49effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 0x55a0517c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35238/0 (socket says 172.21.15.31:35238) 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.479+0000 7fb49effd640 1 -- 172.21.15.31:0/1362530233 learned_addr learned my addr 172.21.15.31:0/1362530233 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49ffff640 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a05140c350 0x55a0517b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49effd640 1 -- 172.21.15.31:0/1362530233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a05140c350 msgr2=0x55a0517b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49effd640 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a05140c350 0x55a0517b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49effd640 1 -- 172.21.15.31:0/1362530233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 msgr2=0x55a0517c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49effd640 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49effd640 1 -- 172.21.15.31:0/1362530233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0517b74e0 con 0x55a0513eb670 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb49effd640 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 0x55a0517c5070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb48800f010 tx=0x7fb48800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f5f3231c87454c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb4957fa640 1 -- 172.21.15.31:0/1362530233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488028630 con 0x55a0513eb670 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb4957fa640 1 -- 172.21.15.31:0/1362530233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb488028c80 con 0x55a0513eb670 2024-09-21T02:56:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.480+0000 7fb4957fa640 1 -- 172.21.15.31:0/1362530233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48802ebc0 con 0x55a0513eb670 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 -- 172.21.15.31:0/1362530233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 msgr2=0x55a0517c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 0x55a0517c5070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb48800f010 tx=0x7fb48800c330 comp rx=0 tx=0).stop 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 -- 172.21.15.31:0/1362530233 shutdown_connections 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a05140c350 0x55a0517b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0513eb670 0x55a0517c5070 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/1362530233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 -- 172.21.15.31:0/1362530233 >> 172.21.15.31:0/1362530233 conn(0x55a051400740 msgr2=0x55a0517b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 -- 172.21.15.31:0/1362530233 shutdown_connections 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 -- 172.21.15.31:0/1362530233 wait complete. 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.481+0000 7fb4ac34f740 1 -- start start 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 Processor -- start 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 0x55a051838ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01aea10 con 0x55a05140c350 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01ae340 con 0x55a0513e3490 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01ab580 con 0x55a0513eb670 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb4ac34f740 10 client.0 _ll_register_callbacks cb 0x7fb49000cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 0x55a051838ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 0x55a051838ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35246/0 (socket says 172.21.15.31:35246) 2024-09-21T02:56:56.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb49ffff640 1 -- 172.21.15.31:0/2470434001 learned_addr learned my addr 172.21.15.31:0/2470434001 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.482+0000 7fb49effd640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49ffff640 1 -- 172.21.15.31:0/2470434001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 msgr2=0x55a0517c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49ffff640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49ffff640 1 -- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 msgr2=0x55a0517c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49ffff640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49ffff640 1 -- 172.21.15.31:0/2470434001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a051839660 con 0x55a05140c350 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49effd640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb49ffff640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 0x55a051838ff0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb498009900 tx=0x7fb498009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39e0ea84d34423c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb47effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb47effd640 1 -- 172.21.15.31:0/2470434001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498035040 con 0x55a05140c350 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb47effd640 1 -- 172.21.15.31:0/2470434001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb498003970 con 0x55a05140c350 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb488028c80 con 0x55a05140c350 2024-09-21T02:56:56.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb47effd640 1 -- 172.21.15.31:0/2470434001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb498003b70 con 0x55a05140c350 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb4ac34f740 10 client.16584 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.483+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0518409c0 con 0x55a05140c350 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb47effd640 1 -- 172.21.15.31:0/2470434001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb488028c80 con 0x55a05140c350 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb47effd640 1 -- 172.21.15.31:0/2470434001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb49805a070 con 0x55a05140c350 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb4ac34f740 10 client.16584 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb4ac34f740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb4ac34f740 20 client.16584 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513a0080 0x55a051843200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb4ac34f740 4 client.16584 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a051727860 con 0x55a0513a0080 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.484+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513a0080 0x55a051843200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.485+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513a0080 0x55a051843200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae21f259b50f468 server_cookie=f4ecf2600c88fd1e in_seq=0 out_seq=0 2024-09-21T02:56:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.485+0000 7fb47effd640 10 client.16584 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb47effd640 1 -- 172.21.15.31:0/2470434001 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a051727860 con 0x55a0513a0080 2024-09-21T02:56:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb47effd640 10 client.16584 handle_command_reply: tid=0 2024-09-21T02:56:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 client.16584 shutdown 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 msgr2=0x55a051838ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 0x55a051838ff0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb498009900 tx=0x7fb498009260 comp rx=0 tx=0).stop 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 shutdown_connections 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55a0513a0080 0x55a051843200 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0513eb670 0x55a0517c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a05140c350 0x55a051838ff0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb47effd640 0 client.16584 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 --2- 172.21.15.31:0/2470434001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0513e3490 0x55a0517c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 >> 172.21.15.31:0/2470434001 conn(0x55a051400740 msgr2=0x55a0517b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 shutdown_connections 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 1 -- 172.21.15.31:0/2470434001 wait complete. 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.487+0000 7fb4ac34f740 20 client.16584 trim_cache size 0 max 16384 2024-09-21T02:56:56.491 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:56:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:56:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "a751537f-fbc5-4cd8-b5f6-405f14d5969d", 2024-09-21T02:56:56.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:56:56.492 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb48c0b6680 msgr2=0x7fb48c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb48c0b6680 0x7fb48c0b8b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb498005940 tx=0x7fb498009660 comp rx=0 tx=0).stop 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 msgr2=0x7fb4a019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 0x7fb4a019a480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb49000cce0 tx=0x7fb49000a620 comp rx=0 tx=0).stop 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 shutdown_connections 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb48c0b6680 0x7fb48c0b8b90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0108cf0 0x7fb4a019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a01083a0 0x7fb4a019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 --2- 172.21.15.31:0/3527532497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a0102a60 0x7fb4a019a480 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 >> 172.21.15.31:0/3527532497 conn(0x7fb4a0079730 msgr2=0x7fb4a006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 shutdown_connections 2024-09-21T02:56:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:56.488+0000 7fb4a5ddb640 1 -- 172.21.15.31:0/3527532497 wait complete. 2024-09-21T02:56:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:56 smithi165 ceph-mon[34818]: pgmap v446: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:56 smithi178 ceph-mon[34759]: pgmap v446: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:57.325 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:56 smithi031 ceph-mon[31209]: pgmap v446: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:56:57.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:56:57.615 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:57 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:56:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:57 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:58.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:57 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:56:58.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:57 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 -- 172.21.15.31:0/1466265856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1048c0 msgr2=0x7f96cc104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 --2- 172.21.15.31:0/1466265856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1048c0 0x7f96cc104cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009ca0 tx=0x7f96bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 -- 172.21.15.31:0/1466265856 shutdown_connections 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 --2- 172.21.15.31:0/1466265856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc1052a0 0x7f96cc10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 --2- 172.21.15.31:0/1466265856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96cc10de70 0x7f96cc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 --2- 172.21.15.31:0/1466265856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1048c0 0x7f96cc104cd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 -- 172.21.15.31:0/1466265856 >> 172.21.15.31:0/1466265856 conn(0x7f96cc079760 msgr2=0x7f96cc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 -- 172.21.15.31:0/1466265856 shutdown_connections 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.104+0000 7f96d0daf640 1 -- 172.21.15.31:0/1466265856 wait complete. 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.105+0000 7f96d0daf640 1 Processor -- start 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.105+0000 7f96d0daf640 1 -- start start 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.105+0000 7f96d0daf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96cc1048c0 0x7f96cc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.105+0000 7f96d0daf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 0x7f96cc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.105+0000 7f96d0daf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc10de70 0x7f96cc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96d0daf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc113ad0 con 0x7f96cc1052a0 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96d0daf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc113950 con 0x7f96cc1048c0 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96d0daf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc113c50 con 0x7f96cc10de70 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 0x7f96cc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 0x7f96cc19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35272/0 (socket says 172.21.15.31:35272) 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 -- 172.21.15.31:0/2126874188 learned_addr learned my addr 172.21.15.31:0/2126874188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96ca59c640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96cc1048c0 0x7f96cc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96cad9d640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc10de70 0x7f96cc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 -- 172.21.15.31:0/2126874188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc10de70 msgr2=0x7f96cc19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc10de70 0x7f96cc19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 -- 172.21.15.31:0/2126874188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96cc1048c0 msgr2=0x7f96cc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96cc1048c0 0x7f96cc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 -- 172.21.15.31:0/2126874188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1a04d0 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 0x7f96cc19ab00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f96c000d740 tx=0x7f96c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96bb7fe640 1 -- 172.21.15.31:0/2126874188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0017070 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.106+0000 7f96bb7fe640 1 -- 172.21.15.31:0/2126874188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c000de50 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.107+0000 7f96bb7fe640 1 -- 172.21.15.31:0/2126874188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000f620 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.107+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1a07c0 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.107+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1a6f20 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.109+0000 7f96bb7fe640 1 -- 172.21.15.31:0/2126874188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f96c001a070 con 0x7f96cc1052a0 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.109+0000 7f96d7323740 1 Processor -- start 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 -- start start 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cc5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268f51b0 0x559c26cc5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cc60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559c26c3be50 con 0x559c268fd390 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c26c2b280 con 0x559c2691e070 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96d7323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c26c2f090 con 0x559c268f51b0 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cc60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96cad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cc5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268f51b0 0x559c26cc5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96cad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cc5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:52794/0 (socket says 172.21.15.31:52794) 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96cad9d640 1 -- 172.21.15.31:0/1162274130 learned_addr learned my addr 172.21.15.31:0/1162274130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.110+0000 7f96bb7fe640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96b40b6680 0x7f96b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 -- 172.21.15.31:0/1162274130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268f51b0 msgr2=0x559c26cc5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268f51b0 0x559c26cc5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 -- 172.21.15.31:0/1162274130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 msgr2=0x559c26cc60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cc60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 -- 172.21.15.31:0/1162274130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c26989650 con 0x559c2691e070 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96ca59c640 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268f51b0 0x559c26cc5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:56:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96b40b6680 0x7f96b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96bb7fe640 1 -- 172.21.15.31:0/2126874188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f96c00e7060 con 0x7f96cc1052a0 2024-09-21T02:56:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96cad9d640 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cc5650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f96c400b7a0 tx=0x7f96c400bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fa7ed0275024b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96b8ff9640 1 -- 172.21.15.31:0/1162274130 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4024070 con 0x559c2691e070 2024-09-21T02:56:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96b8ff9640 1 -- 172.21.15.31:0/1162274130 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c401e930 con 0x559c2691e070 2024-09-21T02:56:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.111+0000 7f96b8ff9640 1 -- 172.21.15.31:0/1162274130 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c401c7f0 con 0x559c2691e070 2024-09-21T02:56:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 -- 172.21.15.31:0/1162274130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 msgr2=0x559c26cc5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cc5650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f96c400b7a0 tx=0x7f96c400bc60 comp rx=0 tx=0).stop 2024-09-21T02:56:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96cad9d640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96b40b6680 0x7f96b40b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f96c4021000 tx=0x7f96c401e000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 -- 172.21.15.31:0/1162274130 shutdown_connections 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268f51b0 0x559c26cc5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cc60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 --2- 172.21.15.31:0/1162274130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cc5650 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 -- 172.21.15.31:0/1162274130 >> 172.21.15.31:0/1162274130 conn(0x559c26912460 msgr2=0x559c26cc3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 -- 172.21.15.31:0/1162274130 shutdown_connections 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.112+0000 7f96d7323740 1 -- 172.21.15.31:0/1162274130 wait complete. 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 -- start start 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 Processor -- start 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268f51b0 0x559c26cd0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cdd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559c26a8c700 con 0x559c268fd390 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c26bc6110 con 0x559c268f51b0 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c26cd3e30 con 0x559c2691e070 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96d7323740 10 client.0 _ll_register_callbacks cb 0x559c26bdf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.113+0000 7f96c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cd1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35292/0 (socket says 172.21.15.31:35292) 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 -- 172.21.15.31:0/4222793154 learned_addr learned my addr 172.21.15.31:0/4222793154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96ca59c640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268f51b0 0x559c26cd0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96cad9d640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cdd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 -- 172.21.15.31:0/4222793154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 msgr2=0x559c26cdd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cdd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 -- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268f51b0 msgr2=0x559c26cd0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268f51b0 0x559c26cd0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 -- 172.21.15.31:0/4222793154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c26bf3c40 con 0x559c268fd390 2024-09-21T02:56:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96cad9d640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cdd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cd1260 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f96c00e4830 tx=0x7f96c00ea7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb20c00e79c5a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96b1ffb640 1 -- 172.21.15.31:0/4222793154 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0021070 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96b1ffb640 1 -- 172.21.15.31:0/4222793154 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f96c00e92d0 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c401e930 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96b1ffb640 1 -- 172.21.15.31:0/4222793154 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96c00acd50 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96d7323740 10 client.16596 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.114+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c26cdfb80 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96b1ffb640 1 -- 172.21.15.31:0/4222793154 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c401e930 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96b1ffb640 1 -- 172.21.15.31:0/4222793154 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96c00a88d0 con 0x559c268fd390 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96d7323740 10 client.16596 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96d7323740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96d7323740 20 client.16596 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96d7323740 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268dc9f0 0x559c26ce2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96d7323740 4 client.16596 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c26c3dda0 con 0x559c268dc9f0 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.115+0000 7f96ca59c640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268dc9f0 0x559c26ce2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.116+0000 7f96ca59c640 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268dc9f0 0x559c26ce2ff0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8380e2a7f73fceff server_cookie=9f55555239565054 in_seq=0 out_seq=0 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.116+0000 7f96b1ffb640 10 client.16596 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96b1ffb640 1 -- 172.21.15.31:0/4222793154 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c26c3dda0 con 0x559c268dc9f0 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96b1ffb640 10 client.16596 handle_command_reply: tid=0 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 client.16596 shutdown 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 msgr2=0x559c26cd1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cd1260 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f96c00e4830 tx=0x7f96c00ea7a0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 shutdown_connections 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268dc9f0 0x559c26ce2ff0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cdd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268fd390 0x559c26cd1260 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 --2- 172.21.15.31:0/4222793154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268f51b0 0x559c26cd0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 >> 172.21.15.31:0/4222793154 conn(0x559c26912460 msgr2=0x559c26cc87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96b1ffb640 0 client.16596 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 shutdown_connections 2024-09-21T02:56:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 1 -- 172.21.15.31:0/4222793154 wait complete. 2024-09-21T02:56:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.117+0000 7f96d7323740 20 client.16596 trim_cache size 0 max 16384 2024-09-21T02:56:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 Processor -- start 2024-09-21T02:56:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 -- start start 2024-09-21T02:56:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268fd390 0x559c26cc6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cdd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1af810 con 0x559c268f51b0 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1b1350 con 0x559c2691e070 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96d7323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1afee0 con 0x559c268fd390 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35294/0 (socket says 172.21.15.31:35294) 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.135+0000 7f96ca59c640 1 -- 172.21.15.31:0/2914287219 learned_addr learned my addr 172.21.15.31:0/2914287219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268fd390 0x559c26cc6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96ca59c640 1 -- 172.21.15.31:0/2914287219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268fd390 msgr2=0x559c26cc6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96ca59c640 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268fd390 0x559c26cc6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96ca59c640 1 -- 172.21.15.31:0/2914287219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 msgr2=0x559c26cdd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96ca59c640 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cdd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96ca59c640 1 -- 172.21.15.31:0/2914287219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c26cc7e90 con 0x559c268f51b0 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96ca59c640 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd2c00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f96cc06c890 tx=0x7f96bc009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac875d5c23e9cec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96b3fff640 1 -- 172.21.15.31:0/2914287219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc031650 con 0x559c268f51b0 2024-09-21T02:56:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96b3fff640 1 -- 172.21.15.31:0/2914287219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96bc036040 con 0x559c268f51b0 2024-09-21T02:56:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.136+0000 7f96b3fff640 1 -- 172.21.15.31:0/2914287219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc004600 con 0x559c268f51b0 2024-09-21T02:56:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 -- 172.21.15.31:0/2914287219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 msgr2=0x559c26cd2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd2c00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f96cc06c890 tx=0x7f96bc009410 comp rx=0 tx=0).stop 2024-09-21T02:56:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 -- 172.21.15.31:0/2914287219 shutdown_connections 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c268fd390 0x559c26cc6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c2691e070 0x559c26cdd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 --2- 172.21.15.31:0/2914287219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd2c00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 -- 172.21.15.31:0/2914287219 >> 172.21.15.31:0/2914287219 conn(0x559c26912460 msgr2=0x559c26cc6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 -- 172.21.15.31:0/2914287219 shutdown_connections 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 -- 172.21.15.31:0/2914287219 wait complete. 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.137+0000 7f96d7323740 1 -- start start 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 Processor -- start 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268fd390 0x559c26cd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cd5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1b0c80 con 0x559c268f51b0 2024-09-21T02:56:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1b05b0 con 0x559c268fd390 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1ad7f0 con 0x559c2691e070 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96d7323740 10 client.0 _ll_register_callbacks cb 0x7f96c0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35310/0 (socket says 172.21.15.31:35310) 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 -- 172.21.15.31:0/1604777016 learned_addr learned my addr 172.21.15.31:0/1604777016 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96cad9d640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cd5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268fd390 0x559c26cd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 -- 172.21.15.31:0/1604777016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 msgr2=0x559c26cd5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cd5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 -- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268fd390 msgr2=0x559c26cd5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268fd390 0x559c26cd5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96ca59c640 1 -- 172.21.15.31:0/1604777016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c26d4ca10 con 0x559c268f51b0 2024-09-21T02:56:58.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96cad9d640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cd5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.138+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268fd390 0x559c26cd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96ca59c640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd5110 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f96bc02ade0 tx=0x7f96bc02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ef81e99cb767b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96b1ffb640 1 -- 172.21.15.31:0/1604777016 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc05b070 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96b1ffb640 1 -- 172.21.15.31:0/1604777016 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f96bc031860 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96bc036040 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96b1ffb640 1 -- 172.21.15.31:0/1604777016 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96bc004b40 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 10 client.16602 fetch_fsmap learned FSMap version 27 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c26d4dcd0 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96b1ffb640 1 -- 172.21.15.31:0/1604777016 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96bc036040 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96b1ffb640 1 -- 172.21.15.31:0/1604777016 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f96bc045070 con 0x559c268f51b0 2024-09-21T02:56:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 10 client.16602 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 20 client.16602 populate_metadata read hostname 'smithi031' 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268b1da0 0x559c26d50540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.139+0000 7f96d7323740 4 client.16602 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.140+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c26c39850 con 0x559c268b1da0 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.140+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268b1da0 0x559c26d50540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.140+0000 7f96c9d9b640 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268b1da0 0x559c26d50540 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e805eeac1adcc1da server_cookie=9d89c10b879d1cf6 in_seq=0 out_seq=0 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.140+0000 7f96b1ffb640 10 client.16602 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96b1ffb640 1 -- 172.21.15.31:0/1604777016 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x559c26c39850 con 0x559c268b1da0 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96b1ffb640 10 client.16602 handle_command_reply: tid=0 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 client.16602 shutdown 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 msgr2=0x559c26cd5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd5110 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f96bc02ade0 tx=0x7f96bc02f470 comp rx=0 tx=0).stop 2024-09-21T02:56:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 shutdown_connections 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c268b1da0 0x559c26d50540 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c2691e070 0x559c26cd5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c268fd390 0x559c26cd5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96b1ffb640 0 client.16602 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 --2- 172.21.15.31:0/1604777016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c268f51b0 0x559c26cd5110 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 >> 172.21.15.31:0/1604777016 conn(0x559c26912460 msgr2=0x559c26cd29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 shutdown_connections 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 1 -- 172.21.15.31:0/1604777016 wait complete. 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.141+0000 7f96d7323740 20 client.16602 trim_cache size 0 max 16384 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:56:58.148 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "a751537f-fbc5-4cd8-b5f6-405f14d5969d", 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:56:58.150 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:56:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:56:58.154 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96b40b6680 msgr2=0x7f96b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96b40b6680 0x7f96b40b8b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f96c4021000 tx=0x7f96c401e000 comp rx=0 tx=0).stop 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 msgr2=0x7f96cc19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 0x7f96cc19ab00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f96c000d740 tx=0x7f96c000dc00 comp rx=0 tx=0).stop 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 shutdown_connections 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f96b40b6680 0x7f96b40b8b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96cc10de70 0x7f96cc19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96cc1052a0 0x7f96cc19ab00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 --2- 172.21.15.31:0/2126874188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96cc1048c0 0x7f96cc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 >> 172.21.15.31:0/2126874188 conn(0x7f96cc079760 msgr2=0x7f96cc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 shutdown_connections 2024-09-21T02:56:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:56:58.142+0000 7f96d0daf640 1 -- 172.21.15.31:0/2126874188 wait complete. 2024-09-21T02:56:58.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:57 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:56:58.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:57 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:58.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:57 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:56:58.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:57 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:57 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:56:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:57 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:57 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:56:58.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:57 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:56:58.931 INFO:tasks.cephfs.filesystem:scrub status for tag:a751537f-fbc5-4cd8-b5f6-405f14d5969d - {'path': '/', 'tag': 'a751537f-fbc5-4cd8-b5f6-405f14d5969d', 'options': 'recursive,force'} 2024-09-21T02:56:59.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:56:58 smithi178 ceph-mon[34759]: pgmap v447: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-09-21T02:56:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:56:58 smithi165 ceph-mon[34818]: pgmap v447: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-09-21T02:56:59.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:56:58 smithi031 ceph-mon[31209]: pgmap v447: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-09-21T02:57:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:00 smithi165 ceph-mon[34818]: pgmap v448: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:00 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:57:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:00 smithi178 ceph-mon[34759]: pgmap v448: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:00 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:57:01.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:00 smithi031 ceph-mon[31209]: pgmap v448: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:01.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:00 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:57:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:02 smithi165 ceph-mon[34818]: pgmap v449: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:57:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:02 smithi178 ceph-mon[34759]: pgmap v449: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:57:03.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:02 smithi031 ceph-mon[31209]: pgmap v449: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:57:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:04 smithi165 ceph-mon[34818]: pgmap v450: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:57:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:04 smithi178 ceph-mon[34759]: pgmap v450: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:57:05.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:04 smithi031 ceph-mon[31209]: pgmap v450: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:57:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:06 smithi165 ceph-mon[34818]: pgmap v451: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:57:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:06 smithi178 ceph-mon[34759]: pgmap v451: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:57:07.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:06 smithi031 ceph-mon[31209]: pgmap v451: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:57:09.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:08 smithi031 ceph-mon[31209]: pgmap v452: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T02:57:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:08 smithi165 ceph-mon[34818]: pgmap v452: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T02:57:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:08 smithi178 ceph-mon[34759]: pgmap v452: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T02:57:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:10 smithi165 ceph-mon[34818]: pgmap v453: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:57:11.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:10 smithi178 ceph-mon[34759]: pgmap v453: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:57:11.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:10 smithi031 ceph-mon[31209]: pgmap v453: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:57:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:12 smithi165 ceph-mon[34818]: pgmap v454: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:57:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:12 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:12 smithi178 ceph-mon[34759]: pgmap v454: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:57:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:12 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:13.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:12 smithi031 ceph-mon[31209]: pgmap v454: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:57:13.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:12 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:57:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:57:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:57:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:14 smithi165 ceph-mon[34818]: pgmap v455: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:57:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:14 smithi178 ceph-mon[34759]: pgmap v455: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:57:15.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:14 smithi031 ceph-mon[31209]: pgmap v455: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:57:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:16 smithi165 ceph-mon[34818]: pgmap v456: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:57:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:16 smithi178 ceph-mon[34759]: pgmap v456: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:57:17.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:16 smithi031 ceph-mon[31209]: pgmap v456: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:57:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:18 smithi165 ceph-mon[34818]: pgmap v457: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:57:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:18 smithi178 ceph-mon[34759]: pgmap v457: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:57:19.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:18 smithi031 ceph-mon[31209]: pgmap v457: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:57:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:20 smithi165 ceph-mon[34818]: pgmap v458: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:57:21.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:20 smithi178 ceph-mon[34759]: pgmap v458: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:57:21.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:20 smithi031 ceph-mon[31209]: pgmap v458: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:57:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:22 smithi165 ceph-mon[34818]: pgmap v459: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:57:23.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:22 smithi178 ceph-mon[34759]: pgmap v459: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:57:23.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:22 smithi031 ceph-mon[31209]: pgmap v459: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:57:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:24 smithi165 ceph-mon[34818]: pgmap v460: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:57:25.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:24 smithi178 ceph-mon[34759]: pgmap v460: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:57:25.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:24 smithi031 ceph-mon[31209]: pgmap v460: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:57:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:26 smithi165 ceph-mon[34818]: pgmap v461: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:27.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:26 smithi178 ceph-mon[34759]: pgmap v461: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:27.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:26 smithi031 ceph-mon[31209]: pgmap v461: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:28.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:57:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:28 smithi165 ceph-mon[34818]: pgmap v462: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:57:29.205 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:29.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:28 smithi178 ceph-mon[34759]: pgmap v462: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:57:29.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:28 smithi031 ceph-mon[31209]: pgmap v462: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:57:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 -- 172.21.15.31:0/952822236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01020f0 msgr2=0x7f92d0102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/952822236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01020f0 0x7f92d0102500 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009900 tx=0x7f92cc02a410 comp rx=0 tx=0).stop 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 -- 172.21.15.31:0/952822236 shutdown_connections 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/952822236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d0106c30 0x7f92d0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/952822236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92d01062e0 0x7f92d01066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/952822236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01020f0 0x7f92d0102500 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 -- 172.21.15.31:0/952822236 >> 172.21.15.31:0/952822236 conn(0x7f92d0077cc0 msgr2=0x7f92d0078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 -- 172.21.15.31:0/952822236 shutdown_connections 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.768+0000 7f92d7cf7640 1 -- 172.21.15.31:0/952822236 wait complete. 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 Processor -- start 2024-09-21T02:57:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 -- start start 2024-09-21T02:57:29.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 0x7f92d019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01062e0 0x7f92d019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92d0106c30 0x7f92d019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0113aa0 con 0x7f92d01062e0 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0113920 con 0x7f92d0106c30 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.769+0000 7f92d7cf7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0113c20 con 0x7f92d01020f0 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 0x7f92d019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 0x7f92d019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:53234/0 (socket says 172.21.15.31:53234) 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 -- 172.21.15.31:0/4019588316 learned_addr learned my addr 172.21.15.31:0/4019588316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5292640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01062e0 0x7f92d019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 -- 172.21.15.31:0/4019588316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92d0106c30 msgr2=0x7f92d019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92d0106c30 0x7f92d019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 -- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01062e0 msgr2=0x7f92d019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01062e0 0x7f92d019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 -- 172.21.15.31:0/4019588316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d01a0440 con 0x7f92d01020f0 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.770+0000 7f92d5a93640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 0x7f92d019a4d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005940 tx=0x7f92cc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:29.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.771+0000 7f92c6ffd640 1 -- 172.21.15.31:0/4019588316 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc03f030 con 0x7f92d01020f0 2024-09-21T02:57:29.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.771+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d01a06d0 con 0x7f92d01020f0 2024-09-21T02:57:29.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.771+0000 7f92c6ffd640 1 -- 172.21.15.31:0/4019588316 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92cc003aa0 con 0x7f92d01020f0 2024-09-21T02:57:29.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.771+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d01a6e90 con 0x7f92d01020f0 2024-09-21T02:57:29.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.771+0000 7f92c6ffd640 1 -- 172.21.15.31:0/4019588316 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc035070 con 0x7f92d01020f0 2024-09-21T02:57:29.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.773+0000 7f92de26b740 1 Processor -- start 2024-09-21T02:57:29.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 -- start start 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d67b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d67b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d67bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56328d5f1e50 con 0x56328d2ab1b0 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56328d5e1280 con 0x56328d2d4070 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92de26b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56328d5e5090 con 0x56328d2b3390 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92c6ffd640 1 -- 172.21.15.31:0/4019588316 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f92cc042030 con 0x7f92d01020f0 2024-09-21T02:57:29.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92d5292640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d67bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92d5292640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d67bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:53248/0 (socket says 172.21.15.31:53248) 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92d5292640 1 -- 172.21.15.31:0/3184599192 learned_addr learned my addr 172.21.15.31:0/3184599192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92d6294640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d67b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.774+0000 7f92d5a93640 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d67b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5292640 1 -- 172.21.15.31:0/3184599192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 msgr2=0x56328d67b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5292640 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d67b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5292640 1 -- 172.21.15.31:0/3184599192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 msgr2=0x56328d67b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5292640 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d67b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5292640 1 -- 172.21.15.31:0/3184599192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328d33f650 con 0x56328d2b3390 2024-09-21T02:57:29.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5a93640 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d67b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:29.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92d5292640 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d67bcf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92c000bc70 tx=0x7f92c00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f06a9fc20eb2308 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92bbfff640 1 -- 172.21.15.31:0/3184599192 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0005220 con 0x56328d2b3390 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92c6ffd640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f92bc0b6700 0x7f92bc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92bbfff640 1 -- 172.21.15.31:0/3184599192 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c000be20 con 0x56328d2b3390 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.775+0000 7f92bbfff640 1 -- 172.21.15.31:0/3184599192 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00116a0 con 0x56328d2b3390 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.776+0000 7f92c6ffd640 1 -- 172.21.15.31:0/4019588316 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f92cc107e00 con 0x7f92d01020f0 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.776+0000 7f92d6294640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f92bc0b6700 0x7f92bc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 -- 172.21.15.31:0/3184599192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 msgr2=0x56328d67bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d67bcf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92c000bc70 tx=0x7f92c00107a0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92d6294640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f92bc0b6700 0x7f92bc0b8c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009270 tx=0x7f92c8009200 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 -- 172.21.15.31:0/3184599192 shutdown_connections 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d67bcf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d67b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 --2- 172.21.15.31:0/3184599192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d67b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 -- 172.21.15.31:0/3184599192 >> 172.21.15.31:0/3184599192 conn(0x56328d2c8460 msgr2=0x56328d679160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:29.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 -- 172.21.15.31:0/3184599192 shutdown_connections 2024-09-21T02:57:29.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 -- 172.21.15.31:0/3184599192 wait complete. 2024-09-21T02:57:29.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 -- start start 2024-09-21T02:57:29.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.777+0000 7f92de26b740 1 Processor -- start 2024-09-21T02:57:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d686950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2b3390 0x56328d686e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2d4070 0x56328d692d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56328d442700 con 0x56328d2ab1b0 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56328d57c110 con 0x56328d2b3390 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56328d689a50 con 0x56328d2d4070 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92de26b740 10 client.0 _ll_register_callbacks cb 0x56328d595990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d686950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d686950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48390/0 (socket says 172.21.15.31:48390) 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 -- 172.21.15.31:0/2671394993 learned_addr learned my addr 172.21.15.31:0/2671394993 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5292640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2b3390 0x56328d686e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d6294640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2d4070 0x56328d692d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 -- 172.21.15.31:0/2671394993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2d4070 msgr2=0x56328d692d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2d4070 0x56328d692d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 -- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2b3390 msgr2=0x56328d686e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2b3390 0x56328d686e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.778+0000 7f92d5a93640 1 -- 172.21.15.31:0/2671394993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328d5a9c40 con 0x56328d2ab1b0 2024-09-21T02:57:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92d5a93640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d686950 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f92cc006010 tx=0x7f92cc003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac81dd1451b4098 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92b97fa640 1 -- 172.21.15.31:0/2671394993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc049070 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92b97fa640 1 -- 172.21.15.31:0/2671394993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f92cc0f10b0 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c000be20 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92b97fa640 1 -- 172.21.15.31:0/2671394993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92cc0cda40 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92de26b740 10 client.16614 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56328d695790 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.779+0000 7f92b97fa640 1 -- 172.21.15.31:0/2671394993 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c000be20 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92b97fa640 1 -- 172.21.15.31:0/2671394993 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f92cc0d7200 con 0x56328d2ab1b0 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92de26b740 10 client.16614 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92de26b740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92de26b740 20 client.16614 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92de26b740 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d2929f0 0x56328d698bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92de26b740 4 client.16614 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328d5f3da0 con 0x56328d2929f0 2024-09-21T02:57:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.780+0000 7f92d5292640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d2929f0 0x56328d698bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.781+0000 7f92d5292640 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d2929f0 0x56328d698bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ed6809d2e09cd1 server_cookie=f03d1a2a3610d6b3 in_seq=0 out_seq=0 2024-09-21T02:57:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.781+0000 7f92b97fa640 10 client.16614 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92b97fa640 1 -- 172.21.15.31:0/2671394993 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56328d5f3da0 con 0x56328d2929f0 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92b97fa640 10 client.16614 handle_command_reply: tid=0 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 client.16614 shutdown 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 msgr2=0x56328d686950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d686950 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f92cc006010 tx=0x7f92cc003e80 comp rx=0 tx=0).stop 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 shutdown_connections 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d2929f0 0x56328d698bd0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2d4070 0x56328d692d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2b3390 0x56328d686e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 --2- 172.21.15.31:0/2671394993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d686950 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 >> 172.21.15.31:0/2671394993 conn(0x56328d2c8460 msgr2=0x56328d67e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.782+0000 7f92b97fa640 0 client.16614 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.783+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 shutdown_connections 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.783+0000 7f92de26b740 1 -- 172.21.15.31:0/2671394993 wait complete. 2024-09-21T02:57:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.783+0000 7f92de26b740 20 client.16614 trim_cache size 0 max 16384 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 Processor -- start 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 -- start start 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2ab1b0 0x56328d67cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d688580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 0x56328d692c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01af620 con 0x56328d2d4070 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01b1160 con 0x56328d2ab1b0 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92de26b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01afcf0 con 0x56328d2b3390 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92d6294640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 0x56328d692c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92d6294640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 0x56328d692c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48402/0 (socket says 172.21.15.31:48402) 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.801+0000 7f92d6294640 1 -- 172.21.15.31:0/705877167 learned_addr learned my addr 172.21.15.31:0/705877167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:29.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d5a93640 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2ab1b0 0x56328d67cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d5292640 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d688580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d6294640 1 -- 172.21.15.31:0/705877167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 msgr2=0x56328d688580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d6294640 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d688580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d6294640 1 -- 172.21.15.31:0/705877167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2ab1b0 msgr2=0x56328d67cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d6294640 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2ab1b0 0x56328d67cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d6294640 1 -- 172.21.15.31:0/705877167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328d684e50 con 0x56328d2d4070 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d5a93640 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2ab1b0 0x56328d67cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92d6294640 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 0x56328d692c40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009ae0 tx=0x7f92c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e7ed5459805337 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92bb7fe640 1 -- 172.21.15.31:0/705877167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004b20 con 0x56328d2d4070 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92bb7fe640 1 -- 172.21.15.31:0/705877167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c8004cc0 con 0x56328d2d4070 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.802+0000 7f92bb7fe640 1 -- 172.21.15.31:0/705877167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c801e780 con 0x56328d2d4070 2024-09-21T02:57:29.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 -- 172.21.15.31:0/705877167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 msgr2=0x56328d692c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 0x56328d692c40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009ae0 tx=0x7f92c8014000 comp rx=0 tx=0).stop 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 -- 172.21.15.31:0/705877167 shutdown_connections 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d688580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2d4070 0x56328d692c40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 --2- 172.21.15.31:0/705877167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2ab1b0 0x56328d67cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 -- 172.21.15.31:0/705877167 >> 172.21.15.31:0/705877167 conn(0x56328d2c8460 msgr2=0x56328d687e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 -- 172.21.15.31:0/705877167 shutdown_connections 2024-09-21T02:57:29.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.803+0000 7f92de26b740 1 -- 172.21.15.31:0/705877167 wait complete. 2024-09-21T02:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.804+0000 7f92de26b740 1 -- start start 2024-09-21T02:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.804+0000 7f92de26b740 1 Processor -- start 2024-09-21T02:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.804+0000 7f92de26b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d68ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92de26b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d68b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92de26b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d68b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92de26b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01b0a90 con 0x56328d2ab1b0 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92de26b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01b03c0 con 0x56328d2d4070 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92de26b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ad600 con 0x56328d2b3390 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92de26b740 10 client.0 _ll_register_callbacks cb 0x7f92cc005e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d68ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d68ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48418/0 (socket says 172.21.15.31:48418) 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 -- 172.21.15.31:0/901971669 learned_addr learned my addr 172.21.15.31:0/901971669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d6294640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d68b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5292640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d68b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 -- 172.21.15.31:0/901971669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 msgr2=0x56328d68b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d68b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 -- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 msgr2=0x56328d68b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d68b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 -- 172.21.15.31:0/901971669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328d7027c0 con 0x56328d2ab1b0 2024-09-21T02:57:29.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92d5a93640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d68ae90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f92cc006010 tx=0x7f92cc0f12c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673f17b0a15ddcab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.805+0000 7f92b97fa640 1 -- 172.21.15.31:0/901971669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc049070 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92b97fa640 1 -- 172.21.15.31:0/901971669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f92cc0037f0 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c8004cc0 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92b97fa640 1 -- 172.21.15.31:0/901971669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92cc0cd3e0 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 10 client.16620 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56328d707e90 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92b97fa640 1 -- 172.21.15.31:0/901971669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c8004cc0 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92b97fa640 1 -- 172.21.15.31:0/901971669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f92cc0c95c0 con 0x56328d2ab1b0 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 10 client.16620 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 20 client.16620 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d267da0 0x56328d70a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:29.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 4 client.16620 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:57:29.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.806+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56328d5ef850 con 0x56328d267da0 2024-09-21T02:57:29.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.807+0000 7f92d5292640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d267da0 0x56328d70a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:29.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.807+0000 7f92d5292640 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d267da0 0x56328d70a510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45d1dd9a933a328 server_cookie=b148d3b38365112 in_seq=0 out_seq=0 2024-09-21T02:57:29.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.807+0000 7f92b97fa640 10 client.16620 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.807+0000 7f92b97fa640 1 -- 172.21.15.31:0/901971669 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x56328d5ef850 con 0x56328d267da0 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.807+0000 7f92b97fa640 10 client.16620 handle_command_reply: tid=0 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.807+0000 7f92de26b740 1 client.16620 shutdown 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 msgr2=0x56328d68ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d68ae90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f92cc006010 tx=0x7f92cc0f12c0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 shutdown_connections 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56328d267da0 0x56328d70a510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56328d2b3390 0x56328d68b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56328d2d4070 0x56328d68b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92b97fa640 0 client.16620 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:29.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 --2- 172.21.15.31:0/901971669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56328d2ab1b0 0x56328d68ae90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 >> 172.21.15.31:0/901971669 conn(0x56328d2c8460 msgr2=0x56328d67ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 shutdown_connections 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 1 -- 172.21.15.31:0/901971669 wait complete. 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.808+0000 7f92de26b740 20 client.16620 trim_cache size 0 max 16384 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:57:29.814 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.815 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:29.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:57:29.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:57:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:57:29.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f92bc0b6700 msgr2=0x7f92bc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f92bc0b6700 0x7f92bc0b8c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009270 tx=0x7f92c8009200 comp rx=0 tx=0).stop 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 msgr2=0x7f92d019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 0x7f92d019a4d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005940 tx=0x7f92cc004270 comp rx=0 tx=0).stop 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 shutdown_connections 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f92bc0b6700 0x7f92bc0b8c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92d0106c30 0x7f92d019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d01062e0 0x7f92d019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 --2- 172.21.15.31:0/4019588316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d01020f0 0x7f92d019a4d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 >> 172.21.15.31:0/4019588316 conn(0x7f92d0077cc0 msgr2=0x7f92d01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 shutdown_connections 2024-09-21T02:57:29.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:29.809+0000 7f92d7cf7640 1 -- 172.21.15.31:0/4019588316 wait complete. 2024-09-21T02:57:30.724 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:57:30.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:57:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:30 smithi031 ceph-mon[31209]: pgmap v463: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:30 smithi165 ceph-mon[34818]: pgmap v463: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:31.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:30 smithi178 ceph-mon[34759]: pgmap v463: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 -- 172.21.15.31:0/3270362660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670109df0 msgr2=0x7fc670110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 --2- 172.21.15.31:0/3270362660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670109df0 0x7fc670110620 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc668009ca0 tx=0x7fc66802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 -- 172.21.15.31:0/3270362660 shutdown_connections 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 --2- 172.21.15.31:0/3270362660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670109df0 0x7fc670110620 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 --2- 172.21.15.31:0/3270362660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6701094a0 0x7fc6701098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 --2- 172.21.15.31:0/3270362660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc670103b60 0x7fc670103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 -- 172.21.15.31:0/3270362660 >> 172.21.15.31:0/3270362660 conn(0x7fc67007b850 msgr2=0x7fc67007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 -- 172.21.15.31:0/3270362660 shutdown_connections 2024-09-21T02:57:31.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.492+0000 7fc67693e640 1 -- 172.21.15.31:0/3270362660 wait complete. 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 Processor -- start 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 -- start start 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 0x7fc67019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6701094a0 0x7fc67019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc670109df0 0x7fc67019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc670113ab0 con 0x7fc670103b60 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc670113930 con 0x7fc6701094a0 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc67693e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc670113c30 con 0x7fc670109df0 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 0x7fc67019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 0x7fc67019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48436/0 (socket says 172.21.15.31:48436) 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 -- 172.21.15.31:0/771082631 learned_addr learned my addr 172.21.15.31:0/771082631 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc667fff640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6701094a0 0x7fc67019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc674edb640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc670109df0 0x7fc67019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 -- 172.21.15.31:0/771082631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc670109df0 msgr2=0x7fc67019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc670109df0 0x7fc67019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 -- 172.21.15.31:0/771082631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6701094a0 msgr2=0x7fc67019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6701094a0 0x7fc67019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.493+0000 7fc66ffff640 1 -- 172.21.15.31:0/771082631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6701a03f0 con 0x7fc670103b60 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.494+0000 7fc66ffff640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 0x7fc67019a480 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc66000d770 tx=0x7fc66000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.494+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660017070 con 0x7fc670103b60 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.494+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6701a06e0 con 0x7fc670103b60 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.494+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6600054d0 con 0x7fc670103b60 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.494+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660004c30 con 0x7fc670103b60 2024-09-21T02:57:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.494+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6701a6e40 con 0x7fc670103b60 2024-09-21T02:57:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.495+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc670103f70 con 0x7fc670103b60 2024-09-21T02:57:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.497+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc660004dd0 con 0x7fc670103b60 2024-09-21T02:57:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.498+0000 7fc66dffb640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc65c0b6680 0x7fc65c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.498+0000 7fc667fff640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc65c0b6680 0x7fc65c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.498+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc6600e7600 con 0x7fc670103b60 2024-09-21T02:57:31.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.498+0000 7fc667fff640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc65c0b6680 0x7fc65c0b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc670106f40 tx=0x7fc65800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:31.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.501+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6600adb10 con 0x7fc670103b60 2024-09-21T02:57:31.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.727+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67002e640 con 0x7fc670103b60 2024-09-21T02:57:31.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.728+0000 7fc66dffb640 1 -- 172.21.15.31:0/771082631 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fc6600b1ef0 con 0x7fc670103b60 2024-09-21T02:57:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:57:31.730 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:57:31.730 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc65c0b6680 msgr2=0x7fc65c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc65c0b6680 0x7fc65c0b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc670106f40 tx=0x7fc65800b390 comp rx=0 tx=0).stop 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 msgr2=0x7fc67019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 0x7fc67019a480 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc66000d770 tx=0x7fc66000dc30 comp rx=0 tx=0).stop 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 shutdown_connections 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc65c0b6680 0x7fc65c0b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc670109df0 0x7fc67019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6701094a0 0x7fc67019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 --2- 172.21.15.31:0/771082631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc670103b60 0x7fc67019a480 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 >> 172.21.15.31:0/771082631 conn(0x7fc67007b850 msgr2=0x7fc670105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 shutdown_connections 2024-09-21T02:57:31.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:31.730+0000 7fc67693e640 1 -- 172.21.15.31:0/771082631 wait complete. 2024-09-21T02:57:32.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:31 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/771082631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:32.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:31 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/771082631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:32.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:31 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/771082631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:32.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:57:32.798 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:32 smithi165 ceph-mon[34818]: pgmap v464: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:57:33.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:32 smithi178 ceph-mon[34759]: pgmap v464: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:57:33.251 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:32 smithi031 ceph-mon[31209]: pgmap v464: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 -- 172.21.15.31:0/1141809273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 msgr2=0x7f963410fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 --2- 172.21.15.31:0/1141809273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963410fcc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9628009900 tx=0x7f962802a480 comp rx=0 tx=0).stop 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 -- 172.21.15.31:0/1141809273 shutdown_connections 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 --2- 172.21.15.31:0/1141809273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634110200 0x7f9634112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 --2- 172.21.15.31:0/1141809273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963410fcc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 --2- 172.21.15.31:0/1141809273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634102650 0x7f9634102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 -- 172.21.15.31:0/1141809273 >> 172.21.15.31:0/1141809273 conn(0x7f9634079740 msgr2=0x7f9634079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 -- 172.21.15.31:0/1141809273 shutdown_connections 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.345+0000 7f963aaae640 1 -- 172.21.15.31:0/1141809273 wait complete. 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 Processor -- start 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 -- start start 2024-09-21T02:57:33.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634102650 0x7f963419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634110200 0x7f96341a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9634115c70 con 0x7f9634102fa0 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9634115af0 con 0x7f9634110200 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963aaae640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9634115df0 con 0x7f9634102650 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48468/0 (socket says 172.21.15.31:48468) 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 -- 172.21.15.31:0/164947627 learned_addr learned my addr 172.21.15.31:0/164947627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963904b640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634110200 0x7f96341a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963884a640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634102650 0x7f963419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 -- 172.21.15.31:0/164947627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634102650 msgr2=0x7f963419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634102650 0x7f963419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 -- 172.21.15.31:0/164947627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634110200 msgr2=0x7f96341a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634110200 0x7f96341a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f9633fff640 1 -- 172.21.15.31:0/164947627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96341a25b0 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963884a640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634102650 0x7f963419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.346+0000 7f963904b640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634110200 0x7f96341a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.347+0000 7f9633fff640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963419cca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9628005940 tx=0x7f9628004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.347+0000 7f9631ffb640 1 -- 172.21.15.31:0/164947627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962803f030 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.347+0000 7f9631ffb640 1 -- 172.21.15.31:0/164947627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9628003aa0 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.347+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96341a2840 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.347+0000 7f9631ffb640 1 -- 172.21.15.31:0/164947627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628035070 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.347+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96341a2d20 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 Processor -- start 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 -- start start 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9631ffb640 1 -- 172.21.15.31:0/164947627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9628042030 con 0x7f9634102fa0 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eda271fea0 con 0x55eda23d91b0 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eda270f2d0 con 0x55eda2402070 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f9641022740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eda27130e0 con 0x55eda23e1390 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.350+0000 7f963884a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48484/0 (socket says 172.21.15.31:48484) 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 -- 172.21.15.31:0/4111105614 learned_addr learned my addr 172.21.15.31:0/4111105614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963904b640 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f9633fff640 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 -- 172.21.15.31:0/4111105614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 msgr2=0x55eda27a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 -- 172.21.15.31:0/4111105614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 msgr2=0x55eda27a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 -- 172.21.15.31:0/4111105614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda246d650 con 0x55eda23d91b0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963904b640 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f9631ffb640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f961c0b66b0 0x7f961c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963904b640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f961c0b66b0 0x7f961c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f963884a640 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27a9800 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9624009d50 tx=0x7f96240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1988b0b153ee43a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f9631ffb640 1 -- 172.21.15.31:0/164947627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9628107c30 con 0x7f9634102fa0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.351+0000 7f961affd640 1 -- 172.21.15.31:0/4111105614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624012650 con 0x55eda23d91b0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f961affd640 1 -- 172.21.15.31:0/4111105614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9624009e40 con 0x55eda23d91b0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f961affd640 1 -- 172.21.15.31:0/4111105614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962400fb90 con 0x55eda23d91b0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f9641022740 1 -- 172.21.15.31:0/4111105614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 msgr2=0x55eda27a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f963904b640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f961c0b66b0 0x7f961c0b8bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f962c007fd0 tx=0x7f962c0076a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f9641022740 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27a9800 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9624009d50 tx=0x7f96240117b0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f9641022740 1 -- 172.21.15.31:0/4111105614 shutdown_connections 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f9641022740 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f9641022740 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27a9800 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.352+0000 7f9641022740 1 --2- 172.21.15.31:0/4111105614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.353+0000 7f9641022740 1 -- 172.21.15.31:0/4111105614 >> 172.21.15.31:0/4111105614 conn(0x55eda23f6460 msgr2=0x55eda27a71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.353+0000 7f9641022740 1 -- 172.21.15.31:0/4111105614 shutdown_connections 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.353+0000 7f9641022740 1 -- 172.21.15.31:0/4111105614 wait complete. 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.353+0000 7f9641022740 1 -- start start 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.353+0000 7f9641022740 1 Processor -- start 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23d91b0 0x55eda27b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eda2570700 con 0x55eda23e1390 2024-09-21T02:57:33.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eda26aa160 con 0x55eda2402070 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eda27b7aa0 con 0x55eda23d91b0 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9641022740 10 client.0 _ll_register_callbacks cb 0x55eda26c39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48492/0 (socket says 172.21.15.31:48492) 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 -- 172.21.15.31:0/1669591092 learned_addr learned my addr 172.21.15.31:0/1669591092 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f963884a640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23d91b0 0x55eda27b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f963904b640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 -- 172.21.15.31:0/1669591092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23d91b0 msgr2=0x55eda27b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23d91b0 0x55eda27b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 -- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 msgr2=0x55eda27b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 -- 172.21.15.31:0/1669591092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda26d7c90 con 0x55eda23e1390 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f963904b640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:33.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9633fff640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27b4e70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9628105400 tx=0x7f962810b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e961f65717637bad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.354+0000 7f9613fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9613fff640 1 -- 172.21.15.31:0/1669591092 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628049070 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9613fff640 1 -- 172.21.15.31:0/1669591092 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f96280037f0 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eda2721df0 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9613fff640 1 -- 172.21.15.31:0/1669591092 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f96280ca0f0 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 10 client.16638 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eda27c3780 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9613fff640 1 -- 172.21.15.31:0/1669591092 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eda2721df0 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9613fff640 1 -- 172.21.15.31:0/1669591092 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9628109070 con 0x55eda23e1390 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 10 client.16638 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 10 client.16638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 20 client.16638 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda23c09f0 0x55eda27c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 4 client.16638 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.355+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eda271d8a0 con 0x55eda23c09f0 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.356+0000 7f963884a640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda23c09f0 0x55eda27c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.356+0000 7f963884a640 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda23c09f0 0x55eda27c6d00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b04e3c08ff71d93 server_cookie=d7850c16e1a8b135 in_seq=0 out_seq=0 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.356+0000 7f9613fff640 10 client.16638 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9613fff640 1 -- 172.21.15.31:0/1669591092 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eda271d8a0 con 0x55eda23c09f0 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9613fff640 10 client.16638 handle_command_reply: tid=0 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 client.16638 shutdown 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 msgr2=0x55eda27b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27b4e70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9628105400 tx=0x7f962810b580 comp rx=0 tx=0).stop 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 shutdown_connections 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27b4e70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda23c09f0 0x55eda27c6d00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 --2- 172.21.15.31:0/1669591092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23d91b0 0x55eda27b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9613fff640 0 client.16638 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:57:33.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 >> 172.21.15.31:0/1669591092 conn(0x55eda23f6460 msgr2=0x55eda27ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 shutdown_connections 2024-09-21T02:57:33.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 1 -- 172.21.15.31:0/1669591092 wait complete. 2024-09-21T02:57:33.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.357+0000 7f9641022740 20 client.16638 trim_cache size 0 max 16384 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.371+0000 7f9641022740 1 Processor -- start 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.371+0000 7f9641022740 1 -- start start 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.371+0000 7f9641022740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda23d91b0 0x55eda27aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.371+0000 7f9641022740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.371+0000 7f9641022740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda2402070 0x55eda27c0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9641022740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b1820 con 0x55eda23e1390 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9641022740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b3360 con 0x55eda23d91b0 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9641022740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b1ef0 con 0x55eda2402070 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27c03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48494/0 (socket says 172.21.15.31:48494) 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 -- 172.21.15.31:0/3207066513 learned_addr learned my addr 172.21.15.31:0/3207066513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f963884a640 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda23d91b0 0x55eda27aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 -- 172.21.15.31:0/3207066513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda2402070 msgr2=0x55eda27c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda2402070 0x55eda27c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 -- 172.21.15.31:0/3207066513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda23d91b0 msgr2=0x55eda27aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda23d91b0 0x55eda27aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 -- 172.21.15.31:0/3207066513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eda27a7f70 con 0x55eda23e1390 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f9633fff640 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27c03f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f96280cda30 tx=0x7f96280d7600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13099d486c43e5a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f961a7fc640 1 -- 172.21.15.31:0/3207066513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628049070 con 0x55eda23e1390 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f961a7fc640 1 -- 172.21.15.31:0/3207066513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962810b330 con 0x55eda23e1390 2024-09-21T02:57:33.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.372+0000 7f961a7fc640 1 -- 172.21.15.31:0/3207066513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96280c9210 con 0x55eda23e1390 2024-09-21T02:57:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 -- 172.21.15.31:0/3207066513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 msgr2=0x55eda27c03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27c03f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f96280cda30 tx=0x7f96280d7600 comp rx=0 tx=0).stop 2024-09-21T02:57:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 -- 172.21.15.31:0/3207066513 shutdown_connections 2024-09-21T02:57:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda2402070 0x55eda27c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23e1390 0x55eda27c03f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 --2- 172.21.15.31:0/3207066513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda23d91b0 0x55eda27aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 -- 172.21.15.31:0/3207066513 >> 172.21.15.31:0/3207066513 conn(0x55eda23f6460 msgr2=0x55eda27bfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 -- 172.21.15.31:0/3207066513 shutdown_connections 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.373+0000 7f9641022740 1 -- 172.21.15.31:0/3207066513 wait complete. 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 -- start start 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 Processor -- start 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b2c90 con 0x55eda23d91b0 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b25c0 con 0x55eda2402070 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96341af800 con 0x55eda23e1390 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.374+0000 7f9641022740 10 client.0 _ll_register_callbacks cb 0x7f9628105850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27bd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48500/0 (socket says 172.21.15.31:48500) 2024-09-21T02:57:33.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 -- 172.21.15.31:0/1217686503 learned_addr learned my addr 172.21.15.31:0/1217686503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963904b640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f9633fff640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 -- 172.21.15.31:0/1217686503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 msgr2=0x55eda27b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27b89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 -- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 msgr2=0x55eda27b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 -- 172.21.15.31:0/1217686503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eda27b9620 con 0x55eda23d91b0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f963884a640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27bd4f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9624009c70 tx=0x7f96240050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc26ac22ad5cc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f9613fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f9613fff640 1 -- 172.21.15.31:0/1217686503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624005470 con 0x55eda23d91b0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f9613fff640 1 -- 172.21.15.31:0/1217686503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f96240035f0 con 0x55eda23d91b0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.375+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962810b330 con 0x55eda23d91b0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9613fff640 1 -- 172.21.15.31:0/1217686503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9624005610 con 0x55eda23d91b0 2024-09-21T02:57:33.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 10 client.16644 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eda28315b0 con 0x55eda23d91b0 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9613fff640 1 -- 172.21.15.31:0/1217686503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962810b330 con 0x55eda23d91b0 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9613fff640 1 -- 172.21.15.31:0/1217686503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9624038070 con 0x55eda23d91b0 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 10 client.16644 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 10 client.16644 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 20 client.16644 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda2395da0 0x55eda2833c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 4 client.16644 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eda271acd0 con 0x55eda2395da0 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.376+0000 7f9633fff640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda2395da0 0x55eda2833c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9633fff640 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda2395da0 0x55eda2833c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f542a69e00a4b7e server_cookie=1ec72010caf85923 in_seq=0 out_seq=0 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9613fff640 10 client.16644 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:57:33.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9613fff640 1 -- 172.21.15.31:0/1217686503 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eda271acd0 con 0x55eda2395da0 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9613fff640 10 client.16644 handle_command_reply: tid=0 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 client.16644 shutdown 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 msgr2=0x55eda27bd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27bd4f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9624009c70 tx=0x7f96240050b0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 shutdown_connections 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55eda2395da0 0x55eda2833c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eda23e1390 0x55eda27b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eda2402070 0x55eda27b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 --2- 172.21.15.31:0/1217686503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eda23d91b0 0x55eda27bd4f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 >> 172.21.15.31:0/1217686503 conn(0x55eda23f6460 msgr2=0x55eda27aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9613fff640 0 client.16644 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 shutdown_connections 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.377+0000 7f9641022740 1 -- 172.21.15.31:0/1217686503 wait complete. 2024-09-21T02:57:33.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f9641022740 20 client.16644 trim_cache size 0 max 16384 2024-09-21T02:57:33.384 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:57:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f961c0b66b0 msgr2=0x7f961c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f961c0b66b0 0x7f961c0b8bc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f962c007fd0 tx=0x7f962c0076a0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 msgr2=0x7f963419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963419cca0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9628005940 tx=0x7f9628004270 comp rx=0 tx=0).stop 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 shutdown_connections 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f961c0b66b0 0x7f961c0b8bc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634110200 0x7f96341a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9634102fa0 0x7f963419cca0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 --2- 172.21.15.31:0/164947627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634102650 0x7f963419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 >> 172.21.15.31:0/164947627 conn(0x7f9634079740 msgr2=0x7f9634109da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 shutdown_connections 2024-09-21T02:57:33.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:33.378+0000 7f963aaae640 1 -- 172.21.15.31:0/164947627 wait complete. 2024-09-21T02:57:34.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:57:34.440 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3973388450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 msgr2=0x7f5b1810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3973388450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b1810f5a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009ca0 tx=0x7f5b1002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3973388450 shutdown_connections 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3973388450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b1810f5a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3973388450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b18108420 0x7f5b18108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3973388450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b18102ae0 0x7f5b18102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.930+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3973388450 >> 172.21.15.31:0/3973388450 conn(0x7f5b18079770 msgr2=0x7f5b18079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.931+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3973388450 shutdown_connections 2024-09-21T02:57:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.931+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3973388450 wait complete. 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.931+0000 7f5b1f2eb640 1 Processor -- start 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.931+0000 7f5b1f2eb640 1 -- start start 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1f2eb640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b18102ae0 0x7f5b180ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1f2eb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b18108420 0x7f5b180ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1f2eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b181000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1f2eb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18112a80 con 0x7f5b18108d70 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1f2eb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18112900 con 0x7f5b18108420 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1f2eb640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18112c00 con 0x7f5b18102ae0 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b181000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b181000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48530/0 (socket says 172.21.15.31:48530) 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 -- 172.21.15.31:0/3509404549 learned_addr learned my addr 172.21.15.31:0/3509404549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d087640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b18102ae0 0x7f5b180ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1c886640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b18108420 0x7f5b180ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 -- 172.21.15.31:0/3509404549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b18102ae0 msgr2=0x7f5b180ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b18102ae0 0x7f5b180ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 -- 172.21.15.31:0/3509404549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b18108420 msgr2=0x7f5b180ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b18108420 0x7f5b180ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 -- 172.21.15.31:0/3509404549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b181a8780 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.932+0000 7f5b1d888640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b181000f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b1002f540 tx=0x7f5b100316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.933+0000 7f5b0e7fc640 1 -- 172.21.15.31:0/3509404549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10003b70 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.933+0000 7f5b0e7fc640 1 -- 172.21.15.31:0/3509404549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b10003d10 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.933+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b181a8a10 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.933+0000 7f5b0e7fc640 1 -- 172.21.15.31:0/3509404549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1002f990 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.933+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b181a8f50 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.936+0000 7f5b2585f740 1 Processor -- start 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.936+0000 7f5b0e7fc640 1 -- 172.21.15.31:0/3509404549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5b1002fb30 con 0x7f5b18108d70 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.936+0000 7f5b2585f740 1 -- start start 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.936+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654f5070 0x55bc6589c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654cc1b0 0x55bc6589c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc6589cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b2585f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bc65812ea0 con 0x55bc654d4390 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b2585f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bc658022d0 con 0x55bc654cc1b0 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b2585f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc658060e0 con 0x55bc654f5070 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc6589cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc6589cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48542/0 (socket says 172.21.15.31:48542) 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 -- 172.21.15.31:0/1520903648 learned_addr learned my addr 172.21.15.31:0/1520903648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1d087640 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654cc1b0 0x55bc6589c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654f5070 0x55bc6589c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 -- 172.21.15.31:0/1520903648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654f5070 msgr2=0x55bc6589c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654f5070 0x55bc6589c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b0e7fc640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b040b6680 0x7f5b040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 -- 172.21.15.31:0/1520903648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654cc1b0 msgr2=0x55bc6589c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654cc1b0 0x55bc6589c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 -- 172.21.15.31:0/1520903648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc65560650 con 0x55bc654d4390 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1d888640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b040b6680 0x7f5b040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b1c886640 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc6589cd40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009f30 tx=0x7f5b08005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec399b5143251cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b0e7fc640 1 -- 172.21.15.31:0/3509404549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5b101088b0 con 0x7f5b18108d70 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.937+0000 7f5b037fe640 1 -- 172.21.15.31:0/1520903648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08004050 con 0x55bc654d4390 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.938+0000 7f5b037fe640 1 -- 172.21.15.31:0/1520903648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b080041f0 con 0x55bc654d4390 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.938+0000 7f5b037fe640 1 -- 172.21.15.31:0/1520903648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800f620 con 0x55bc654d4390 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.938+0000 7f5b2585f740 1 -- 172.21.15.31:0/1520903648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 msgr2=0x55bc6589cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.938+0000 7f5b1d888640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b040b6680 0x7f5b040b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5b100ce420 tx=0x7f5b100c2f80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.938+0000 7f5b2585f740 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc6589cd40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009f30 tx=0x7f5b08005850 comp rx=0 tx=0).stop 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 -- 172.21.15.31:0/1520903648 shutdown_connections 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc6589cd40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654cc1b0 0x55bc6589c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 --2- 172.21.15.31:0/1520903648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654f5070 0x55bc6589c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 -- 172.21.15.31:0/1520903648 >> 172.21.15.31:0/1520903648 conn(0x55bc654e9460 msgr2=0x55bc6589a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 -- 172.21.15.31:0/1520903648 shutdown_connections 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 -- 172.21.15.31:0/1520903648 wait complete. 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 -- start start 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.939+0000 7f5b2585f740 1 Processor -- start 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc658a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bc65663700 con 0x55bc654cc1b0 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bc6579d160 con 0x55bc654f5070 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc658aaaa0 con 0x55bc654d4390 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b2585f740 10 client.0 _ll_register_callbacks cb 0x55bc657b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc658a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc658a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48556/0 (socket says 172.21.15.31:48556) 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 -- 172.21.15.31:0/2521561349 learned_addr learned my addr 172.21.15.31:0/2521561349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1d087640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 -- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 msgr2=0x55bc658a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658a7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 -- 172.21.15.31:0/2521561349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 msgr2=0x55bc658a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658a82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 -- 172.21.15.31:0/2521561349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc657cac90 con 0x55bc654cc1b0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b1c886640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc658a7870 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005bc0 tx=0x7f5b080098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690945029affee35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.940+0000 7f5b00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b00ff9640 1 -- 172.21.15.31:0/2521561349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08022070 con 0x55bc654cc1b0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b00ff9640 1 -- 172.21.15.31:0/2521561349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5b08005040 con 0x55bc654cc1b0 2024-09-21T02:57:34.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b080041f0 con 0x55bc654cc1b0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b00ff9640 1 -- 172.21.15.31:0/2521561349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b08033050 con 0x55bc654cc1b0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 10 client.16656 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc658b6700 con 0x55bc654cc1b0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b00ff9640 1 -- 172.21.15.31:0/2521561349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b080041f0 con 0x55bc654cc1b0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b00ff9640 1 -- 172.21.15.31:0/2521561349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5b0802c060 con 0x55bc654cc1b0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 10 client.16656 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 10 client.16656 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 20 client.16656 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc654b39f0 0x55bc658b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 4 client.16656 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.941+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc65814df0 con 0x55bc654b39f0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.942+0000 7f5b1d087640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc654b39f0 0x55bc658b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.942+0000 7f5b1d087640 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc654b39f0 0x55bc658b9b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3382f3d2669fea7c server_cookie=ea6f1caf1fcf9a31 in_seq=0 out_seq=0 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.942+0000 7f5b00ff9640 10 client.16656 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:57:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b00ff9640 1 -- 172.21.15.31:0/2521561349 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc65814df0 con 0x55bc654b39f0 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b00ff9640 10 client.16656 handle_command_reply: tid=0 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 client.16656 shutdown 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 msgr2=0x55bc658a7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc658a7870 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005bc0 tx=0x7f5b080098a0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 shutdown_connections 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc654b39f0 0x55bc658b9b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658a82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b00ff9640 0 client.16656 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 --2- 172.21.15.31:0/2521561349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc658a7870 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 >> 172.21.15.31:0/2521561349 conn(0x55bc654e9460 msgr2=0x55bc6589f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 shutdown_connections 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 1 -- 172.21.15.31:0/2521561349 wait complete. 2024-09-21T02:57:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.943+0000 7f5b2585f740 20 client.16656 trim_cache size 0 max 16384 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.956+0000 7f5b2585f740 1 Processor -- start 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.956+0000 7f5b2585f740 1 -- start start 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc6589dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc65899760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b2585f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181b17c0 con 0x55bc654cc1b0 2024-09-21T02:57:34.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b2585f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181b3300 con 0x55bc654f5070 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b2585f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181b1e90 con 0x55bc654d4390 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc6589dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc6589dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48564/0 (socket says 172.21.15.31:48564) 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1c886640 1 -- 172.21.15.31:0/598872340 learned_addr learned my addr 172.21.15.31:0/598872340 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1c886640 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc65899760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 -- 172.21.15.31:0/598872340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 msgr2=0x55bc65899760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc65899760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 -- 172.21.15.31:0/598872340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 msgr2=0x55bc6589dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc6589dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 -- 172.21.15.31:0/598872340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6589e5b0 con 0x55bc654d4390 2024-09-21T02:57:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1c886640 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc6589dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:57:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.957+0000 7f5b1d087640 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658b33f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55bc6579d270 tx=0x7f5b14005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5664c83a8f927b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.958+0000 7f5b02ffd640 1 -- 172.21.15.31:0/598872340 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14020040 con 0x55bc654d4390 2024-09-21T02:57:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.958+0000 7f5b02ffd640 1 -- 172.21.15.31:0/598872340 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b140054d0 con 0x55bc654d4390 2024-09-21T02:57:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.958+0000 7f5b02ffd640 1 -- 172.21.15.31:0/598872340 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004da0 con 0x55bc654d4390 2024-09-21T02:57:34.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.958+0000 7f5b2585f740 1 -- 172.21.15.31:0/598872340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 msgr2=0x55bc658b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.958+0000 7f5b2585f740 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658b33f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55bc6579d270 tx=0x7f5b14005f90 comp rx=0 tx=0).stop 2024-09-21T02:57:34.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 -- 172.21.15.31:0/598872340 shutdown_connections 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654d4390 0x55bc658b33f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc65899760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 --2- 172.21.15.31:0/598872340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654cc1b0 0x55bc6589dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 -- 172.21.15.31:0/598872340 >> 172.21.15.31:0/598872340 conn(0x55bc654e9460 msgr2=0x55bc658b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 -- 172.21.15.31:0/598872340 shutdown_connections 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 -- 172.21.15.31:0/598872340 wait complete. 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 -- start start 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 Processor -- start 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654cc1b0 0x55bc658b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.959+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc658abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b2585f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b2585f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181b2c30 con 0x55bc654d4390 2024-09-21T02:57:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b2585f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181b2560 con 0x55bc654f5070 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b2585f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181af7a0 con 0x55bc654cc1b0 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b2585f740 10 client.0 _ll_register_callbacks cb 0x7f5b14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc658abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc658abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48572/0 (socket says 172.21.15.31:48572) 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1c886640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 -- 172.21.15.31:0/468983038 learned_addr learned my addr 172.21.15.31:0/468983038 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1c886640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654cc1b0 0x55bc658b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 -- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654cc1b0 msgr2=0x55bc658b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654cc1b0 0x55bc658b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 -- 172.21.15.31:0/468983038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 msgr2=0x55bc658ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 -- 172.21.15.31:0/468983038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc658ac890 con 0x55bc654d4390 2024-09-21T02:57:34.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1c886640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b1d087640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc658abad0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000c00 tx=0x7f5b140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0326885ba114903 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.960+0000 7f5b00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b00ff9640 1 -- 172.21.15.31:0/468983038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14020050 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b00ff9640 1 -- 172.21.15.31:0/468983038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5b14005220 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b140054d0 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b00ff9640 1 -- 172.21.15.31:0/468983038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b14035050 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 10 client.16662 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc65924410 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b00ff9640 1 -- 172.21.15.31:0/468983038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b140054d0 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b00ff9640 1 -- 172.21.15.31:0/468983038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5b140036a0 con 0x55bc654d4390 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 10 client.16662 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 10 client.16662 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 20 client.16662 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:34.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc65488da0 0x55bc65926b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 4 client.16662 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.961+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc658108a0 con 0x55bc65488da0 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b1c886640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc65488da0 0x55bc65926b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b1c886640 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc65488da0 0x55bc65926b70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5df7d3d0fb089466 server_cookie=5a3d7f678dfd831 in_seq=0 out_seq=0 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b00ff9640 10 client.16662 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b00ff9640 1 -- 172.21.15.31:0/468983038 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc658108a0 con 0x55bc65488da0 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b00ff9640 10 client.16662 handle_command_reply: tid=0 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b2585f740 1 client.16662 shutdown 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 msgr2=0x55bc658abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.962+0000 7f5b2585f740 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc658abad0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000c00 tx=0x7f5b140047b0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 shutdown_connections 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55bc65488da0 0x55bc65926b70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bc654f5070 0x55bc658ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bc654d4390 0x55bc658abad0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 --2- 172.21.15.31:0/468983038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc654cc1b0 0x55bc658b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 >> 172.21.15.31:0/468983038 conn(0x55bc654e9460 msgr2=0x55bc6589da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b00ff9640 0 client.16662 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 shutdown_connections 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 1 -- 172.21.15.31:0/468983038 wait complete. 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b2585f740 20 client.16662 trim_cache size 0 max 16384 2024-09-21T02:57:34.969 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b040b6680 msgr2=0x7f5b040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b040b6680 0x7f5b040b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5b100ce420 tx=0x7f5b100c2f80 comp rx=0 tx=0).stop 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 msgr2=0x7f5b181000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b181000f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b1002f540 tx=0x7f5b100316f0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 shutdown_connections 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5b040b6680 0x7f5b040b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b18108d70 0x7f5b181000f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b18108420 0x7f5b180ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 --2- 172.21.15.31:0/3509404549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b18102ae0 0x7f5b180ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:34.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.963+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 >> 172.21.15.31:0/3509404549 conn(0x7f5b18079770 msgr2=0x7f5b18101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:34.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.964+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 shutdown_connections 2024-09-21T02:57:34.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:34.964+0000 7f5b1f2eb640 1 -- 172.21.15.31:0/3509404549 wait complete. 2024-09-21T02:57:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:34 smithi165 ceph-mon[34818]: pgmap v465: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:57:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:34 smithi178 ceph-mon[34759]: pgmap v465: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:57:35.274 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:34 smithi031 ceph-mon[31209]: pgmap v465: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:57:35.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:57:36.090 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 -- 172.21.15.31:0/1221779246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 msgr2=0x7fe494104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1221779246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe494104040 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe484009ca0 tx=0x7fe48402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 -- 172.21.15.31:0/1221779246 shutdown_connections 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1221779246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe494109570 0x7fe494109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1221779246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe494109ec0 0x7fe4941106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1221779246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe494104040 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 -- 172.21.15.31:0/1221779246 >> 172.21.15.31:0/1221779246 conn(0x7fe49407b8d0 msgr2=0x7fe49407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 -- 172.21.15.31:0/1221779246 shutdown_connections 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.573+0000 7fe498fc4640 1 -- 172.21.15.31:0/1221779246 wait complete. 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 Processor -- start 2024-09-21T02:57:36.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 -- start start 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe49419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe494109570 0x7fe49419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe494109ec0 0x7fe49419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe494113ae0 con 0x7fe494103c30 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe494113960 con 0x7fe494109ec0 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe498fc4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe494113c60 con 0x7fe494109570 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe49259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe49419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe49259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe49419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48588/0 (socket says 172.21.15.31:48588) 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.574+0000 7fe49259c640 1 -- 172.21.15.31:0/1198765742 learned_addr learned my addr 172.21.15.31:0/1198765742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe494109ec0 0x7fe49419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe491d9b640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe494109570 0x7fe49419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe49259c640 1 -- 172.21.15.31:0/1198765742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe494109570 msgr2=0x7fe49419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe49259c640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe494109570 0x7fe49419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe49259c640 1 -- 172.21.15.31:0/1198765742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe494109ec0 msgr2=0x7fe49419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe49259c640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe494109ec0 0x7fe49419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe49259c640 1 -- 172.21.15.31:0/1198765742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4941a0300 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe49259c640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe49419a450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe48402ade0 tx=0x7fe484003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe4837fe640 1 -- 172.21.15.31:0/1198765742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484003d60 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe4837fe640 1 -- 172.21.15.31:0/1198765742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4840044f0 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4941a0590 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe4837fe640 1 -- 172.21.15.31:0/1198765742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48402fd10 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.575+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4941a6e10 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.578+0000 7fe4837fe640 1 -- 172.21.15.31:0/1198765742 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe4840046a0 con 0x7fe494103c30 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.578+0000 7fe49f538740 1 Processor -- start 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.578+0000 7fe49f538740 1 -- start start 2024-09-21T02:57:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49f538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49f538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b755899be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49f538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b75589a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49f538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b75580fea0 con 0x55b7554d1390 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49f538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b7557ff2d0 con 0x55b7554c91b0 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49f538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7558030e0 con 0x55b7554f2070 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b75589a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b75589a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48592/0 (socket says 172.21.15.31:48592) 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 -- 172.21.15.31:0/3276664954 learned_addr learned my addr 172.21.15.31:0/3276664954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe492d9d640 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe49259c640 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b755899be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 -- 172.21.15.31:0/3276664954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 msgr2=0x55b7558996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 -- 172.21.15.31:0/3276664954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 msgr2=0x55b755899be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b755899be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe4837fe640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe47c0b6730 0x7fe47c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 -- 172.21.15.31:0/3276664954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75555d650 con 0x55b7554d1390 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe47c0b6730 0x7fe47c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.579+0000 7fe491d9b640 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b75589a120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe48800ac30 tx=0x7fe48800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea391084e8216b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.580+0000 7fe4837fe640 1 -- 172.21.15.31:0/1198765742 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe484108b40 con 0x7fe494103c30 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.580+0000 7fe480ff9640 1 -- 172.21.15.31:0/3276664954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48801d070 con 0x55b7554d1390 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.580+0000 7fe480ff9640 1 -- 172.21.15.31:0/3276664954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe488018af0 con 0x55b7554d1390 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.580+0000 7fe480ff9640 1 -- 172.21.15.31:0/3276664954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488010ab0 con 0x55b7554d1390 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 -- 172.21.15.31:0/3276664954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 msgr2=0x55b75589a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe47c0b6730 0x7fe47c0b8c40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00ac90 tx=0x7fe48c00a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b75589a120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe48800ac30 tx=0x7fe48800f790 comp rx=0 tx=0).stop 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 -- 172.21.15.31:0/3276664954 shutdown_connections 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b75589a120 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b755899be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 --2- 172.21.15.31:0/3276664954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 -- 172.21.15.31:0/3276664954 >> 172.21.15.31:0/3276664954 conn(0x55b7554e6460 msgr2=0x55b755897590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 -- 172.21.15.31:0/3276664954 shutdown_connections 2024-09-21T02:57:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 -- 172.21.15.31:0/3276664954 wait complete. 2024-09-21T02:57:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 -- start start 2024-09-21T02:57:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.581+0000 7fe49f538740 1 Processor -- start 2024-09-21T02:57:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b7558a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b755660700 con 0x55b7554d1390 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b75579a160 con 0x55b7554c91b0 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7558ac1e0 con 0x55b7554f2070 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49f538740 10 client.0 _ll_register_callbacks cb 0x55b7557b39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b7558a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b7558a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48604/0 (socket says 172.21.15.31:48604) 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 -- 172.21.15.31:0/3968614011 learned_addr learned my addr 172.21.15.31:0/3968614011 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe49259c640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe492d9d640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 -- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 msgr2=0x55b7558b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 -- 172.21.15.31:0/3968614011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 msgr2=0x55b7558a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.582+0000 7fe491d9b640 1 -- 172.21.15.31:0/3968614011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7557c7c90 con 0x55b7554d1390 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe491d9b640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b7558a96c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe488010690 tx=0x7fe488018ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53292b949b34f253 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe479ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe479ffb640 1 -- 172.21.15.31:0/3968614011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488027020 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe479ffb640 1 -- 172.21.15.31:0/3968614011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe488005450 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe488018af0 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe479ffb640 1 -- 172.21.15.31:0/3968614011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe488032050 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 10 client.16674 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7558b7f10 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe479ffb640 1 -- 172.21.15.31:0/3968614011 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe488018af0 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe479ffb640 1 -- 172.21.15.31:0/3968614011 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe4880046c0 con 0x55b7554d1390 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 10 client.16674 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 10 client.16674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 20 client.16674 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b7554b09f0 0x55b7558bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 4 client.16674 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.583+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b755811df0 con 0x55b7554b09f0 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.584+0000 7fe49259c640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b7554b09f0 0x55b7558bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.584+0000 7fe49259c640 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b7554b09f0 0x55b7558bb320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7898d2d71012bb18 server_cookie=53da26f55f004013 in_seq=0 out_seq=0 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.584+0000 7fe479ffb640 10 client.16674 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe479ffb640 1 -- 172.21.15.31:0/3968614011 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b755811df0 con 0x55b7554b09f0 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe479ffb640 10 client.16674 handle_command_reply: tid=0 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 client.16674 shutdown 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 msgr2=0x55b7558a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b7558a96c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe488010690 tx=0x7fe488018ed0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 shutdown_connections 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b7554b09f0 0x55b7558bb320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554f2070 0x55b7558b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554d1390 0x55b7558a96c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 --2- 172.21.15.31:0/3968614011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 >> 172.21.15.31:0/3968614011 conn(0x55b7554e6460 msgr2=0x55b755895bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.585+0000 7fe479ffb640 0 client.16674 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:57:36.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.586+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 shutdown_connections 2024-09-21T02:57:36.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.586+0000 7fe49f538740 1 -- 172.21.15.31:0/3968614011 wait complete. 2024-09-21T02:57:36.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.586+0000 7fe49f538740 20 client.16674 trim_cache size 0 max 16384 2024-09-21T02:57:36.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 Processor -- start 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 -- start start 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554d1390 0x55b7558975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b7558b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ad380 con 0x55b7554f2070 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941aeec0 con 0x55b7554c91b0 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe49f538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ada50 con 0x55b7554d1390 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b7558b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b7558b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48618/0 (socket says 172.21.15.31:48618) 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe492d9d640 1 -- 172.21.15.31:0/1575308646 learned_addr learned my addr 172.21.15.31:0/1575308646 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.599+0000 7fe491d9b640 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554d1390 0x55b7558975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe49259c640 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe492d9d640 1 -- 172.21.15.31:0/1575308646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554d1390 msgr2=0x55b7558975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554d1390 0x55b7558975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe492d9d640 1 -- 172.21.15.31:0/1575308646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 msgr2=0x55b7558a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe492d9d640 1 -- 172.21.15.31:0/1575308646 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75589a490 con 0x55b7554f2070 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b7558b5480 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe48c01b510 tx=0x7fe48c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b92f3914b1f5180 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe47bfff640 1 -- 172.21.15.31:0/1575308646 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c01bbc0 con 0x55b7554f2070 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe47bfff640 1 -- 172.21.15.31:0/1575308646 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe48c02b070 con 0x55b7554f2070 2024-09-21T02:57:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.600+0000 7fe47bfff640 1 -- 172.21.15.31:0/1575308646 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00f680 con 0x55b7554f2070 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 -- 172.21.15.31:0/1575308646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 msgr2=0x55b7558b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b7558b5480 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe48c01b510 tx=0x7fe48c015000 comp rx=0 tx=0).stop 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 -- 172.21.15.31:0/1575308646 shutdown_connections 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554d1390 0x55b7558975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b7558b5480 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 --2- 172.21.15.31:0/1575308646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554c91b0 0x55b7558a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 -- 172.21.15.31:0/1575308646 >> 172.21.15.31:0/1575308646 conn(0x55b7554e6460 msgr2=0x55b755896eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 -- 172.21.15.31:0/1575308646 shutdown_connections 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 -- 172.21.15.31:0/1575308646 wait complete. 2024-09-21T02:57:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.601+0000 7fe49f538740 1 -- start start 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 Processor -- start 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554c91b0 0x55b7558bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554d1390 0x55b7558bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b755923a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ae7f0 con 0x55b7554f2070 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ae120 con 0x55b7554d1390 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ab360 con 0x55b7554c91b0 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49f538740 10 client.0 _ll_register_callbacks cb 0x7fe48801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b755923a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b755923a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48624/0 (socket says 172.21.15.31:48624) 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 -- 172.21.15.31:0/1087144062 learned_addr learned my addr 172.21.15.31:0/1087144062 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe491d9b640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554d1390 0x55b7558bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe49259c640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554c91b0 0x55b7558bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 -- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554c91b0 msgr2=0x55b7558bbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554c91b0 0x55b7558bbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 -- 172.21.15.31:0/1087144062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554d1390 msgr2=0x55b7558bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554d1390 0x55b7558bc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe492d9d640 1 -- 172.21.15.31:0/1087144062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b755924020 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.602+0000 7fe491d9b640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554d1390 0x55b7558bc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe492d9d640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b755923a90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe48c017040 tx=0x7fe48c00ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5694cd5d2c8db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe479ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe479ffb640 1 -- 172.21.15.31:0/1087144062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c035020 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe479ffb640 1 -- 172.21.15.31:0/1087144062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fe48c0263f0 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe48c02b070 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe479ffb640 1 -- 172.21.15.31:0/1087144062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe48c040050 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 10 client.16680 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7559252a0 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe479ffb640 1 -- 172.21.15.31:0/1087144062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe48c02b070 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe479ffb640 1 -- 172.21.15.31:0/1087144062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe48c031440 con 0x55b7554f2070 2024-09-21T02:57:36.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 10 client.16680 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 10 client.16680 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 20 client.16680 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.603+0000 7fe49f538740 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b755485da0 0x55b755927970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.604+0000 7fe49f538740 4 client.16680 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.604+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b75580d8a0 con 0x55b755485da0 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.604+0000 7fe49259c640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b755485da0 0x55b755927970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.604+0000 7fe49259c640 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b755485da0 0x55b755927970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bcf07d5dadc1a38 server_cookie=d061c21dc29adce9 in_seq=0 out_seq=0 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.604+0000 7fe479ffb640 10 client.16680 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe479ffb640 1 -- 172.21.15.31:0/1087144062 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b75580d8a0 con 0x55b755485da0 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe479ffb640 10 client.16680 handle_command_reply: tid=0 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 client.16680 shutdown 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 msgr2=0x55b755923a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b755923a90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe48c017040 tx=0x7fe48c00ae00 comp rx=0 tx=0).stop 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 shutdown_connections 2024-09-21T02:57:36.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55b755485da0 0x55b755927970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7554f2070 0x55b755923a90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b7554d1390 0x55b7558bc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe479ffb640 0 client.16680 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 --2- 172.21.15.31:0/1087144062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7554c91b0 0x55b7558bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 >> 172.21.15.31:0/1087144062 conn(0x55b7554e6460 msgr2=0x55b75589e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 shutdown_connections 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 1 -- 172.21.15.31:0/1087144062 wait complete. 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe49f538740 20 client.16680 trim_cache size 0 max 16384 2024-09-21T02:57:36.611 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe47c0b6730 msgr2=0x7fe47c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe47c0b6730 0x7fe47c0b8c40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00ac90 tx=0x7fe48c00a250 comp rx=0 tx=0).stop 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 msgr2=0x7fe49419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.605+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe49419a450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe48402ade0 tx=0x7fe484003970 comp rx=0 tx=0).stop 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 shutdown_connections 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe47c0b6730 0x7fe47c0b8c40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe494109570 0x7fe49419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe494109ec0 0x7fe49419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 --2- 172.21.15.31:0/1198765742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe494103c30 0x7fe49419a450 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 >> 172.21.15.31:0/1198765742 conn(0x7fe49407b8d0 msgr2=0x7fe4941075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 shutdown_connections 2024-09-21T02:57:36.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:36.606+0000 7fe498fc4640 1 -- 172.21.15.31:0/1198765742 wait complete. 2024-09-21T02:57:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:36 smithi165 ceph-mon[34818]: pgmap v466: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:37.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:36 smithi178 ceph-mon[34759]: pgmap v466: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:37.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:36 smithi031 ceph-mon[31209]: pgmap v466: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:37.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:57:37.705 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/3007522920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04108d10 msgr2=0x7f1e0410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/3007522920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04108d10 0x7f1e0410f540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009ca0 tx=0x7f1dfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/3007522920 shutdown_connections 2024-09-21T02:57:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/3007522920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e04108d10 0x7f1e0410f540 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/3007522920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e041083c0 0x7f1e041087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/3007522920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e04102a80 0x7f1e04102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/3007522920 >> 172.21.15.31:0/3007522920 conn(0x7f1e04079740 msgr2=0x7f1e04079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.202+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/3007522920 shutdown_connections 2024-09-21T02:57:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.203+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/3007522920 wait complete. 2024-09-21T02:57:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.203+0000 7f1e0b4f9640 1 Processor -- start 2024-09-21T02:57:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.203+0000 7f1e0b4f9640 1 -- start start 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e0b4f9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e04102a80 0x7f1e0419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e0b4f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 0x7f1e0419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e0b4f9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e04108d10 0x7f1e0419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e0b4f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112980 con 0x7f1e041083c0 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e0b4f9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112800 con 0x7f1e04102a80 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e0b4f9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112b00 con 0x7f1e04108d10 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 0x7f1e0419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 0x7f1e0419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48646/0 (socket says 172.21.15.31:48646) 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 -- 172.21.15.31:0/425971311 learned_addr learned my addr 172.21.15.31:0/425971311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e09295640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e04102a80 0x7f1e0419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e09a96640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e04108d10 0x7f1e0419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 -- 172.21.15.31:0/425971311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e04108d10 msgr2=0x7f1e0419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e04108d10 0x7f1e0419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 -- 172.21.15.31:0/425971311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e04102a80 msgr2=0x7f1e0419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e04102a80 0x7f1e0419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 -- 172.21.15.31:0/425971311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e041a65a0 con 0x7f1e041083c0 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.204+0000 7f1e08a94640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 0x7f1e0419b7d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1df400d770 tx=0x7f1df400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.205+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/425971311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4017070 con 0x7f1e041083c0 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.205+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e041a6890 con 0x7f1e041083c0 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.205+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/425971311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df4009c30 con 0x7f1e041083c0 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.205+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/425971311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400f620 con 0x7f1e041083c0 2024-09-21T02:57:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.205+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e041a6dd0 con 0x7f1e041083c0 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.207+0000 7f1e11a6d740 1 Processor -- start 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 -- start start 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/425971311 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1df401a070 con 0x7f1e041083c0 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87e0b070 0x55ed881b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 0x55ed881b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed881b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed88128ea0 con 0x55ed87de21b0 2024-09-21T02:57:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed881182d0 con 0x55ed87e0b070 2024-09-21T02:57:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed8811c0e0 con 0x55ed87dea390 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 0x55ed881b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 0x55ed881b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48656/0 (socket says 172.21.15.31:48656) 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 -- 172.21.15.31:0/1675792493 learned_addr learned my addr 172.21.15.31:0/1675792493 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e08a94640 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed881b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09a96640 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87e0b070 0x55ed881b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 -- 172.21.15.31:0/1675792493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 msgr2=0x55ed881b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed881b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 -- 172.21.15.31:0/1675792493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87e0b070 msgr2=0x55ed881b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87e0b070 0x55ed881b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.208+0000 7f1e09295640 1 -- 172.21.15.31:0/1675792493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed87e76650 con 0x55ed87de21b0 2024-09-21T02:57:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1e09a96640 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87e0b070 0x55ed881b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1dfa7fc640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1df00b6680 0x7f1df00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1e09295640 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 0x55ed881b2800 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009ca0 tx=0x7f1e00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6665743b03cf7b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1e09a96640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1df00b6680 0x7f1df00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/425971311 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1df40e71d0 con 0x7f1e041083c0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1def7fe640 1 -- 172.21.15.31:0/1675792493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00005220 con 0x55ed87de21b0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1def7fe640 1 -- 172.21.15.31:0/1675792493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00004c50 con 0x55ed87de21b0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.209+0000 7f1def7fe640 1 -- 172.21.15.31:0/1675792493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000196d0 con 0x55ed87de21b0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 -- 172.21.15.31:0/1675792493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 msgr2=0x55ed881b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e09a96640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1df00b6680 0x7f1df00b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0094e0 tx=0x7f1dfc0093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 0x55ed881b2800 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009ca0 tx=0x7f1e00011810 comp rx=0 tx=0).stop 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 -- 172.21.15.31:0/1675792493 shutdown_connections 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed881b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87de21b0 0x55ed881b2800 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/1675792493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87e0b070 0x55ed881b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 -- 172.21.15.31:0/1675792493 >> 172.21.15.31:0/1675792493 conn(0x55ed87dff460 msgr2=0x55ed881b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 -- 172.21.15.31:0/1675792493 shutdown_connections 2024-09-21T02:57:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 -- 172.21.15.31:0/1675792493 wait complete. 2024-09-21T02:57:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.210+0000 7f1e11a6d740 1 -- start start 2024-09-21T02:57:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 Processor -- start 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881bd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dea390 0x55ed881bde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 0x55ed881be360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed87f79700 con 0x55ed87e0b070 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed880b3160 con 0x55ed87dea390 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed881c0aa0 con 0x55ed87de21b0 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e11a6d740 10 client.0 _ll_register_callbacks cb 0x55ed880cc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e09a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 0x55ed881be360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e09a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 0x55ed881be360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48660/0 (socket says 172.21.15.31:48660) 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e09a96640 1 -- 172.21.15.31:0/2811503670 learned_addr learned my addr 172.21.15.31:0/2811503670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.211+0000 7f1e09295640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881bd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e08a94640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dea390 0x55ed881bde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e09a96640 1 -- 172.21.15.31:0/2811503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 msgr2=0x55ed881bd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e09a96640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881bd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e09a96640 1 -- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dea390 msgr2=0x55ed881bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e09a96640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dea390 0x55ed881bde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e09a96640 1 -- 172.21.15.31:0/2811503670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed880e0c90 con 0x55ed87e0b070 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e08a94640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dea390 0x55ed881bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e09a96640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 0x55ed881be360 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0316d0 tx=0x7f1dfc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd419093c6685 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1decff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1decff9640 1 -- 172.21.15.31:0/2811503670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc04f070 con 0x55ed87e0b070 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1decff9640 1 -- 172.21.15.31:0/2811503670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1dfc004240 con 0x55ed87e0b070 2024-09-21T02:57:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e00004c50 con 0x55ed87e0b070 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1decff9640 1 -- 172.21.15.31:0/2811503670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1dfc004440 con 0x55ed87e0b070 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e11a6d740 10 client.16692 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.212+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed881cc6b0 con 0x55ed87e0b070 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1decff9640 1 -- 172.21.15.31:0/2811503670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00004c50 con 0x55ed87e0b070 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1decff9640 1 -- 172.21.15.31:0/2811503670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1dfc044040 con 0x55ed87e0b070 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e11a6d740 10 client.16692 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e11a6d740 10 client.16692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e11a6d740 20 client.16692 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87dc99f0 0x55ed881cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e11a6d740 4 client.16692 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8812adf0 con 0x55ed87dc99f0 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.213+0000 7f1e09295640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87dc99f0 0x55ed881cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.214+0000 7f1decff9640 10 client.16692 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.214+0000 7f1e09295640 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87dc99f0 0x55ed881cfbc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e1bdc3c95c2582d server_cookie=290a046b7d28f6f8 in_seq=0 out_seq=0 2024-09-21T02:57:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.214+0000 7f1decff9640 1 -- 172.21.15.31:0/2811503670 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed8812adf0 con 0x55ed87dc99f0 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.214+0000 7f1decff9640 10 client.16692 handle_command_reply: tid=0 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.214+0000 7f1e11a6d740 1 client.16692 shutdown 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 msgr2=0x55ed881be360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 0x55ed881be360 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0316d0 tx=0x7f1dfc0095c0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 shutdown_connections 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87dc99f0 0x55ed881cfbc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87e0b070 0x55ed881be360 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dea390 0x55ed881bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2811503670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1decff9640 0 client.16692 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 >> 172.21.15.31:0/2811503670 conn(0x55ed87dff460 msgr2=0x55ed881b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 shutdown_connections 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2811503670 wait complete. 2024-09-21T02:57:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.215+0000 7f1e11a6d740 20 client.16692 trim_cache size 0 max 16384 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 Processor -- start 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 -- start start 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dc99f0 0x55ed881b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 0x55ed881bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041af7d0 con 0x55ed87dea390 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b1310 con 0x55ed87dc99f0 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.228+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041afea0 con 0x55ed87de21b0 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 0x55ed881bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 0x55ed881bb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48668/0 (socket says 172.21.15.31:48668) 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 -- 172.21.15.31:0/811804217 learned_addr learned my addr 172.21.15.31:0/811804217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09295640 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dc99f0 0x55ed881b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e08a94640 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 -- 172.21.15.31:0/811804217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 msgr2=0x55ed881baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881baad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 -- 172.21.15.31:0/811804217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dc99f0 msgr2=0x55ed881b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dc99f0 0x55ed881b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 -- 172.21.15.31:0/811804217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed881aa440 con 0x55ed87dea390 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e08a94640 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1e09a96640 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 0x55ed881bb010 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0096c0 tx=0x7f1dfc031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa59ef8363935972 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1deeffd640 1 -- 172.21.15.31:0/811804217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc04f070 con 0x55ed87dea390 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1deeffd640 1 -- 172.21.15.31:0/811804217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dfc036040 con 0x55ed87dea390 2024-09-21T02:57:38.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.229+0000 7f1deeffd640 1 -- 172.21.15.31:0/811804217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc02fa80 con 0x55ed87dea390 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 -- 172.21.15.31:0/811804217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 msgr2=0x55ed881bb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 0x55ed881bb010 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0096c0 tx=0x7f1dfc031f90 comp rx=0 tx=0).stop 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 -- 172.21.15.31:0/811804217 shutdown_connections 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87de21b0 0x55ed881baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dea390 0x55ed881bb010 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/811804217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87dc99f0 0x55ed881b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 -- 172.21.15.31:0/811804217 >> 172.21.15.31:0/811804217 conn(0x55ed87dff460 msgr2=0x55ed881b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 -- 172.21.15.31:0/811804217 shutdown_connections 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.230+0000 7f1e11a6d740 1 -- 172.21.15.31:0/811804217 wait complete. 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 -- start start 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 Processor -- start 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 0x55ed881c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87de21b0 0x55ed881c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed88239580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b0c40 con 0x55ed87dc99f0 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b0570 con 0x55ed87de21b0 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041ad7b0 con 0x55ed87dea390 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e11a6d740 10 client.0 _ll_register_callbacks cb 0x7f1e00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e09295640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 0x55ed881c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e09295640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 0x55ed881c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48672/0 (socket says 172.21.15.31:48672) 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e09295640 1 -- 172.21.15.31:0/2555433280 learned_addr learned my addr 172.21.15.31:0/2555433280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.231+0000 7f1e09a96640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed88239580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e08a94640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87de21b0 0x55ed881c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e09295640 1 -- 172.21.15.31:0/2555433280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 msgr2=0x55ed88239580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e09295640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed88239580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e09295640 1 -- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87de21b0 msgr2=0x55ed881c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e09295640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87de21b0 0x55ed881c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e09295640 1 -- 172.21.15.31:0/2555433280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed881c2cc0 con 0x55ed87dc99f0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e08a94640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87de21b0 0x55ed881c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e09295640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 0x55ed881c1f60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009a30 tx=0x7f1e00009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f447c3b15723ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1decff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1decff9640 1 -- 172.21.15.31:0/2555433280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00004450 con 0x55ed87dc99f0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1decff9640 1 -- 172.21.15.31:0/2555433280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f1e000035f0 con 0x55ed87dc99f0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dfc036040 con 0x55ed87dc99f0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1decff9640 1 -- 172.21.15.31:0/2555433280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e000045f0 con 0x55ed87dc99f0 2024-09-21T02:57:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e11a6d740 10 client.16698 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.232+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed8823aa90 con 0x55ed87dc99f0 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1decff9640 1 -- 172.21.15.31:0/2555433280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dfc036040 con 0x55ed87dc99f0 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1decff9640 1 -- 172.21.15.31:0/2555433280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1e0003c020 con 0x55ed87dc99f0 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e11a6d740 10 client.16698 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e11a6d740 10 client.16698 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e11a6d740 20 client.16698 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87e0b070 0x55ed8823d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e11a6d740 4 client.16698 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed881268a0 con 0x55ed87e0b070 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e08a94640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87e0b070 0x55ed8823d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1e08a94640 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87e0b070 0x55ed8823d060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee6ba335ffa5964e server_cookie=fd088d0004a397af in_seq=0 out_seq=0 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.233+0000 7f1decff9640 10 client.16698 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1decff9640 1 -- 172.21.15.31:0/2555433280 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed881268a0 con 0x55ed87e0b070 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1decff9640 10 client.16698 handle_command_reply: tid=0 2024-09-21T02:57:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 client.16698 shutdown 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 msgr2=0x55ed881c1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 0x55ed881c1f60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009a30 tx=0x7f1e00009ba0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 shutdown_connections 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ed87e0b070 0x55ed8823d060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed87dea390 0x55ed88239580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1decff9640 0 client.16698 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed87de21b0 0x55ed881c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 --2- 172.21.15.31:0/2555433280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed87dc99f0 0x55ed881c1f60 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 >> 172.21.15.31:0/2555433280 conn(0x55ed87dff460 msgr2=0x55ed881b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 shutdown_connections 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 1 -- 172.21.15.31:0/2555433280 wait complete. 2024-09-21T02:57:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.234+0000 7f1e11a6d740 20 client.16698 trim_cache size 0 max 16384 2024-09-21T02:57:38.241 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:57:38.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1df00b6680 msgr2=0x7f1df00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1df00b6680 0x7f1df00b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0094e0 tx=0x7f1dfc0093f0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 msgr2=0x7f1e0419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:38.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 0x7f1e0419b7d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1df400d770 tx=0x7f1df400dc30 comp rx=0 tx=0).stop 2024-09-21T02:57:38.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 shutdown_connections 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1df00b6680 0x7f1df00b8b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e04108d10 0x7f1e0419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e041083c0 0x7f1e0419b7d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 --2- 172.21.15.31:0/425971311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e04102a80 0x7f1e0419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 >> 172.21.15.31:0/425971311 conn(0x7f1e04079740 msgr2=0x7f1e041043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 shutdown_connections 2024-09-21T02:57:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:38.235+0000 7f1e0b4f9640 1 -- 172.21.15.31:0/425971311 wait complete. 2024-09-21T02:57:38.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:57:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:38 smithi165 ceph-mon[34818]: pgmap v467: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:57:39.218 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:39.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:38 smithi178 ceph-mon[34759]: pgmap v467: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:57:39.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:38 smithi031 ceph-mon[31209]: pgmap v467: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 -- 172.21.15.31:0/3870924883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 msgr2=0x7fd1ac103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 --2- 172.21.15.31:0/3870924883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac103f90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009900 tx=0x7fd1a802a410 comp rx=0 tx=0).stop 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 -- 172.21.15.31:0/3870924883 shutdown_connections 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 --2- 172.21.15.31:0/3870924883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1ac1094c0 0x7fd1ac1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 --2- 172.21.15.31:0/3870924883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1ac109e10 0x7fd1ac110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 --2- 172.21.15.31:0/3870924883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac103f90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 -- 172.21.15.31:0/3870924883 >> 172.21.15.31:0/3870924883 conn(0x7fd1ac0ff610 msgr2=0x7fd1ac101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 -- 172.21.15.31:0/3870924883 shutdown_connections 2024-09-21T02:57:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.756+0000 7fd1b3684640 1 -- 172.21.15.31:0/3870924883 wait complete. 2024-09-21T02:57:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 Processor -- start 2024-09-21T02:57:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 -- start start 2024-09-21T02:57:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1ac1094c0 0x7fd1ac19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1ac109e10 0x7fd1ac19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac113ad0 con 0x7fd1ac103b80 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac113950 con 0x7fd1ac109e10 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.757+0000 7fd1b3684640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac113c50 con 0x7fd1ac1094c0 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50374/0 (socket says 172.21.15.31:50374) 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 -- 172.21.15.31:0/2411085211 learned_addr learned my addr 172.21.15.31:0/2411085211 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1ac1094c0 0x7fd1ac19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1ac109e10 0x7fd1ac19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 -- 172.21.15.31:0/2411085211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1ac1094c0 msgr2=0x7fd1ac19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1ac1094c0 0x7fd1ac19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 -- 172.21.15.31:0/2411085211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1ac109e10 msgr2=0x7fd1ac19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1ac109e10 0x7fd1ac19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 -- 172.21.15.31:0/2411085211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac1a0370 con 0x7fd1ac103b80 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b1420640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac19a4c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005940 tx=0x7fd1a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1a27fc640 1 -- 172.21.15.31:0/2411085211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a803f030 con 0x7fd1ac103b80 2024-09-21T02:57:39.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1a27fc640 1 -- 172.21.15.31:0/2411085211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a8003cb0 con 0x7fd1ac103b80 2024-09-21T02:57:39.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac1a0600 con 0x7fd1ac103b80 2024-09-21T02:57:39.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.758+0000 7fd1a27fc640 1 -- 172.21.15.31:0/2411085211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8035070 con 0x7fd1ac103b80 2024-09-21T02:57:39.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.759+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac1a0b30 con 0x7fd1ac103b80 2024-09-21T02:57:39.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.761+0000 7fd1b9bf8740 1 Processor -- start 2024-09-21T02:57:39.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.761+0000 7fd1b9bf8740 1 -- start start 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1a27fc640 1 -- 172.21.15.31:0/2411085211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd1a8042030 con 0x7fd1ac103b80 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 0x564541c3a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c3abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541872390 0x564541c3b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564541bb0ea0 con 0x56454186a1b0 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564541ba02d0 con 0x564541893070 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564541ba40e0 con 0x564541872390 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b1420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c3abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b1c21640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 0x564541c3a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b1c21640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 0x564541c3a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:34142/0 (socket says 172.21.15.31:34142) 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b1420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c3abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50390/0 (socket says 172.21.15.31:50390) 2024-09-21T02:57:39.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b1c21640 1 -- 172.21.15.31:0/803230655 learned_addr learned my addr 172.21.15.31:0/803230655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.762+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541872390 0x564541c3b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1420640 1 -- 172.21.15.31:0/803230655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541872390 msgr2=0x564541c3b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1420640 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541872390 0x564541c3b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1420640 1 -- 172.21.15.31:0/803230655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 msgr2=0x564541c3a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1420640 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 0x564541c3a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1420640 1 -- 172.21.15.31:0/803230655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645418fe650 con 0x56454186a1b0 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 0x564541c3a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1a27fc640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd1980b6730 0x7fd1980b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1420640 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c3abe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8105310 tx=0x7fd1a810bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feca70b73e3e048 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd1980b6730 0x7fd1980b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1a27fc640 1 -- 172.21.15.31:0/2411085211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd1a8107bd0 con 0x7fd1ac103b80 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1977fe640 1 -- 172.21.15.31:0/803230655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80ca090 con 0x56454186a1b0 2024-09-21T02:57:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1977fe640 1 -- 172.21.15.31:0/803230655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a80c9190 con 0x56454186a1b0 2024-09-21T02:57:39.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.763+0000 7fd1977fe640 1 -- 172.21.15.31:0/803230655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80cd1b0 con 0x56454186a1b0 2024-09-21T02:57:39.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/803230655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 msgr2=0x564541c3abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c3abe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8105310 tx=0x7fd1a810bea0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd1980b6730 0x7fd1980b8c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4007fd0 tx=0x7fd1a40076c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/803230655 shutdown_connections 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541872390 0x564541c3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c3abe0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/803230655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541893070 0x564541c3a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/803230655 >> 172.21.15.31:0/803230655 conn(0x564541887460 msgr2=0x564541c38590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/803230655 shutdown_connections 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.764+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/803230655 wait complete. 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 -- start start 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 Processor -- start 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56454186a1b0 0x564541c4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 0x564541c561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564541a01700 con 0x564541893070 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564541b3b160 con 0x564541872390 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564541c4cf70 con 0x56454186a1b0 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b9bf8740 10 client.0 _ll_register_callbacks cb 0x564541b549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b1c21640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 0x564541c561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.765+0000 7fd1b1c21640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 0x564541c561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50402/0 (socket says 172.21.15.31:50402) 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 -- 172.21.15.31:0/622186295 learned_addr learned my addr 172.21.15.31:0/622186295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1420640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56454186a1b0 0x564541c4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 -- 172.21.15.31:0/622186295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56454186a1b0 msgr2=0x564541c4a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56454186a1b0 0x564541c4a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 -- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 msgr2=0x564541c4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 -- 172.21.15.31:0/622186295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564541b68c90 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 0x564541c561e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400fca0 tx=0x7fd1a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9564dabccb18566b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd194ff9640 1 -- 172.21.15.31:0/622186295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4003b70 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd194ff9640 1 -- 172.21.15.31:0/622186295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd1a4003d10 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564541bb2df0 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd194ff9640 1 -- 172.21.15.31:0/622186295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1a4004050 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b9bf8740 10 client.16710 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.766+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564541c58c00 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd194ff9640 1 -- 172.21.15.31:0/622186295 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564541bb2df0 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd194ff9640 1 -- 172.21.15.31:0/622186295 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd1a401e6c0 con 0x564541893070 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b9bf8740 10 client.16710 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:39.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b9bf8740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b9bf8740 20 client.16710 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5645418519f0 0x564541c5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b9bf8740 4 client.16710 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564541bae8a0 con 0x5645418519f0 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.767+0000 7fd1b1420640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5645418519f0 0x564541c5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.768+0000 7fd1b1420640 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5645418519f0 0x564541c5c070 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ec7c49e1f92b01 server_cookie=813781b0416adcf6 in_seq=0 out_seq=0 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.768+0000 7fd194ff9640 10 client.16710 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.768+0000 7fd194ff9640 1 -- 172.21.15.31:0/622186295 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564541bae8a0 con 0x5645418519f0 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.768+0000 7fd194ff9640 10 client.16710 handle_command_reply: tid=0 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.768+0000 7fd1b9bf8740 1 client.16710 shutdown 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 msgr2=0x564541c561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 0x564541c561e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400fca0 tx=0x7fd1a4004a40 comp rx=0 tx=0).stop 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 shutdown_connections 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5645418519f0 0x564541c5c070 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541893070 0x564541c561e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/622186295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56454186a1b0 0x564541c4a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 >> 172.21.15.31:0/622186295 conn(0x564541887460 msgr2=0x564541c3d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd194ff9640 0 client.16710 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 shutdown_connections 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/622186295 wait complete. 2024-09-21T02:57:39.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.769+0000 7fd1b9bf8740 20 client.16710 trim_cache size 0 max 16384 2024-09-21T02:57:39.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 Processor -- start 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 -- start start 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56454186a1b0 0x564541c385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 0x564541c56160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541c3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ad560 con 0x564541872390 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1af0a0 con 0x56454186a1b0 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.782+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1adc30 con 0x564541893070 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 0x564541c56160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 0x564541c56160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50416/0 (socket says 172.21.15.31:50416) 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 -- 172.21.15.31:0/3408379642 learned_addr learned my addr 172.21.15.31:0/3408379642 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541c3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b1420640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56454186a1b0 0x564541c385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 -- 172.21.15.31:0/3408379642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 msgr2=0x564541c3d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541c3d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 -- 172.21.15.31:0/3408379642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56454186a1b0 msgr2=0x564541c385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56454186a1b0 0x564541c385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 -- 172.21.15.31:0/3408379642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564541c48610 con 0x564541872390 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541c3d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b1420640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56454186a1b0 0x564541c385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 0x564541c56160 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00bdc0 tx=0x7fd19c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f06f1ab85d500c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd196ffd640 1 -- 172.21.15.31:0/3408379642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c01b5f0 con 0x564541872390 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd196ffd640 1 -- 172.21.15.31:0/3408379642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19c020070 con 0x564541872390 2024-09-21T02:57:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.783+0000 7fd196ffd640 1 -- 172.21.15.31:0/3408379642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c010a50 con 0x564541872390 2024-09-21T02:57:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3408379642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 msgr2=0x564541c56160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 0x564541c56160 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00bdc0 tx=0x7fd19c011810 comp rx=0 tx=0).stop 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3408379642 shutdown_connections 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541c3d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564541872390 0x564541c56160 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3408379642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56454186a1b0 0x564541c385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3408379642 >> 172.21.15.31:0/3408379642 conn(0x564541887460 msgr2=0x564541c37ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3408379642 shutdown_connections 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.784+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3408379642 wait complete. 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 -- start start 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 Processor -- start 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c4e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541cc5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ae9d0 con 0x56454186a1b0 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ae300 con 0x564541872390 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac1ab540 con 0x564541893070 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b9bf8740 10 client.0 _ll_register_callbacks cb 0x7fd1a4007d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b1420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c4e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b1420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c4e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50418/0 (socket says 172.21.15.31:50418) 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b1420640 1 -- 172.21.15.31:0/3197899807 learned_addr learned my addr 172.21.15.31:0/3197899807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.785+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1c21640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541cc5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1420640 1 -- 172.21.15.31:0/3197899807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 msgr2=0x564541cc5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1420640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541cc5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1420640 1 -- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 msgr2=0x564541c4eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1420640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1420640 1 -- 172.21.15.31:0/3197899807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564541cc5f60 con 0x56454186a1b0 2024-09-21T02:57:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b1420640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c4e5d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8105940 tx=0x7fd1a803b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cacef20c85b619b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd194ff9640 1 -- 172.21.15.31:0/3197899807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8049040 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd194ff9640 1 -- 172.21.15.31:0/3197899807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd1a8005d50 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd19c020070 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd194ff9640 1 -- 172.21.15.31:0/3197899807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1a80cd390 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b9bf8740 10 client.16716 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.786+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564541cc7050 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd194ff9640 1 -- 172.21.15.31:0/3197899807 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19c020070 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd194ff9640 1 -- 172.21.15.31:0/3197899807 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd1a81055c0 con 0x56454186a1b0 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b9bf8740 10 client.16716 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b9bf8740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b9bf8740 20 client.16716 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564541826da0 0x564541cc96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b9bf8740 4 client.16716 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:57:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564541babcd0 con 0x564541826da0 2024-09-21T02:57:39.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.787+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564541826da0 0x564541cc96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b0c1f640 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564541826da0 0x564541cc96d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d180f1418df724 server_cookie=fc2518405e25d8eb in_seq=0 out_seq=0 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd194ff9640 10 client.16716 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd194ff9640 1 -- 172.21.15.31:0/3197899807 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564541babcd0 con 0x564541826da0 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd194ff9640 10 client.16716 handle_command_reply: tid=0 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 client.16716 shutdown 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 msgr2=0x564541c4e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c4e5d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8105940 tx=0x7fd1a803b6b0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 shutdown_connections 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564541826da0 0x564541cc96d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564541893070 0x564541cc5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564541872390 0x564541c4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 --2- 172.21.15.31:0/3197899807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56454186a1b0 0x564541c4e5d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 >> 172.21.15.31:0/3197899807 conn(0x564541887460 msgr2=0x564541c3d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd194ff9640 0 client.16716 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 shutdown_connections 2024-09-21T02:57:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 1 -- 172.21.15.31:0/3197899807 wait complete. 2024-09-21T02:57:39.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.788+0000 7fd1b9bf8740 20 client.16716 trim_cache size 0 max 16384 2024-09-21T02:57:39.794 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd1980b6730 msgr2=0x7fd1980b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd1980b6730 0x7fd1980b8c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4007fd0 tx=0x7fd1a40076c0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 msgr2=0x7fd1ac19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac19a4c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005940 tx=0x7fd1a80042f0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 shutdown_connections 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd1980b6730 0x7fd1980b8c40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1ac1094c0 0x7fd1ac19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1ac109e10 0x7fd1ac19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 --2- 172.21.15.31:0/2411085211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ac103b80 0x7fd1ac19a4c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 >> 172.21.15.31:0/2411085211 conn(0x7fd1ac0ff610 msgr2=0x7fd1ac101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 shutdown_connections 2024-09-21T02:57:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:39.789+0000 7fd1b3684640 1 -- 172.21.15.31:0/2411085211 wait complete. 2024-09-21T02:57:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:40 smithi165 ceph-mon[34818]: pgmap v468: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:40 smithi178 ceph-mon[34759]: pgmap v468: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:41.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:40 smithi031 ceph-mon[31209]: pgmap v468: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:41.673 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:57:41.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:57:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 -- 172.21.15.31:0/2752663930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b901092d0 msgr2=0x7f3b901096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2752663930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b901092d0 0x7f3b901096e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b8002a410 comp rx=0 tx=0).stop 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 -- 172.21.15.31:0/2752663930 shutdown_connections 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2752663930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b90109c20 0x7f3b90110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2752663930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b901092d0 0x7f3b901096e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2752663930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b90103990 0x7f3b90103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 -- 172.21.15.31:0/2752663930 >> 172.21.15.31:0/2752663930 conn(0x7f3b900ff3e0 msgr2=0x7f3b90101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 -- 172.21.15.31:0/2752663930 shutdown_connections 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.427+0000 7f3b9679f640 1 -- 172.21.15.31:0/2752663930 wait complete. 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 Processor -- start 2024-09-21T02:57:42.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 -- start start 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b90103990 0x7f3b9019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b901092d0 0x7f3b9019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 0x7f3b9019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901138e0 con 0x7f3b90109c20 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90113760 con 0x7f3b901092d0 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.428+0000 7f3b9679f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90113a60 con 0x7f3b90103990 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 0x7f3b9019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 0x7f3b9019fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50446/0 (socket says 172.21.15.31:50446) 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 -- 172.21.15.31:0/2552637497 learned_addr learned my addr 172.21.15.31:0/2552637497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b901092d0 0x7f3b9019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b9579d640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b90103990 0x7f3b9019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 -- 172.21.15.31:0/2552637497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b90103990 msgr2=0x7f3b9019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b90103990 0x7f3b9019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 -- 172.21.15.31:0/2552637497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b901092d0 msgr2=0x7f3b9019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b901092d0 0x7f3b9019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 -- 172.21.15.31:0/2552637497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b901a0260 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 0x7f3b9019fb30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800eaa0 tx=0x7f3b8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b867fc640 1 -- 172.21.15.31:0/2552637497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800cdb0 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b867fc640 1 -- 172.21.15.31:0/2552637497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b88005020 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b901a0550 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b867fc640 1 -- 172.21.15.31:0/2552637497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880116d0 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.429+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b901a6cb0 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.432+0000 7f3b9df75740 1 Processor -- start 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.432+0000 7f3b867fc640 1 -- 172.21.15.31:0/2552637497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3b880051d0 con 0x7f3b90109c20 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 -- start start 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d07b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d07bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d07c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55786cff1e60 con 0x55786ccb3670 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55786cfe1290 con 0x55786ccab490 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9df75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55786cfe50a0 con 0x55786ccd4350 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d07c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d07c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50450/0 (socket says 172.21.15.31:50450) 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 -- 172.21.15.31:0/262753340 learned_addr learned my addr 172.21.15.31:0/262753340 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b9579d640 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d07bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 -- 172.21.15.31:0/262753340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 msgr2=0x55786d07b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d07b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 -- 172.21.15.31:0/262753340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 msgr2=0x55786d07bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d07bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.433+0000 7f3b94f9c640 1 -- 172.21.15.31:0/262753340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786cd3f930 con 0x55786ccb3670 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b867fc640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b7c0b6680 0x7f3b7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d07c090 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005f70 tx=0x7f3b80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df485d0b128b3d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b9579d640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b7c0b6680 0x7f3b7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b867fc640 1 -- 172.21.15.31:0/2552637497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3b880e77c0 con 0x7f3b90109c20 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b7b7fe640 1 -- 172.21.15.31:0/262753340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80035070 con 0x55786ccb3670 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b7b7fe640 1 -- 172.21.15.31:0/262753340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b80005030 con 0x55786ccb3670 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.434+0000 7f3b7b7fe640 1 -- 172.21.15.31:0/262753340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b800044e0 con 0x55786ccb3670 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 -- 172.21.15.31:0/262753340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 msgr2=0x55786d07c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d07c090 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005f70 tx=0x7f3b80000bc0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9579d640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b7c0b6680 0x7f3b7c0b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55786d07a290 tx=0x7f3b8c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 -- 172.21.15.31:0/262753340 shutdown_connections 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d07c090 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d07bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 --2- 172.21.15.31:0/262753340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d07b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 -- 172.21.15.31:0/262753340 >> 172.21.15.31:0/262753340 conn(0x55786ccc8740 msgr2=0x55786d079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 -- 172.21.15.31:0/262753340 shutdown_connections 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 -- 172.21.15.31:0/262753340 wait complete. 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 -- start start 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.435+0000 7f3b9df75740 1 Processor -- start 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccab490 0x55786d087240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccb3670 0x55786d0877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d093080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55786ce42890 con 0x55786ccd4350 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55786cf7bd40 con 0x55786ccb3670 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55786d089d90 con 0x55786ccab490 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9df75740 10 client.0 _ll_register_callbacks cb 0x55786cfdfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d093080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d093080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50454/0 (socket says 172.21.15.31:50454) 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 -- 172.21.15.31:0/3376849091 learned_addr learned my addr 172.21.15.31:0/3376849091 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b9579d640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccab490 0x55786d087240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccb3670 0x55786d0877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 -- 172.21.15.31:0/3376849091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccab490 msgr2=0x55786d087240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccab490 0x55786d087240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 -- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccb3670 msgr2=0x55786d0877e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccb3670 0x55786d0877e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b95f9e640 1 -- 172.21.15.31:0/3376849091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786d0936c0 con 0x55786ccd4350 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.436+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccb3670 0x55786d0877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d093080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3b880ad670 tx=0x7f3b880057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1520da0b6b134126 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b78ff9640 1 -- 172.21.15.31:0/3376849091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8801f040 con 0x55786ccd4350 2024-09-21T02:57:42.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b78ff9640 1 -- 172.21.15.31:0/3376849091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3b880a9030 con 0x55786ccd4350 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b80005030 con 0x55786ccd4350 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b78ff9640 1 -- 172.21.15.31:0/3376849091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b880b7050 con 0x55786ccd4350 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 10 client.16728 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55786d0960a0 con 0x55786ccd4350 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b78ff9640 1 -- 172.21.15.31:0/3376849091 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b80005030 con 0x55786ccd4350 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b78ff9640 1 -- 172.21.15.31:0/3376849091 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b8800f040 con 0x55786ccd4350 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 10 client.16728 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 20 client.16728 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.437+0000 7f3b9df75740 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc92cd0 0x55786d099570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.438+0000 7f3b9df75740 4 client.16728 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.438+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55786cff3db0 con 0x55786cc92cd0 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.438+0000 7f3b9579d640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc92cd0 0x55786d099570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.438+0000 7f3b78ff9640 10 client.16728 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.438+0000 7f3b9579d640 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc92cd0 0x55786d099570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39923e8c7a48b877 server_cookie=40503f0fa778718c in_seq=0 out_seq=0 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.439+0000 7f3b78ff9640 1 -- 172.21.15.31:0/3376849091 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55786cff3db0 con 0x55786cc92cd0 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.439+0000 7f3b78ff9640 10 client.16728 handle_command_reply: tid=0 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.439+0000 7f3b9df75740 1 client.16728 shutdown 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.439+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 msgr2=0x55786d093080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.439+0000 7f3b9df75740 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d093080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3b880ad670 tx=0x7f3b880057a0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.439+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 shutdown_connections 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc92cd0 0x55786d099570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d093080 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccb3670 0x55786d0877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b78ff9640 0 client.16728 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 --2- 172.21.15.31:0/3376849091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccab490 0x55786d087240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 >> 172.21.15.31:0/3376849091 conn(0x55786ccc8740 msgr2=0x55786d07e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 shutdown_connections 2024-09-21T02:57:42.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 1 -- 172.21.15.31:0/3376849091 wait complete. 2024-09-21T02:57:42.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.440+0000 7f3b9df75740 20 client.16728 trim_cache size 0 max 16384 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.457+0000 7f3b9df75740 1 Processor -- start 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.457+0000 7f3b9df75740 1 -- start start 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d092f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d080720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d080c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9df75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901af5a0 con 0x55786ccb3670 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9df75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901b10e0 con 0x55786ccab490 2024-09-21T02:57:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9df75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901afc70 con 0x55786ccd4350 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d080720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d080720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50468/0 (socket says 172.21.15.31:50468) 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 -- 172.21.15.31:0/2881485434 learned_addr learned my addr 172.21.15.31:0/2881485434 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b9579d640 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d092f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d080c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 -- 172.21.15.31:0/2881485434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 msgr2=0x55786d080c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d080c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 -- 172.21.15.31:0/2881485434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 msgr2=0x55786d092f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d092f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b94f9c640 1 -- 172.21.15.31:0/2881485434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786d074bb0 con 0x55786ccb3670 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.458+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.459+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d080720 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3b8002a930 tx=0x7f3b80009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f8b469ac45975 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.459+0000 7f3b7affd640 1 -- 172.21.15.31:0/2881485434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8004e070 con 0x55786ccb3670 2024-09-21T02:57:42.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.459+0000 7f3b7affd640 1 -- 172.21.15.31:0/2881485434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b800054e0 con 0x55786ccb3670 2024-09-21T02:57:42.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.459+0000 7f3b7affd640 1 -- 172.21.15.31:0/2881485434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80035040 con 0x55786ccb3670 2024-09-21T02:57:42.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 -- 172.21.15.31:0/2881485434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 msgr2=0x55786d080720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d080720 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3b8002a930 tx=0x7f3b80009230 comp rx=0 tx=0).stop 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 -- 172.21.15.31:0/2881485434 shutdown_connections 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccd4350 0x55786d080c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccb3670 0x55786d080720 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 --2- 172.21.15.31:0/2881485434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d092f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 -- 172.21.15.31:0/2881485434 >> 172.21.15.31:0/2881485434 conn(0x55786ccc8740 msgr2=0x55786d0858c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 -- 172.21.15.31:0/2881485434 shutdown_connections 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 -- 172.21.15.31:0/2881485434 wait complete. 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 -- start start 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.460+0000 7f3b9df75740 1 Processor -- start 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d08bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccb3670 0x55786d08c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d0fd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901b0a10 con 0x55786ccd4350 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901b0340 con 0x55786ccab490 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901ad580 con 0x55786ccb3670 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9df75740 10 client.0 _ll_register_callbacks cb 0x7f3b880a9dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d0fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d0fd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50470/0 (socket says 172.21.15.31:50470) 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 -- 172.21.15.31:0/1483054980 learned_addr learned my addr 172.21.15.31:0/1483054980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b9579d640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d08bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b94f9c640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccb3670 0x55786d08c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 -- 172.21.15.31:0/1483054980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccb3670 msgr2=0x55786d08c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccb3670 0x55786d08c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 -- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 msgr2=0x55786d08bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d08bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 -- 172.21.15.31:0/1483054980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786d0fd680 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b95f9e640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d0fd010 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3b880ad3e0 tx=0x7f3b880a9690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec40e2b27ec6a392 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.461+0000 7f3b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b78ff9640 1 -- 172.21.15.31:0/1483054980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8801f040 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b78ff9640 1 -- 172.21.15.31:0/1483054980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3b880ac3b0 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b800054e0 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b78ff9640 1 -- 172.21.15.31:0/1483054980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b88004de0 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 10 client.16734 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55786d104ab0 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b78ff9640 1 -- 172.21.15.31:0/1483054980 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b800054e0 con 0x55786ccd4350 2024-09-21T02:57:42.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b78ff9640 1 -- 172.21.15.31:0/1483054980 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b880ad6e0 con 0x55786ccd4350 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 10 client.16734 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 20 client.16734 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc68080 0x55786d106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 4 client.16734 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.462+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55786cfef860 con 0x55786cc68080 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.463+0000 7f3b9579d640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc68080 0x55786d106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.463+0000 7f3b78ff9640 10 client.16734 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.463+0000 7f3b9579d640 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc68080 0x55786d106fa0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f9592d5ab6ce6b server_cookie=448030b3e5fb3b3f in_seq=0 out_seq=0 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b78ff9640 1 -- 172.21.15.31:0/1483054980 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55786cfef860 con 0x55786cc68080 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b78ff9640 10 client.16734 handle_command_reply: tid=0 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 client.16734 shutdown 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 msgr2=0x55786d0fd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d0fd010 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3b880ad3e0 tx=0x7f3b880a9690 comp rx=0 tx=0).stop 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 shutdown_connections 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55786cc68080 0x55786d106fa0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55786ccb3670 0x55786d08c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55786ccd4350 0x55786d0fd010 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 --2- 172.21.15.31:0/1483054980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55786ccab490 0x55786d08bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 >> 172.21.15.31:0/1483054980 conn(0x55786ccc8740 msgr2=0x55786d07c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b78ff9640 0 client.16734 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 shutdown_connections 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 1 -- 172.21.15.31:0/1483054980 wait complete. 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.465+0000 7f3b9df75740 20 client.16734 trim_cache size 0 max 16384 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:57:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "affa0fa7-6833-420c-a39f-8c6ea226e382", 2024-09-21T02:57:42.471 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:57:42.471 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:57:42.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b7c0b6680 msgr2=0x7f3b7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b7c0b6680 0x7f3b7c0b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55786d07a290 tx=0x7f3b8c00b390 comp rx=0 tx=0).stop 2024-09-21T02:57:42.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 msgr2=0x7f3b9019fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 0x7f3b9019fb30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800eaa0 tx=0x7f3b8800ef60 comp rx=0 tx=0).stop 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 shutdown_connections 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b7c0b6680 0x7f3b7c0b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b90109c20 0x7f3b9019fb30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b901092d0 0x7f3b9019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 --2- 172.21.15.31:0/2552637497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b90103990 0x7f3b9019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 >> 172.21.15.31:0/2552637497 conn(0x7f3b900ff3e0 msgr2=0x7f3b90101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 shutdown_connections 2024-09-21T02:57:42.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:42.466+0000 7f3b9679f640 1 -- 172.21.15.31:0/2552637497 wait complete. 2024-09-21T02:57:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:42 smithi165 ceph-mon[34818]: pgmap v469: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:57:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:42 smithi178 ceph-mon[34759]: pgmap v469: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:57:43.228 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:42 smithi031 ceph-mon[31209]: pgmap v469: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:57:43.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:57:43.520 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 -- 172.21.15.31:0/3183082172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388105280 msgr2=0x7f238810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 --2- 172.21.15.31:0/3183082172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388105280 0x7f238810d8c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2370009900 tx=0x7f237002a410 comp rx=0 tx=0).stop 2024-09-21T02:57:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 -- 172.21.15.31:0/3183082172 shutdown_connections 2024-09-21T02:57:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 --2- 172.21.15.31:0/3183082172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f238810de00 0x7f2388110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 --2- 172.21.15.31:0/3183082172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388105280 0x7f238810d8c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 --2- 172.21.15.31:0/3183082172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2388104850 0x7f2388104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 -- 172.21.15.31:0/3183082172 >> 172.21.15.31:0/3183082172 conn(0x7f2388079730 msgr2=0x7f2388079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 -- 172.21.15.31:0/3183082172 shutdown_connections 2024-09-21T02:57:44.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.026+0000 7f238e504640 1 -- 172.21.15.31:0/3183082172 wait complete. 2024-09-21T02:57:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 Processor -- start 2024-09-21T02:57:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 -- start start 2024-09-21T02:57:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 0x7f238819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2388105280 0x7f238819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f238810de00 0x7f238819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2388113a80 con 0x7f2388104850 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2388113900 con 0x7f238810de00 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.027+0000 7f238e504640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2388113c00 con 0x7f2388105280 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 0x7f238819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 0x7f238819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50486/0 (socket says 172.21.15.31:50486) 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 -- 172.21.15.31:0/3417232649 learned_addr learned my addr 172.21.15.31:0/3417232649 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f237ffff640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2388105280 0x7f238819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f238caa1640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f238810de00 0x7f238819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 -- 172.21.15.31:0/3417232649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2388105280 msgr2=0x7f238819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2388105280 0x7f238819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 -- 172.21.15.31:0/3417232649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f238810de00 msgr2=0x7f238819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f238810de00 0x7f238819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 -- 172.21.15.31:0/3417232649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23881a0560 con 0x7f2388104850 2024-09-21T02:57:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2387fff640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 0x7f238819a5f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f237800d950 tx=0x7f237800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2385ffb640 1 -- 172.21.15.31:0/3417232649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378017070 con 0x7f2388104850 2024-09-21T02:57:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2385ffb640 1 -- 172.21.15.31:0/3417232649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2378009c30 con 0x7f2388104850 2024-09-21T02:57:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23881a0850 con 0x7f2388104850 2024-09-21T02:57:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f2385ffb640 1 -- 172.21.15.31:0/3417232649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237800f620 con 0x7f2388104850 2024-09-21T02:57:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.028+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881a6fb0 con 0x7f2388104850 2024-09-21T02:57:44.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 Processor -- start 2024-09-21T02:57:44.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 -- start start 2024-09-21T02:57:44.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2385ffb640 1 -- 172.21.15.31:0/3417232649 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f237801a070 con 0x7f2388104850 2024-09-21T02:57:44.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e779006650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 0x55e779006b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e778f7ce50 con 0x55e778c361b0 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e778f6c280 con 0x55e778c5f070 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2394a78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e778f70090 con 0x55e778c3e390 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 0x55e779006b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.031+0000 7f2387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 0x55e779006b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50500/0 (socket says 172.21.15.31:50500) 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 -- 172.21.15.31:0/1145241251 learned_addr learned my addr 172.21.15.31:0/1145241251 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f238caa1640 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e779006650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f237ffff640 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 -- 172.21.15.31:0/1145241251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 msgr2=0x55e7790070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 -- 172.21.15.31:0/1145241251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 msgr2=0x55e779006650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e779006650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 -- 172.21.15.31:0/1145241251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e778cca650 con 0x55e778c361b0 2024-09-21T02:57:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f237ffff640 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2387fff640 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 0x55e779006b90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f23780e4750 tx=0x7f23780ef780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabe653fc8dc85c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f2385ffb640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23740b6700 0x7f23740b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f237e7fc640 1 -- 172.21.15.31:0/1145241251 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780fa070 con 0x55e778c361b0 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.032+0000 7f238caa1640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23740b6700 0x7f23740b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.033+0000 7f237e7fc640 1 -- 172.21.15.31:0/1145241251 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780b7980 con 0x55e778c361b0 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.033+0000 7f237e7fc640 1 -- 172.21.15.31:0/1145241251 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780a8e30 con 0x55e778c361b0 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.033+0000 7f2385ffb640 1 -- 172.21.15.31:0/3417232649 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f23780e7070 con 0x7f2388104850 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.033+0000 7f238caa1640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23740b6700 0x7f23740b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f238000ae90 tx=0x7f238000a4d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.033+0000 7f2394a78740 1 -- 172.21.15.31:0/1145241251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 msgr2=0x55e779006b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.033+0000 7f2394a78740 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 0x55e779006b90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f23780e4750 tx=0x7f23780ef780 comp rx=0 tx=0).stop 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 -- 172.21.15.31:0/1145241251 shutdown_connections 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c361b0 0x55e779006b90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 --2- 172.21.15.31:0/1145241251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e779006650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 -- 172.21.15.31:0/1145241251 >> 172.21.15.31:0/1145241251 conn(0x55e778c53460 msgr2=0x55e779004540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 -- 172.21.15.31:0/1145241251 shutdown_connections 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 -- 172.21.15.31:0/1145241251 wait complete. 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 -- start start 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.034+0000 7f2394a78740 1 Processor -- start 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e779011d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e77901e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e778dcd700 con 0x55e778c5f070 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e778f07110 con 0x55e778c361b0 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e779014e30 con 0x55e778c3e390 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2394a78740 10 client.0 _ll_register_callbacks cb 0x55e778f20990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e77901e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e77901e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50512/0 (socket says 172.21.15.31:50512) 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 -- 172.21.15.31:0/4121592335 learned_addr learned my addr 172.21.15.31:0/4121592335 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:44.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f2387fff640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e779011d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f237ffff640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 -- 172.21.15.31:0/4121592335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 msgr2=0x55e7790122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790122d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 -- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 msgr2=0x55e779011d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e779011d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 -- 172.21.15.31:0/4121592335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e778f34c40 con 0x55e778c5f070 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f238caa1640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e77901e150 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f238001ec20 tx=0x7f2380004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd79871e35491dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.035+0000 7f23677fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f23677fe640 1 -- 172.21.15.31:0/4121592335 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238002b060 con 0x55e778c5f070 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23780b7980 con 0x55e778c5f070 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f23677fe640 1 -- 172.21.15.31:0/4121592335 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2380016030 con 0x55e778c5f070 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f23677fe640 1 -- 172.21.15.31:0/4121592335 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f238001ec50 con 0x55e778c5f070 2024-09-21T02:57:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 10 client.16746 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e779020c50 con 0x55e778c5f070 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f23677fe640 1 -- 172.21.15.31:0/4121592335 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780b7980 con 0x55e778c5f070 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f23677fe640 1 -- 172.21.15.31:0/4121592335 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f238001b400 con 0x55e778c5f070 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 10 client.16746 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 20 client.16746 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778c1d9f0 0x55e7790240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 4 client.16746 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.036+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e778f7eda0 con 0x55e778c1d9f0 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.037+0000 7f2387fff640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778c1d9f0 0x55e7790240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.037+0000 7f2387fff640 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778c1d9f0 0x55e7790240b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8cde722a8eaea7 server_cookie=86e42c23c08d42ab in_seq=0 out_seq=0 2024-09-21T02:57:44.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.037+0000 7f23677fe640 10 client.16746 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f23677fe640 1 -- 172.21.15.31:0/4121592335 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e778f7eda0 con 0x55e778c1d9f0 2024-09-21T02:57:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f23677fe640 10 client.16746 handle_command_reply: tid=0 2024-09-21T02:57:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 client.16746 shutdown 2024-09-21T02:57:44.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 msgr2=0x55e77901e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e77901e150 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f238001ec20 tx=0x7f2380004ab0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 shutdown_connections 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778c1d9f0 0x55e7790240b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e7790122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e77901e150 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 --2- 172.21.15.31:0/4121592335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e779011d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 >> 172.21.15.31:0/4121592335 conn(0x55e778c53460 msgr2=0x55e7790097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f23677fe640 0 client.16746 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 shutdown_connections 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 1 -- 172.21.15.31:0/4121592335 wait complete. 2024-09-21T02:57:44.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.038+0000 7f2394a78740 20 client.16746 trim_cache size 0 max 16384 2024-09-21T02:57:44.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 Processor -- start 2024-09-21T02:57:44.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 -- start start 2024-09-21T02:57:44.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c361b0 0x55e7790139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 0x55e779003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e7790040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ad650 con 0x55e778c3e390 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23881af190 con 0x55e778c5f070 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2394a78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23881add20 con 0x55e778c361b0 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 0x55e779003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 0x55e779003b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50524/0 (socket says 172.21.15.31:50524) 2024-09-21T02:57:44.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 -- 172.21.15.31:0/1144278280 learned_addr learned my addr 172.21.15.31:0/1144278280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:44.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f2387fff640 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c361b0 0x55e7790139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 -- 172.21.15.31:0/1144278280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c361b0 msgr2=0x55e7790139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c361b0 0x55e7790139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 -- 172.21.15.31:0/1144278280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 msgr2=0x55e7790040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e7790040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.056+0000 7f237ffff640 1 -- 172.21.15.31:0/1144278280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e779007460 con 0x55e778c3e390 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.057+0000 7f237ffff640 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 0x55e779003b70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f238806b950 tx=0x7f2370000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb64b1b73212aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.057+0000 7f237dffb640 1 -- 172.21.15.31:0/1144278280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237004b020 con 0x55e778c3e390 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.057+0000 7f237dffb640 1 -- 172.21.15.31:0/1144278280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2370031d10 con 0x55e778c3e390 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.057+0000 7f237dffb640 1 -- 172.21.15.31:0/1144278280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370035040 con 0x55e778c3e390 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 -- 172.21.15.31:0/1144278280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 msgr2=0x55e779003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 0x55e779003b70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f238806b950 tx=0x7f2370000bc0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 -- 172.21.15.31:0/1144278280 shutdown_connections 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c5f070 0x55e7790040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c3e390 0x55e779003b70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 --2- 172.21.15.31:0/1144278280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c361b0 0x55e7790139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 -- 172.21.15.31:0/1144278280 >> 172.21.15.31:0/1144278280 conn(0x55e778c53460 msgr2=0x55e7790132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 -- 172.21.15.31:0/1144278280 shutdown_connections 2024-09-21T02:57:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 -- 172.21.15.31:0/1144278280 wait complete. 2024-09-21T02:57:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 -- start start 2024-09-21T02:57:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.058+0000 7f2394a78740 1 Processor -- start 2024-09-21T02:57:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e77901aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e779015eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e779016550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23881aeac0 con 0x55e778c5f070 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ae3f0 con 0x55e778c361b0 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ab630 con 0x55e778c3e390 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2394a78740 10 client.0 _ll_register_callbacks cb 0x7f238000a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e779016550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e779016550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50532/0 (socket says 172.21.15.31:50532) 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 -- 172.21.15.31:0/3617593238 learned_addr learned my addr 172.21.15.31:0/3617593238 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f2387fff640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e77901aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f237ffff640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e779015eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 -- 172.21.15.31:0/3617593238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 msgr2=0x55e779015eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e779015eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 -- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 msgr2=0x55e77901aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:57:44.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e77901aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 -- 172.21.15.31:0/3617593238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e779016ce0 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f238caa1640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e779016550 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f238000a990 tx=0x7f238001b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415940890fd40c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f23677fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.059+0000 7f23677fe640 1 -- 172.21.15.31:0/3617593238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238001be60 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f23677fe640 1 -- 172.21.15.31:0/3617593238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f2380004c10 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2370031d10 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f23677fe640 1 -- 172.21.15.31:0/3617593238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f238002b030 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 10 client.16752 fetch_fsmap learned FSMap version 27 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e77908eb30 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f23677fe640 1 -- 172.21.15.31:0/3617593238 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2370031d10 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f23677fe640 1 -- 172.21.15.31:0/3617593238 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f238003f070 con 0x55e778c5f070 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 10 client.16752 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:57:44.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 20 client.16752 populate_metadata read hostname 'smithi031' 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778bf2da0 0x55e779091250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 4 client.16752 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e778f7a850 con 0x55e778bf2da0 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.060+0000 7f2387fff640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778bf2da0 0x55e779091250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2387fff640 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778bf2da0 0x55e779091250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f104f445374fe4 server_cookie=7ae15287d25f399a in_seq=0 out_seq=0 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f23677fe640 10 client.16752 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f23677fe640 1 -- 172.21.15.31:0/3617593238 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55e778f7a850 con 0x55e778bf2da0 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f23677fe640 10 client.16752 handle_command_reply: tid=0 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 client.16752 shutdown 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 msgr2=0x55e779016550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e779016550 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f238000a990 tx=0x7f238001b630 comp rx=0 tx=0).stop 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 shutdown_connections 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55e778bf2da0 0x55e779091250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e778c3e390 0x55e779015eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f23677fe640 0 client.16752 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e778c5f070 0x55e779016550 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.061+0000 7f2394a78740 1 --2- 172.21.15.31:0/3617593238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e778c361b0 0x55e77901aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 >> 172.21.15.31:0/3617593238 conn(0x55e778c53460 msgr2=0x55e7790131e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 shutdown_connections 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f2394a78740 1 -- 172.21.15.31:0/3617593238 wait complete. 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f2394a78740 20 client.16752 trim_cache size 0 max 16384 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:57:44.067 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "affa0fa7-6833-420c-a39f-8c6ea226e382", 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.069 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.070 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.071 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:57:44.072 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:57:44.073 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23740b6700 msgr2=0x7f23740b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23740b6700 0x7f23740b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f238000ae90 tx=0x7f238000a4d0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 msgr2=0x7f238819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 0x7f238819a5f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f237800d950 tx=0x7f237800de10 comp rx=0 tx=0).stop 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 shutdown_connections 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23740b6700 0x7f23740b8c10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2388105280 0x7f238819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f238810de00 0x7f238819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 --2- 172.21.15.31:0/3417232649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2388104850 0x7f238819a5f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.062+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 >> 172.21.15.31:0/3417232649 conn(0x7f2388079730 msgr2=0x7f2388100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.063+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 shutdown_connections 2024-09-21T02:57:44.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:57:44.063+0000 7f238e504640 1 -- 172.21.15.31:0/3417232649 wait complete. 2024-09-21T02:57:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:43 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:57:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:43 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:57:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:43 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:57:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:43 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:57:44.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:43 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:57:44.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:43 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:57:44.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:43 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:57:44.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:43 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:57:44.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:43 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:57:44.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:43 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:57:44.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:43 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:57:44.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:43 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:57:45.071 INFO:tasks.cephfs.filesystem:scrub status for tag:affa0fa7-6833-420c-a39f-8c6ea226e382 - {'path': '/', 'tag': 'affa0fa7-6833-420c-a39f-8c6ea226e382', 'options': 'recursive,force'} 2024-09-21T02:57:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:44 smithi178 ceph-mon[34759]: pgmap v470: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:57:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:44 smithi031 ceph-mon[31209]: pgmap v470: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:57:45.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:44 smithi165 ceph-mon[34818]: pgmap v470: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:57:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:45 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:57:46.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:45 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:57:46.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:45 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:57:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:46 smithi178 ceph-mon[34759]: pgmap v471: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:47.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:46 smithi031 ceph-mon[31209]: pgmap v471: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:47.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:46 smithi165 ceph-mon[34818]: pgmap v471: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:57:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:48 smithi178 ceph-mon[34759]: pgmap v472: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:57:49.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:48 smithi031 ceph-mon[31209]: pgmap v472: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:57:49.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:48 smithi165 ceph-mon[34818]: pgmap v472: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:57:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:50 smithi178 ceph-mon[34759]: pgmap v473: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T02:57:51.239 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:50 smithi031 ceph-mon[31209]: pgmap v473: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T02:57:51.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:50 smithi165 ceph-mon[34818]: pgmap v473: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T02:57:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:52 smithi178 ceph-mon[34759]: pgmap v474: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T02:57:53.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:52 smithi031 ceph-mon[31209]: pgmap v474: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T02:57:53.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:52 smithi165 ceph-mon[34818]: pgmap v474: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T02:57:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:54 smithi178 ceph-mon[34759]: pgmap v475: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T02:57:55.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:54 smithi031 ceph-mon[31209]: pgmap v475: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T02:57:55.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:54 smithi165 ceph-mon[34818]: pgmap v475: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T02:57:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:56 smithi178 ceph-mon[34759]: pgmap v476: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T02:57:57.243 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:56 smithi165 ceph-mon[34818]: pgmap v476: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T02:57:57.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:56 smithi031 ceph-mon[31209]: pgmap v476: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T02:57:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:57:58 smithi178 ceph-mon[34759]: pgmap v477: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:57:59.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:57:58 smithi031 ceph-mon[31209]: pgmap v477: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:57:59.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:57:58 smithi165 ceph-mon[34818]: pgmap v477: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:58:01.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:00 smithi031 ceph-mon[31209]: pgmap v478: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:01.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:00 smithi165 ceph-mon[34818]: pgmap v478: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:01.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:00 smithi178 ceph-mon[34759]: pgmap v478: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:03.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:02 smithi031 ceph-mon[31209]: pgmap v479: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:58:03.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:02 smithi165 ceph-mon[34818]: pgmap v479: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:58:03.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:02 smithi178 ceph-mon[34759]: pgmap v479: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:58:05.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:04 smithi031 ceph-mon[31209]: pgmap v480: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:58:05.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:04 smithi165 ceph-mon[34818]: pgmap v480: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:58:05.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:04 smithi178 ceph-mon[34759]: pgmap v480: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:58:07.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:06 smithi031 ceph-mon[31209]: pgmap v481: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:07.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:06 smithi165 ceph-mon[34818]: pgmap v481: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:07.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:06 smithi178 ceph-mon[34759]: pgmap v481: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:09.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:08 smithi031 ceph-mon[31209]: pgmap v482: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:58:09.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:08 smithi165 ceph-mon[34818]: pgmap v482: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:58:09.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:08 smithi178 ceph-mon[34759]: pgmap v482: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:58:11.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:10 smithi031 ceph-mon[31209]: pgmap v483: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:11.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:10 smithi165 ceph-mon[34818]: pgmap v483: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:11.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:10 smithi178 ceph-mon[34759]: pgmap v483: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:13.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:12 smithi031 ceph-mon[31209]: pgmap v484: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:13.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:12 smithi165 ceph-mon[34818]: pgmap v484: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:13.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:12 smithi178 ceph-mon[34759]: pgmap v484: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:58:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:58:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:14.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:58:14.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:14.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:14.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:14.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:58:15.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:58:15.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:15 smithi031 ceph-mon[31209]: pgmap v485: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:58:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:15.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:15 smithi165 ceph-mon[34818]: pgmap v485: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:58:15.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:15 smithi178 ceph-mon[34759]: pgmap v485: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.972+0000 7f0a28e27640 1 -- 172.21.15.31:0/4273976627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 msgr2=0x7f0a24102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.972+0000 7f0a28e27640 1 --2- 172.21.15.31:0/4273976627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a24102e90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009900 tx=0x7f0a0c02a410 comp rx=0 tx=0).stop 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 -- 172.21.15.31:0/4273976627 shutdown_connections 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 --2- 172.21.15.31:0/4273976627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 0x7f0a241087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 --2- 172.21.15.31:0/4273976627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 0x7f0a2410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 --2- 172.21.15.31:0/4273976627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a24102e90 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 -- 172.21.15.31:0/4273976627 >> 172.21.15.31:0/4273976627 conn(0x7f0a24079740 msgr2=0x7f0a24079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 -- 172.21.15.31:0/4273976627 shutdown_connections 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.973+0000 7f0a28e27640 1 -- 172.21.15.31:0/4273976627 wait complete. 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 Processor -- start 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 -- start start 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a241993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 0x7f0a24199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 0x7f0a2419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112980 con 0x7f0a24102a80 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112800 con 0x7f0a24108d10 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a28e27640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112b00 con 0x7f0a241083c0 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a241993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a241993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58092/0 (socket says 172.21.15.31:58092) 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.974+0000 7f0a2259c640 1 -- 172.21.15.31:0/3879848871 learned_addr learned my addr 172.21.15.31:0/3879848871 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:15.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 0x7f0a24199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a2259c640 1 -- 172.21.15.31:0/3879848871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 msgr2=0x7f0a24199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a2259c640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 0x7f0a24199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a2259c640 1 -- 172.21.15.31:0/3879848871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 msgr2=0x7f0a2419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 0x7f0a2419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a2259c640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 0x7f0a2419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a2259c640 1 -- 172.21.15.31:0/3879848871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2419f360 con 0x7f0a24102a80 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 0x7f0a24199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 0x7f0a2419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a2259c640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a241993f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005940 tx=0x7f0a0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a1b7fe640 1 -- 172.21.15.31:0/3879848871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c046070 con 0x7f0a24102a80 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a1b7fe640 1 -- 172.21.15.31:0/3879848871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a0c003aa0 con 0x7f0a24102a80 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2419f5f0 con 0x7f0a24102a80 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a1b7fe640 1 -- 172.21.15.31:0/3879848871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c035050 con 0x7f0a24102a80 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.975+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2406bb40 con 0x7f0a24102a80 2024-09-21T02:58:15.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 Processor -- start 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 -- start start 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a1b7fe640 1 -- 172.21.15.31:0/3879848871 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0a0c049070 con 0x7f0a24102a80 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445587650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445587b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5624454fde50 con 0x5624451bf390 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5624454ed280 con 0x5624451b71b0 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.978+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5624454f1090 con 0x5624451e0070 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445587b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445587650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445587650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:49926/0 (socket says 172.21.15.31:49926) 2024-09-21T02:58:15.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445587b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:43860/0 (socket says 172.21.15.31:43860) 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58106/0 (socket says 172.21.15.31:58106) 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a2259c640 1 -- 172.21.15.31:0/823569959 learned_addr learned my addr 172.21.15.31:0/823569959 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 -- 172.21.15.31:0/823569959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 msgr2=0x562445587650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445587650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 -- 172.21.15.31:0/823569959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 msgr2=0x562445587b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445587b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 -- 172.21.15.31:0/823569959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56244524b650 con 0x5624451bf390 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a2259c640 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445587b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a1b7fe640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0a100b6680 0x7f0a100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0a100b6680 0x7f0a100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455880d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0a2406b6c0 tx=0x7f0a14010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82045bf85054ce98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a1b7fe640 1 -- 172.21.15.31:0/3879848871 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0a0c1092e0 con 0x7f0a24102a80 2024-09-21T02:58:15.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.979+0000 7f0a18ff9640 1 -- 172.21.15.31:0/823569959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14004030 con 0x5624451bf390 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.980+0000 7f0a18ff9640 1 -- 172.21.15.31:0/823569959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a140035f0 con 0x5624451bf390 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.980+0000 7f0a18ff9640 1 -- 172.21.15.31:0/823569959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a140126b0 con 0x5624451bf390 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.980+0000 7f0a2f39b740 1 -- 172.21.15.31:0/823569959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 msgr2=0x5624455880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.980+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0a100b6680 0x7f0a100b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009370 tx=0x7f0a1c009720 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.980+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455880d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0a2406b6c0 tx=0x7f0a14010740 comp rx=0 tx=0).stop 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 -- 172.21.15.31:0/823569959 shutdown_connections 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455880d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445587b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/823569959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445587650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 -- 172.21.15.31:0/823569959 >> 172.21.15.31:0/823569959 conn(0x5624451d4460 msgr2=0x562445585540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 -- 172.21.15.31:0/823569959 shutdown_connections 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 -- 172.21.15.31:0/823569959 wait complete. 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 -- start start 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.981+0000 7f0a2f39b740 1 Processor -- start 2024-09-21T02:58:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445592c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445593710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56244534e700 con 0x5624451bf390 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562445488110 con 0x5624451b71b0 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562445595e30 con 0x5624451e0070 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2f39b740 10 client.0 _ll_register_callbacks cb 0x5624454a1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455931b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58118/0 (socket says 172.21.15.31:58118) 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 -- 172.21.15.31:0/2047688672 learned_addr learned my addr 172.21.15.31:0/2047688672 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445592c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445593710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 -- 172.21.15.31:0/2047688672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 msgr2=0x562445593710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445593710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 -- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 msgr2=0x562445592c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445592c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 -- 172.21.15.31:0/2047688672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624454b5c40 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445592c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445593710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455931b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0a140057b0 tx=0x7f0a14009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5018f2076b11256d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.982+0000 7f0a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2047688672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a140039b0 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a140035f0 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2047688672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0a14003f90 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2047688672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a14004190 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 10 client.16764 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5624455a1ab0 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2047688672 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a140035f0 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2047688672 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0a1402e070 con 0x5624451bf390 2024-09-21T02:58:15.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 10 client.16764 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:15.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:15.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 20 client.16764 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:15.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56244519e9f0 0x5624455a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 4 client.16764 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:15.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.983+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624454ffda0 con 0x56244519e9f0 2024-09-21T02:58:15.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.984+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56244519e9f0 0x5624455a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.984+0000 7f0a09ffb640 10 client.16764 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.984+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56244519e9f0 0x5624455a4e10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63582b01d57fe114 server_cookie=da4ed75b7301835e in_seq=0 out_seq=0 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2047688672 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624454ffda0 con 0x56244519e9f0 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a09ffb640 10 client.16764 handle_command_reply: tid=0 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 client.16764 shutdown 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 msgr2=0x5624455931b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455931b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0a140057b0 tx=0x7f0a14009cd0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 shutdown_connections 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56244519e9f0 0x5624455a4e10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451e0070 0x562445593710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451bf390 0x5624455931b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2047688672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451b71b0 0x562445592c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 >> 172.21.15.31:0/2047688672 conn(0x5624451d4460 msgr2=0x56244558a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a09ffb640 0 client.16764 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 shutdown_connections 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.985+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2047688672 wait complete. 2024-09-21T02:58:15.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:15.986+0000 7f0a2f39b740 20 client.16764 trim_cache size 0 max 16384 2024-09-21T02:58:16.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 Processor -- start 2024-09-21T02:58:16.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 -- start start 2024-09-21T02:58:16.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451b71b0 0x562445589040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451bf390 0x5624455948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 0x562445584b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241ad4c0 con 0x5624451e0070 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241af000 con 0x5624451bf390 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241adb90 con 0x5624451b71b0 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 0x562445584b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a22d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 0x562445584b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58122/0 (socket says 172.21.15.31:58122) 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a22d9d640 1 -- 172.21.15.31:0/966413109 learned_addr learned my addr 172.21.15.31:0/966413109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.003+0000 7f0a2259c640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451b71b0 0x562445589040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451bf390 0x5624455948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a22d9d640 1 -- 172.21.15.31:0/966413109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451b71b0 msgr2=0x562445589040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:16.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451b71b0 0x562445589040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a22d9d640 1 -- 172.21.15.31:0/966413109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451bf390 msgr2=0x5624455948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451bf390 0x5624455948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a22d9d640 1 -- 172.21.15.31:0/966413109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624455957a0 con 0x5624451e0070 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451bf390 0x5624455948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a2259c640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451b71b0 0x562445589040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 0x562445584b70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0127a0 tx=0x7f0a1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d938014676be1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a0bfff640 1 -- 172.21.15.31:0/966413109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c004b20 con 0x5624451e0070 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a0bfff640 1 -- 172.21.15.31:0/966413109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c01ede0 con 0x5624451e0070 2024-09-21T02:58:16.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.004+0000 7f0a0bfff640 1 -- 172.21.15.31:0/966413109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0057b0 con 0x5624451e0070 2024-09-21T02:58:16.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 -- 172.21.15.31:0/966413109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 msgr2=0x562445584b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 0x562445584b70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0127a0 tx=0x7f0a1c014000 comp rx=0 tx=0).stop 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 -- 172.21.15.31:0/966413109 shutdown_connections 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451e0070 0x562445584b70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451bf390 0x5624455948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/966413109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451b71b0 0x562445589040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 -- 172.21.15.31:0/966413109 >> 172.21.15.31:0/966413109 conn(0x5624451d4460 msgr2=0x562445594190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 -- 172.21.15.31:0/966413109 shutdown_connections 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 -- 172.21.15.31:0/966413109 wait complete. 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.005+0000 7f0a2f39b740 1 -- start start 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 Processor -- start 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 0x56244559ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451bf390 0x562445596ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451e0070 0x5624455975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241ae930 con 0x5624451b71b0 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241ae260 con 0x5624451e0070 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241ab4a0 con 0x5624451bf390 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2f39b740 10 client.0 _ll_register_callbacks cb 0x7f0a0c10cd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 0x56244559ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 0x56244559ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58130/0 (socket says 172.21.15.31:58130) 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 -- 172.21.15.31:0/2288824229 learned_addr learned my addr 172.21.15.31:0/2288824229 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451bf390 0x562445596ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a22d9d640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451e0070 0x5624455975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 -- 172.21.15.31:0/2288824229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451bf390 msgr2=0x562445596ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451bf390 0x562445596ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 -- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451e0070 msgr2=0x5624455975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451e0070 0x5624455975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.006+0000 7f0a2259c640 1 -- 172.21.15.31:0/2288824229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244559f020 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a2259c640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 0x56244559ba20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006010 tx=0x7f0a0c10b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c102d2573c76a89e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2288824229 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0caff0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2288824229 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0a0c0d86e0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a1c01ede0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2288824229 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c0ce8c0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a2f39b740 10 client.16770 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56244560f6d0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.007+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2288824229 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c01ede0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2288824229 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0a0c0ba1f0 con 0x5624451b71b0 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a2f39b740 10 client.16770 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a2f39b740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a2f39b740 20 client.16770 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562445173da0 0x562445611d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a2f39b740 4 client.16770 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:58:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624454fb850 con 0x562445173da0 2024-09-21T02:58:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.008+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562445173da0 0x562445611d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a21d9b640 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562445173da0 0x562445611d30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195bed705f138faf server_cookie=50794bd754e08bf0 in_seq=0 out_seq=0 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a09ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a09ffb640 1 -- 172.21.15.31:0/2288824229 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5624454fb850 con 0x562445173da0 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a09ffb640 10 client.16770 handle_command_reply: tid=0 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 client.16770 shutdown 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 msgr2=0x56244559ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 0x56244559ba20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006010 tx=0x7f0a0c10b7f0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 shutdown_connections 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562445173da0 0x562445611d30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5624451bf390 0x562445596ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5624451e0070 0x5624455975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a09ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 --2- 172.21.15.31:0/2288824229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624451b71b0 0x56244559ba20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 >> 172.21.15.31:0/2288824229 conn(0x5624451d4460 msgr2=0x562445588e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 shutdown_connections 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 1 -- 172.21.15.31:0/2288824229 wait complete. 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.009+0000 7f0a2f39b740 20 client.16770 trim_cache size 0 max 16384 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T02:58:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:58:16.016 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.017 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:58:16.020 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0a100b6680 msgr2=0x7f0a100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0a100b6680 0x7f0a100b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009370 tx=0x7f0a1c009720 comp rx=0 tx=0).stop 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 msgr2=0x7f0a241993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a241993f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005940 tx=0x7f0a0c004270 comp rx=0 tx=0).stop 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 shutdown_connections 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f0a100b6680 0x7f0a100b8b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241083c0 0x7f0a24199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a24108d10 0x7f0a2419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 --2- 172.21.15.31:0/3879848871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a24102a80 0x7f0a241993f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 >> 172.21.15.31:0/3879848871 conn(0x7f0a24079740 msgr2=0x7f0a241014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 shutdown_connections 2024-09-21T02:58:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:16.010+0000 7f0a28e27640 1 -- 172.21.15.31:0/3879848871 wait complete. 2024-09-21T02:58:16.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:16 smithi031 ceph-mon[31209]: pgmap v486: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:58:16.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:16 smithi165 ceph-mon[34818]: pgmap v486: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:58:16.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:16 smithi178 ceph-mon[34759]: pgmap v486: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:58:16.960 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:58:16.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:17.218 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:17.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 -- 172.21.15.31:0/3957973905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5854108cf0 msgr2=0x7f585410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:17.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 --2- 172.21.15.31:0/3957973905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5854108cf0 0x7f585410f520 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f584c009ca0 tx=0x7f584c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 -- 172.21.15.31:0/3957973905 shutdown_connections 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 --2- 172.21.15.31:0/3957973905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58541083a0 0x7f58541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 --2- 172.21.15.31:0/3957973905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5854108cf0 0x7f585410f520 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 --2- 172.21.15.31:0/3957973905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854102a60 0x7f5854102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 -- 172.21.15.31:0/3957973905 >> 172.21.15.31:0/3957973905 conn(0x7f5854079730 msgr2=0x7f5854079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 -- 172.21.15.31:0/3957973905 shutdown_connections 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 -- 172.21.15.31:0/3957973905 wait complete. 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.702+0000 7f585bc83640 1 Processor -- start 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 -- start start 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5854102a60 0x7f585419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 0x7f585419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854108cf0 0x7f585419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5854112960 con 0x7f58541083a0 2024-09-21T02:58:17.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58541127e0 con 0x7f5854108cf0 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585bc83640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5854112ae0 con 0x7f5854102a60 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 0x7f585419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 0x7f585419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58148/0 (socket says 172.21.15.31:58148) 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 -- 172.21.15.31:0/988364036 learned_addr learned my addr 172.21.15.31:0/988364036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f5859a1f640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5854102a60 0x7f585419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 -- 172.21.15.31:0/988364036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5854102a60 msgr2=0x7f585419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5854102a60 0x7f585419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585a220640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854108cf0 0x7f585419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 -- 172.21.15.31:0/988364036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854108cf0 msgr2=0x7f585419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854108cf0 0x7f585419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 -- 172.21.15.31:0/988364036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58541a0390 con 0x7f58541083a0 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585a220640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854108cf0 0x7f585419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.703+0000 7f585921e640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 0x7f585419a9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f584400e9f0 tx=0x7f584400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:17.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.704+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584400ccc0 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.704+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58440054d0 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.704+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58541a0680 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.704+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844010430 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.704+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58541a6de0 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.705+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5854103ce0 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.707+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f58440046c0 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.707+0000 7f584affd640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f58400b6700 0x7f58400b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.707+0000 7f5859a1f640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f58400b6700 0x7f58400b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.708+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f58440e7c80 con 0x7f58541083a0 2024-09-21T02:58:17.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.708+0000 7f5859a1f640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f58400b6700 0x7f58400b8c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5850005d00 tx=0x7f5850005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:17.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.710+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5844056710 con 0x7f58541083a0 2024-09-21T02:58:17.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.937+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5854102e70 con 0x7f58541083a0 2024-09-21T02:58:17.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.938+0000 7f584affd640 1 -- 172.21.15.31:0/988364036 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f58440ae1f0 con 0x7f58541083a0 2024-09-21T02:58:17.939 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:58:17.939 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:58:17.940 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f58400b6700 msgr2=0x7f58400b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f58400b6700 0x7f58400b8c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5850005d00 tx=0x7f5850005c50 comp rx=0 tx=0).stop 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 msgr2=0x7f585419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 0x7f585419a9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f584400e9f0 tx=0x7f584400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 shutdown_connections 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f58400b6700 0x7f58400b8c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5854108cf0 0x7f585419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58541083a0 0x7f585419a9c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 --2- 172.21.15.31:0/988364036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5854102a60 0x7f585419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 >> 172.21.15.31:0/988364036 conn(0x7f5854079730 msgr2=0x7f585406bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 shutdown_connections 2024-09-21T02:58:17.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:17.940+0000 7f585bc83640 1 -- 172.21.15.31:0/988364036 wait complete. 2024-09-21T02:58:18.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:18 smithi031 ceph-mon[31209]: pgmap v487: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:58:18.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:18 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/988364036' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:18.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:58:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:18 smithi165 ceph-mon[34818]: pgmap v487: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:58:18.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:18 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/988364036' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:18 smithi178 ceph-mon[34759]: pgmap v487: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:58:18.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:18 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/988364036' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:18.839 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 -- 172.21.15.31:0/458233160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c102a60 msgr2=0x7fbc2c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 --2- 172.21.15.31:0/458233160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c102a60 0x7fbc2c102e70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009900 tx=0x7fbc1c02a410 comp rx=0 tx=0).stop 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 -- 172.21.15.31:0/458233160 shutdown_connections 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 --2- 172.21.15.31:0/458233160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c108cf0 0x7fbc2c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 --2- 172.21.15.31:0/458233160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 --2- 172.21.15.31:0/458233160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c102a60 0x7fbc2c102e70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 -- 172.21.15.31:0/458233160 >> 172.21.15.31:0/458233160 conn(0x7fbc2c079730 msgr2=0x7fbc2c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 -- 172.21.15.31:0/458233160 shutdown_connections 2024-09-21T02:58:19.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 -- 172.21.15.31:0/458233160 wait complete. 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.347+0000 7fbc32235640 1 Processor -- start 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 -- start start 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc2c102a60 0x7fbc2c104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c108cf0 0x7fbc2c104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c112960 con 0x7fbc2c1083a0 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1127e0 con 0x7fbc2c102a60 2024-09-21T02:58:19.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc32235640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c112ae0 con 0x7fbc2c108cf0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59328/0 (socket says 172.21.15.31:59328) 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc2affd640 1 -- 172.21.15.31:0/457437551 learned_addr learned my addr 172.21.15.31:0/457437551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c108cf0 0x7fbc2c104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.348+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc2c102a60 0x7fbc2c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc2affd640 1 -- 172.21.15.31:0/457437551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c108cf0 msgr2=0x7fbc2c104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc2affd640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c108cf0 0x7fbc2c104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc2affd640 1 -- 172.21.15.31:0/457437551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc2c102a60 msgr2=0x7fbc2c104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc2affd640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc2c102a60 0x7fbc2c104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc2affd640 1 -- 172.21.15.31:0/457437551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c1a8700 con 0x7fbc2c1083a0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc2affd640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c104790 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ea80 tx=0x7fbc2000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc28ff9640 1 -- 172.21.15.31:0/457437551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000ce40 con 0x7fbc2c1083a0 2024-09-21T02:58:19.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc28ff9640 1 -- 172.21.15.31:0/457437551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc20011920 con 0x7fbc2c1083a0 2024-09-21T02:58:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc28ff9640 1 -- 172.21.15.31:0/457437551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2001aa20 con 0x7fbc2c1083a0 2024-09-21T02:58:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c1a89f0 con 0x7fbc2c1083a0 2024-09-21T02:58:19.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.349+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a8f80 con 0x7fbc2c1083a0 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 Processor -- start 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 -- start start 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x5625621766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562176be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562177120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5625620ecea0 con 0x562561dae390 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5625620dc2d0 con 0x562561dcf070 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc387a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5625620e00e0 con 0x562561da61b0 2024-09-21T02:58:19.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc28ff9640 1 -- 172.21.15.31:0/457437551 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbc2001abc0 con 0x7fbc2c1083a0 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562177120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x5625621766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562177120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59338/0 (socket says 172.21.15.31:59338) 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc2b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562176be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.352+0000 7fbc2affd640 1 -- 172.21.15.31:0/4250103398 learned_addr learned my addr 172.21.15.31:0/4250103398 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2affd640 1 -- 172.21.15.31:0/4250103398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 msgr2=0x562562176be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2affd640 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562176be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2affd640 1 -- 172.21.15.31:0/4250103398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 msgr2=0x5625621766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2affd640 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x5625621766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2affd640 1 -- 172.21.15.31:0/4250103398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562561e3a650 con 0x562561dae390 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x5625621766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc28ff9640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbc140b6700 0x7fbc140b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2affd640 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562177120 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbc200e53c0 tx=0x7fbc200f0820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44800e2154404566 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbc140b6700 0x7fbc140b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc11ffb640 1 -- 172.21.15.31:0/4250103398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200aa1a0 con 0x562561dae390 2024-09-21T02:58:19.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.353+0000 7fbc28ff9640 1 -- 172.21.15.31:0/457437551 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbc200e7ce0 con 0x7fbc2c1083a0 2024-09-21T02:58:19.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.354+0000 7fbc11ffb640 1 -- 172.21.15.31:0/4250103398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc200ad660 con 0x562561dae390 2024-09-21T02:58:19.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.354+0000 7fbc11ffb640 1 -- 172.21.15.31:0/4250103398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200cfb90 con 0x562561dae390 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.354+0000 7fbc387a9740 1 -- 172.21.15.31:0/4250103398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 msgr2=0x562562177120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.354+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbc140b6700 0x7fbc140b8c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009ec0 tx=0x7fbc24005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.354+0000 7fbc387a9740 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562177120 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbc200e53c0 tx=0x7fbc200f0820 comp rx=0 tx=0).stop 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 -- 172.21.15.31:0/4250103398 shutdown_connections 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562176be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562177120 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 --2- 172.21.15.31:0/4250103398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x5625621766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 -- 172.21.15.31:0/4250103398 >> 172.21.15.31:0/4250103398 conn(0x562561dc3460 msgr2=0x562562174590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 -- 172.21.15.31:0/4250103398 shutdown_connections 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 -- 172.21.15.31:0/4250103398 wait complete. 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 -- start start 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.355+0000 7fbc387a9740 1 Processor -- start 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 0x562562181cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x562562182230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561dcf070 0x56256218e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562561f3d700 con 0x562561da61b0 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562562077160 con 0x562561dae390 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562562184e80 con 0x562561dcf070 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc387a9740 10 client.0 _ll_register_callbacks cb 0x5625620909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 0x562562181cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 0x562562181cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59354/0 (socket says 172.21.15.31:59354) 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 -- 172.21.15.31:0/1168783732 learned_addr learned my addr 172.21.15.31:0/1168783732 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:19.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2affd640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x562562182230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561dcf070 0x56256218e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 -- 172.21.15.31:0/1168783732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561dcf070 msgr2=0x56256218e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561dcf070 0x56256218e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 -- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 msgr2=0x562562182230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x562562182230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.356+0000 7fbc2b7fe640 1 -- 172.21.15.31:0/1168783732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625620a4c90 con 0x562561da61b0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 0x562562181cf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c005ab0 tx=0x7fbc1c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb90bc7ec84f4de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc0affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc0affd640 1 -- 172.21.15.31:0/1168783732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c035040 con 0x562561da61b0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc200ad660 con 0x562561da61b0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc0affd640 1 -- 172.21.15.31:0/1168783732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbc1c004240 con 0x562561da61b0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc0affd640 1 -- 172.21.15.31:0/1168783732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc1c004440 con 0x562561da61b0 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc387a9740 10 client.16788 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:19.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562562190b30 con 0x562561da61b0 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc0affd640 1 -- 172.21.15.31:0/1168783732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc200ad660 con 0x562561da61b0 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.357+0000 7fbc0affd640 1 -- 172.21.15.31:0/1168783732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbc1c004aa0 con 0x562561da61b0 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc387a9740 10 client.16788 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc387a9740 10 client.16788 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc387a9740 20 client.16788 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d8d9f0 0x562562194490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc387a9740 4 client.16788 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625620eedf0 con 0x562561d8d9f0 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.358+0000 7fbc2affd640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d8d9f0 0x562562194490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.359+0000 7fbc0affd640 10 client.16788 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:58:19.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.359+0000 7fbc2affd640 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d8d9f0 0x562562194490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd5cdda56247257b server_cookie=3419f320d6eae348 in_seq=0 out_seq=0 2024-09-21T02:58:19.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.359+0000 7fbc0affd640 1 -- 172.21.15.31:0/1168783732 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625620eedf0 con 0x562561d8d9f0 2024-09-21T02:58:19.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.359+0000 7fbc0affd640 10 client.16788 handle_command_reply: tid=0 2024-09-21T02:58:19.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 client.16788 shutdown 2024-09-21T02:58:19.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 msgr2=0x562562181cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 0x562562181cf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c005ab0 tx=0x7fbc1c0316d0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 shutdown_connections 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d8d9f0 0x562562194490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561dcf070 0x56256218e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x562562182230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc0affd640 0 client.16788 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1168783732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561da61b0 0x562562181cf0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 >> 172.21.15.31:0/1168783732 conn(0x562561dc3460 msgr2=0x562562179880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 shutdown_connections 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 1 -- 172.21.15.31:0/1168783732 wait complete. 2024-09-21T02:58:19.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.360+0000 7fbc387a9740 20 client.16788 trim_cache size 0 max 16384 2024-09-21T02:58:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.373+0000 7fbc387a9740 1 Processor -- start 2024-09-21T02:58:19.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.373+0000 7fbc387a9740 1 -- start start 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x5625621821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x56256218e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 0x56256217eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc387a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b1900 con 0x562561dcf070 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc387a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b3440 con 0x562561dae390 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc387a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b1fd0 con 0x562561da61b0 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 0x56256217eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 0x56256217eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59356/0 (socket says 172.21.15.31:59356) 2024-09-21T02:58:19.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 -- 172.21.15.31:0/363967465 learned_addr learned my addr 172.21.15.31:0/363967465 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x5625621821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2affd640 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x56256218e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 -- 172.21.15.31:0/363967465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 msgr2=0x5625621821b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x5625621821b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 -- 172.21.15.31:0/363967465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 msgr2=0x56256218e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x56256218e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.374+0000 7fbc2bfff640 1 -- 172.21.15.31:0/363967465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625621746c0 con 0x562561dcf070 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 0x56256217eee0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbc240041c0 tx=0x7fbc24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01aa273e8d6c14d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc117fa640 1 -- 172.21.15.31:0/363967465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24003920 con 0x562561dcf070 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc117fa640 1 -- 172.21.15.31:0/363967465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc24005020 con 0x562561dcf070 2024-09-21T02:58:19.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc117fa640 1 -- 172.21.15.31:0/363967465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24004be0 con 0x562561dcf070 2024-09-21T02:58:19.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc387a9740 1 -- 172.21.15.31:0/363967465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 msgr2=0x56256217eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc387a9740 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 0x56256217eee0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbc240041c0 tx=0x7fbc24017000 comp rx=0 tx=0).stop 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc387a9740 1 -- 172.21.15.31:0/363967465 shutdown_connections 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc387a9740 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dcf070 0x56256217eee0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.375+0000 7fbc387a9740 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dae390 0x56256218e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 --2- 172.21.15.31:0/363967465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x5625621821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 -- 172.21.15.31:0/363967465 >> 172.21.15.31:0/363967465 conn(0x562561dc3460 msgr2=0x562562181aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 -- 172.21.15.31:0/363967465 shutdown_connections 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 -- 172.21.15.31:0/363967465 wait complete. 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 -- start start 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 Processor -- start 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562186220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562186760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x562562186ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.376+0000 7fbc387a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b2d70 con 0x562561dae390 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc387a9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1b26a0 con 0x562561dcf070 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc387a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c1af8e0 con 0x562561da61b0 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc387a9740 10 client.0 _ll_register_callbacks cb 0x7fbc1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562186760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562186760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59366/0 (socket says 172.21.15.31:59366) 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 -- 172.21.15.31:0/1797369278 learned_addr learned my addr 172.21.15.31:0/1797369278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2bfff640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x562562186ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562186220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 -- 172.21.15.31:0/1797369278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 msgr2=0x562562186220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562186220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 -- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 msgr2=0x562562186ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x562562186ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 -- 172.21.15.31:0/1797369278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625621fe010 con 0x562561dae390 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc2affd640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562186760 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbc200e5a50 tx=0x7fbc2001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a46ae0ca38007e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc0affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc0affd640 1 -- 172.21.15.31:0/1797369278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200d0c20 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc0affd640 1 -- 172.21.15.31:0/1797369278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fbc200eb600 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.377+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc24005020 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc0affd640 1 -- 172.21.15.31:0/1797369278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc200d0df0 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 10 client.16794 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625621ff290 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc0affd640 1 -- 172.21.15.31:0/1797369278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc24005020 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc0affd640 1 -- 172.21.15.31:0/1797369278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbc20022030 con 0x562561dae390 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 10 client.16794 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 10 client.16794 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 20 client.16794 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d62da0 0x562562201470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 4 client.16794 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:58:19.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625620ea8a0 con 0x562561d62da0 2024-09-21T02:58:19.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.378+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d62da0 0x562562201470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc0affd640 10 client.16794 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc2b7fe640 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d62da0 0x562562201470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29cc3e8499088df9 server_cookie=4e06299e3515384d in_seq=0 out_seq=0 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc0affd640 1 -- 172.21.15.31:0/1797369278 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625620ea8a0 con 0x562561d62da0 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc0affd640 10 client.16794 handle_command_reply: tid=0 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 client.16794 shutdown 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 msgr2=0x562562186760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562186760 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbc200e5a50 tx=0x7fbc2001af90 comp rx=0 tx=0).stop 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 shutdown_connections 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562561dcf070 0x562562186ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562561dae390 0x562562186760 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x562561d62da0 0x562562201470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 --2- 172.21.15.31:0/1797369278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562561da61b0 0x562562186220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc0affd640 0 client.16794 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:58:19.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 >> 172.21.15.31:0/1797369278 conn(0x562561dc3460 msgr2=0x5625621831a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 shutdown_connections 2024-09-21T02:58:19.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 1 -- 172.21.15.31:0/1797369278 wait complete. 2024-09-21T02:58:19.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.379+0000 7fbc387a9740 20 client.16794 trim_cache size 0 max 16384 2024-09-21T02:58:19.386 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:58:19.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbc140b6700 msgr2=0x7fbc140b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbc140b6700 0x7fbc140b8c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009ec0 tx=0x7fbc24005f90 comp rx=0 tx=0).stop 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 msgr2=0x7fbc2c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c104790 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ea80 tx=0x7fbc2000ef40 comp rx=0 tx=0).stop 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 shutdown_connections 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbc140b6700 0x7fbc140b8c10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c108cf0 0x7fbc2c104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc2c1083a0 0x7fbc2c104790 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 --2- 172.21.15.31:0/457437551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc2c102a60 0x7fbc2c104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 >> 172.21.15.31:0/457437551 conn(0x7fbc2c079730 msgr2=0x7fbc2c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 shutdown_connections 2024-09-21T02:58:19.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:19.380+0000 7fbc32235640 1 -- 172.21.15.31:0/457437551 wait complete. 2024-09-21T02:58:20.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:58:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:20.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:20 smithi031 ceph-mon[31209]: pgmap v488: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:20.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:20 smithi165 ceph-mon[34818]: pgmap v488: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:20.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:20 smithi178 ceph-mon[34759]: pgmap v488: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 -- 172.21.15.31:0/3137495574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 msgr2=0x7f8488110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 --2- 172.21.15.31:0/3137495574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f8488110300 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ca0 tx=0x7f848002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 -- 172.21.15.31:0/3137495574 shutdown_connections 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 --2- 172.21.15.31:0/3137495574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8488105270 0x7f848810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 --2- 172.21.15.31:0/3137495574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f8488110300 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 --2- 172.21.15.31:0/3137495574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8488104840 0x7f8488104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 -- 172.21.15.31:0/3137495574 >> 172.21.15.31:0/3137495574 conn(0x7f8488079730 msgr2=0x7f8488079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 -- 172.21.15.31:0/3137495574 shutdown_connections 2024-09-21T02:58:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.819+0000 7f84905b9640 1 -- 172.21.15.31:0/3137495574 wait complete. 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 Processor -- start 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 -- start start 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8488104840 0x7f848819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8488105270 0x7f848819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f848819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8488113a90 con 0x7f848810ddf0 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8488113910 con 0x7f8488105270 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f84905b9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8488113c10 con 0x7f8488104840 2024-09-21T02:58:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f848eb56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f848819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f848eb56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f848819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59392/0 (socket says 172.21.15.31:59392) 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.820+0000 7f848eb56640 1 -- 172.21.15.31:0/1668856628 learned_addr learned my addr 172.21.15.31:0/1668856628 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848e355640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8488104840 0x7f848819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848db54640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8488105270 0x7f848819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848eb56640 1 -- 172.21.15.31:0/1668856628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8488104840 msgr2=0x7f848819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848eb56640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8488104840 0x7f848819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848eb56640 1 -- 172.21.15.31:0/1668856628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8488105270 msgr2=0x7f848819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848eb56640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8488105270 0x7f848819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848eb56640 1 -- 172.21.15.31:0/1668856628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84881a0390 con 0x7f848810ddf0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f848eb56640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f848819fc60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8480005cc0 tx=0x7f8480031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f847f7fe640 1 -- 172.21.15.31:0/1668856628 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480003b70 con 0x7f848810ddf0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f847f7fe640 1 -- 172.21.15.31:0/1668856628 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84800318f0 con 0x7f848810ddf0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84881a0620 con 0x7f848810ddf0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f847f7fe640 1 -- 172.21.15.31:0/1668856628 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848002f950 con 0x7f848810ddf0 2024-09-21T02:58:20.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.821+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84881a6de0 con 0x7f848810ddf0 2024-09-21T02:58:20.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.824+0000 7f8496b2d740 1 Processor -- start 2024-09-21T02:58:20.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.825+0000 7f847f7fe640 1 -- 172.21.15.31:0/1668856628 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f848002fb10 con 0x7f848810ddf0 2024-09-21T02:58:20.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.825+0000 7f8496b2d740 1 -- start start 2024-09-21T02:58:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.825+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bc08070 0x55c67bfaf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.825+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 0x55c67bfaf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbe7390 0x55c67bfafd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f8496b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c67bf25ea0 con 0x55c67bbdf1b0 2024-09-21T02:58:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f8496b2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c67bf152d0 con 0x55c67bc08070 2024-09-21T02:58:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f8496b2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c67bf190e0 con 0x55c67bbe7390 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 0x55c67bfaf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 0x55c67bfaf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59394/0 (socket says 172.21.15.31:59394) 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 -- 172.21.15.31:0/2850010965 learned_addr learned my addr 172.21.15.31:0/2850010965 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848db54640 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bc08070 0x55c67bfaf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848db54640 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbe7390 0x55c67bfafd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f847f7fe640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84740b6680 0x7f84740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 -- 172.21.15.31:0/2850010965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbe7390 msgr2=0x55c67bfafd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbe7390 0x55c67bfafd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 -- 172.21.15.31:0/2850010965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bc08070 msgr2=0x55c67bfaf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bc08070 0x55c67bfaf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 -- 172.21.15.31:0/2850010965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67bc73650 con 0x55c67bbdf1b0 2024-09-21T02:58:20.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848db54640 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bc08070 0x55c67bfaf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848eb56640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84740b6680 0x7f84740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f848e355640 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 0x55c67bfaf800 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8484009ca0 tx=0x7f8484011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b1648f69bde18e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.826+0000 7f847f7fe640 1 -- 172.21.15.31:0/1668856628 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8480108a20 con 0x7f848810ddf0 2024-09-21T02:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.827+0000 7f847cff9640 1 -- 172.21.15.31:0/2850010965 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484004220 con 0x55c67bbdf1b0 2024-09-21T02:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.827+0000 7f847cff9640 1 -- 172.21.15.31:0/2850010965 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84840043c0 con 0x55c67bbdf1b0 2024-09-21T02:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.827+0000 7f847cff9640 1 -- 172.21.15.31:0/2850010965 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84840197a0 con 0x55c67bbdf1b0 2024-09-21T02:58:20.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 -- 172.21.15.31:0/2850010965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 msgr2=0x55c67bfaf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f848eb56640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84740b6680 0x7f84740b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f84800ce530 tx=0x7f8480097460 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 0x55c67bfaf800 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8484009ca0 tx=0x7f8484011810 comp rx=0 tx=0).stop 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 -- 172.21.15.31:0/2850010965 shutdown_connections 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbe7390 0x55c67bfafd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bbdf1b0 0x55c67bfaf800 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 --2- 172.21.15.31:0/2850010965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bc08070 0x55c67bfaf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 -- 172.21.15.31:0/2850010965 >> 172.21.15.31:0/2850010965 conn(0x55c67bbfc460 msgr2=0x55c67bfad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 -- 172.21.15.31:0/2850010965 shutdown_connections 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 -- 172.21.15.31:0/2850010965 wait complete. 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 -- start start 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.828+0000 7f8496b2d740 1 Processor -- start 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfbaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfcb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c67bd76700 con 0x55c67bc08070 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c67beb0160 con 0x55c67bbe7390 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c67bfc1dc0 con 0x55c67bbdf1b0 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8496b2d740 10 client.0 _ll_register_callbacks cb 0x55c67bec99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfcb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfcb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59406/0 (socket says 172.21.15.31:59406) 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 -- 172.21.15.31:0/384325088 learned_addr learned my addr 172.21.15.31:0/384325088 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848e355640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfbaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 -- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 msgr2=0x55c67bfba9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfba9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 -- 172.21.15.31:0/384325088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 msgr2=0x55c67bfbaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfbaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 -- 172.21.15.31:0/384325088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67beddc90 con 0x55c67bc08070 2024-09-21T02:58:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848e355640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfbaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f848db54640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfcb180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8478005890 tx=0x7f8478009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b6f6773105d1c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.829+0000 7f8471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8471ffb640 1 -- 172.21.15.31:0/384325088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478004030 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8471ffb640 1 -- 172.21.15.31:0/384325088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f84780041d0 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84840043c0 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8471ffb640 1 -- 172.21.15.31:0/384325088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84780043d0 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 10 client.16806 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c67bfcdb70 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8471ffb640 1 -- 172.21.15.31:0/384325088 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84840043c0 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8471ffb640 1 -- 172.21.15.31:0/384325088 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8478026020 con 0x55c67bc08070 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 10 client.16806 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 10 client.16806 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 20 client.16806 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bbc69f0 0x55c67bfd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 4 client.16806 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.830+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c67bf27df0 con 0x55c67bbc69f0 2024-09-21T02:58:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.831+0000 7f848e355640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bbc69f0 0x55c67bfd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.831+0000 7f848e355640 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bbc69f0 0x55c67bfd10a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94f6d220dce204e2 server_cookie=e2cffe2d7f347c71 in_seq=0 out_seq=0 2024-09-21T02:58:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.831+0000 7f8471ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8471ffb640 1 -- 172.21.15.31:0/384325088 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c67bf27df0 con 0x55c67bbc69f0 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8471ffb640 10 client.16806 handle_command_reply: tid=0 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 client.16806 shutdown 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 msgr2=0x55c67bfcb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfcb180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8478005890 tx=0x7f8478009ac0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 shutdown_connections 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bbc69f0 0x55c67bfd10a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfcb180 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfbaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 --2- 172.21.15.31:0/384325088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfba9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 >> 172.21.15.31:0/384325088 conn(0x55c67bbfc460 msgr2=0x55c67bfb2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8471ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:58:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 shutdown_connections 2024-09-21T02:58:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 1 -- 172.21.15.31:0/384325088 wait complete. 2024-09-21T02:58:20.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.832+0000 7f8496b2d740 20 client.16806 trim_cache size 0 max 16384 2024-09-21T02:58:20.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.845+0000 7f8496b2d740 1 Processor -- start 2024-09-21T02:58:20.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 -- start start 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfb98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfb2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfb7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f84881ad500 con 0x55c67bc08070 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84881af040 con 0x55c67bbe7390 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f8496b2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84881adbd0 con 0x55c67bbdf1b0 2024-09-21T02:58:20.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfb7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfb7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59418/0 (socket says 172.21.15.31:59418) 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 -- 172.21.15.31:0/837134831 learned_addr learned my addr 172.21.15.31:0/837134831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfb98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848e355640 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfb2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 -- 172.21.15.31:0/837134831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 msgr2=0x55c67bfb98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfb98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 -- 172.21.15.31:0/837134831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 msgr2=0x55c67bfb2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfb2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848db54640 1 -- 172.21.15.31:0/837134831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67bfa85d0 con 0x55c67bc08070 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.846+0000 7f848e355640 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfb2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f848db54640 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfb7ba0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8478014010 tx=0x7f8478005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c51bc1562e6db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f8473fff640 1 -- 172.21.15.31:0/837134831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84780035f0 con 0x55c67bc08070 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f8473fff640 1 -- 172.21.15.31:0/837134831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8478005670 con 0x55c67bc08070 2024-09-21T02:58:20.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f8473fff640 1 -- 172.21.15.31:0/837134831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478003b60 con 0x55c67bc08070 2024-09-21T02:58:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f8496b2d740 1 -- 172.21.15.31:0/837134831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 msgr2=0x55c67bfb7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f8496b2d740 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfb7ba0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8478014010 tx=0x7f8478005f90 comp rx=0 tx=0).stop 2024-09-21T02:58:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.847+0000 7f8496b2d740 1 -- 172.21.15.31:0/837134831 shutdown_connections 2024-09-21T02:58:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfb7ba0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfb2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 --2- 172.21.15.31:0/837134831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfb98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- 172.21.15.31:0/837134831 >> 172.21.15.31:0/837134831 conn(0x55c67bbfc460 msgr2=0x55c67bfb23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- 172.21.15.31:0/837134831 shutdown_connections 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- 172.21.15.31:0/837134831 wait complete. 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- start start 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 Processor -- start 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfc3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f84881ae970 con 0x55c67bc08070 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84881ae2a0 con 0x55c67bbe7390 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84881ab500 con 0x55c67bbdf1b0 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.848+0000 7f8496b2d740 10 client.0 _ll_register_callbacks cb 0x7f8478009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfc3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59420/0 (socket says 172.21.15.31:59420) 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 -- 172.21.15.31:0/782456840 learned_addr learned my addr 172.21.15.31:0/782456840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848e355640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfc3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 -- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 msgr2=0x55c67bfc7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfc7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 -- 172.21.15.31:0/782456840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 msgr2=0x55c67bfc3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfc3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 -- 172.21.15.31:0/782456840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67c03aa60 con 0x55c67bc08070 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f848db54640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfc3770 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8478012a80 tx=0x7f8478037040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0e8c17ade9d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8471ffb640 1 -- 172.21.15.31:0/782456840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478003a80 con 0x55c67bc08070 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8471ffb640 1 -- 172.21.15.31:0/782456840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f847803e040 con 0x55c67bc08070 2024-09-21T02:58:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8478005670 con 0x55c67bc08070 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8471ffb640 1 -- 172.21.15.31:0/782456840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8478003c20 con 0x55c67bc08070 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8496b2d740 10 client.16812 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.849+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c67c03bb20 con 0x55c67bc08070 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8471ffb640 1 -- 172.21.15.31:0/782456840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8478005670 con 0x55c67bc08070 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8471ffb640 1 -- 172.21.15.31:0/782456840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f84780207f0 con 0x55c67bc08070 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8496b2d740 10 client.16812 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8496b2d740 10 client.16812 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8496b2d740 20 client.16812 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8496b2d740 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bb9bda0 0x55c67c03e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8496b2d740 4 client.16812 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c67bf238a0 con 0x55c67bb9bda0 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.850+0000 7f848e355640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bb9bda0 0x55c67c03e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f848e355640 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bb9bda0 0x55c67c03e1f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b4039a83e134868 server_cookie=c2ef3381c0838060 in_seq=0 out_seq=0 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8471ffb640 10 client.16812 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:58:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8471ffb640 1 -- 172.21.15.31:0/782456840 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c67bf238a0 con 0x55c67bb9bda0 2024-09-21T02:58:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8471ffb640 10 client.16812 handle_command_reply: tid=0 2024-09-21T02:58:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 client.16812 shutdown 2024-09-21T02:58:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 msgr2=0x55c67bfc3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfc3770 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8478012a80 tx=0x7f8478037040 comp rx=0 tx=0).stop 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 shutdown_connections 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55c67bb9bda0 0x55c67c03e1f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c67bc08070 0x55c67bfc3770 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8471ffb640 0 client.16812 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c67bbe7390 0x55c67bfc3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 --2- 172.21.15.31:0/782456840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c67bbdf1b0 0x55c67bfc7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 >> 172.21.15.31:0/782456840 conn(0x55c67bbfc460 msgr2=0x55c67bfb02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 shutdown_connections 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.851+0000 7f8496b2d740 1 -- 172.21.15.31:0/782456840 wait complete. 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f8496b2d740 20 client.16812 trim_cache size 0 max 16384 2024-09-21T02:58:20.858 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:58:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84740b6680 msgr2=0x7f84740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84740b6680 0x7f84740b8b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f84800ce530 tx=0x7f8480097460 comp rx=0 tx=0).stop 2024-09-21T02:58:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 msgr2=0x7f848819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f848819fc60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8480005cc0 tx=0x7f8480031700 comp rx=0 tx=0).stop 2024-09-21T02:58:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 shutdown_connections 2024-09-21T02:58:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f84740b6680 0x7f84740b8b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848810ddf0 0x7f848819fc60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8488105270 0x7f848819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 --2- 172.21.15.31:0/1668856628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8488104840 0x7f848819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 >> 172.21.15.31:0/1668856628 conn(0x7f8488079730 msgr2=0x7f8488100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 shutdown_connections 2024-09-21T02:58:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:20.852+0000 7f84905b9640 1 -- 172.21.15.31:0/1668856628 wait complete. 2024-09-21T02:58:21.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:58:21.830 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:22.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 -- 172.21.15.31:0/1791964560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 msgr2=0x7f5ccc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/1791964560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc110640 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009ca0 tx=0x7f5cbc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 -- 172.21.15.31:0/1791964560 shutdown_connections 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/1791964560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 0x7f5ccc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/1791964560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc110640 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/1791964560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ccc103b80 0x7f5ccc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 -- 172.21.15.31:0/1791964560 >> 172.21.15.31:0/1791964560 conn(0x7f5ccc0ff610 msgr2=0x7f5ccc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 -- 172.21.15.31:0/1791964560 shutdown_connections 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.357+0000 7f5cd0e47640 1 -- 172.21.15.31:0/1791964560 wait complete. 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.358+0000 7f5cd0e47640 1 Processor -- start 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.358+0000 7f5cd0e47640 1 -- start start 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cd0e47640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ccc103b80 0x7f5ccc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cd0e47640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 0x7f5ccc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cd0e47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cd0e47640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc113ad0 con 0x7f5ccc109e10 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cd0e47640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc113950 con 0x7f5ccc103b80 2024-09-21T02:58:22.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cd0e47640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc113c50 con 0x7f5ccc1094c0 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 0x7f5ccc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ccc103b80 0x7f5ccc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59434/0 (socket says 172.21.15.31:59434) 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 0x7f5ccc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:44660/0 (socket says 172.21.15.31:44660) 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 -- 172.21.15.31:0/975097143 learned_addr learned my addr 172.21.15.31:0/975097143 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 -- 172.21.15.31:0/975097143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 msgr2=0x7f5ccc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 0x7f5ccc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 -- 172.21.15.31:0/975097143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ccc103b80 msgr2=0x7f5ccc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ccc103b80 0x7f5ccc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 -- 172.21.15.31:0/975097143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc1a03b0 con 0x7f5ccc109e10 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.359+0000 7f5ccad9d640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc19fc80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc005cc0 tx=0x7f5cbc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.360+0000 7f5cc37fe640 1 -- 172.21.15.31:0/975097143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc003bf0 con 0x7f5ccc109e10 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.360+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc1a0640 con 0x7f5ccc109e10 2024-09-21T02:58:22.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.360+0000 7f5cc37fe640 1 -- 172.21.15.31:0/975097143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cbc003d90 con 0x7f5ccc109e10 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.360+0000 7f5cc37fe640 1 -- 172.21.15.31:0/975097143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc02fbc0 con 0x7f5ccc109e10 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.360+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a6e00 con 0x7f5ccc109e10 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.362+0000 7f5cd73bb740 1 Processor -- start 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cc37fe640 1 -- 172.21.15.31:0/975097143 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5cbc02fd60 con 0x7f5ccc109e10 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 -- start start 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa337be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa338120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5644fa2adea0 con 0x5644f9f6f390 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644fa29d2d0 con 0x5644f9f671b0 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644fa2a10e0 con 0x5644f9f90070 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa338120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa338120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59446/0 (socket says 172.21.15.31:59446) 2024-09-21T02:58:22.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cc9d9b640 1 -- 172.21.15.31:0/221785901 learned_addr learned my addr 172.21.15.31:0/221785901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.363+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cca59c640 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa337be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc37fe640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5cb80b6680 0x7f5cb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc9d9b640 1 -- 172.21.15.31:0/221785901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 msgr2=0x5644fa3376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3376a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc9d9b640 1 -- 172.21.15.31:0/221785901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 msgr2=0x5644fa337be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa337be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc9d9b640 1 -- 172.21.15.31:0/221785901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f9ffb650 con 0x5644f9f6f390 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5ccad9d640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5cb80b6680 0x7f5cb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc37fe640 1 -- 172.21.15.31:0/975097143 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5cbc108b20 con 0x7f5ccc109e10 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa338120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005d00 tx=0x7f5cb4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fab1564f373f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc0ff9640 1 -- 172.21.15.31:0/221785901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4012ca0 con 0x5644f9f6f390 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc0ff9640 1 -- 172.21.15.31:0/221785901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb4012e40 con 0x5644f9f6f390 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.364+0000 7f5cc0ff9640 1 -- 172.21.15.31:0/221785901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40049e0 con 0x5644f9f6f390 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 -- 172.21.15.31:0/221785901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 msgr2=0x5644fa338120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa338120 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005d00 tx=0x7f5cb4005c90 comp rx=0 tx=0).stop 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5ccad9d640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5cb80b6680 0x7f5cb80b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc106340 tx=0x7f5cbc09b750 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 -- 172.21.15.31:0/221785901 shutdown_connections 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa338120 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa337be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/221785901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 -- 172.21.15.31:0/221785901 >> 172.21.15.31:0/221785901 conn(0x5644f9f84460 msgr2=0x5644fa335590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 -- 172.21.15.31:0/221785901 shutdown_connections 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.365+0000 7f5cd73bb740 1 -- 172.21.15.31:0/221785901 wait complete. 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 -- start start 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 Processor -- start 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa342d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa343280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa34f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5644fa0fe700 con 0x5644f9f6f390 2024-09-21T02:58:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644fa238160 con 0x5644f9f671b0 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644fa345e80 con 0x5644f9f90070 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cd73bb740 10 client.0 _ll_register_callbacks cb 0x5644fa2519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa343280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa343280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59454/0 (socket says 172.21.15.31:59454) 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.366+0000 7f5cca59c640 1 -- 172.21.15.31:0/3018060922 learned_addr learned my addr 172.21.15.31:0/3018060922 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa34f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa342d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cca59c640 1 -- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 msgr2=0x5644fa34f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cca59c640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa34f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cca59c640 1 -- 172.21.15.31:0/3018060922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 msgr2=0x5644fa342d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cca59c640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa342d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cca59c640 1 -- 172.21.15.31:0/3018060922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fa265c90 con 0x5644f9f6f390 2024-09-21T02:58:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa342d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cca59c640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa343280 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bad0 tx=0x7f5cc400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f78328e1abf82a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/3018060922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4005020 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/3018060922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5cc40051c0 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644fa2afdf0 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/3018060922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5cc40053c0 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cd73bb740 10 client.16824 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.367+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644fa351c00 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/3018060922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5644fa2afdf0 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/3018060922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5cc4022020 con 0x5644f9f6f390 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cd73bb740 10 client.16824 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cd73bb740 10 client.16824 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cd73bb740 20 client.16824 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f4e9f0 0x5644fa355060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cd73bb740 4 client.16824 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644fa2ab8a0 con 0x5644f9f4e9f0 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.368+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f4e9f0 0x5644fa355060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.369+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f4e9f0 0x5644fa355060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6584799fa28976fb server_cookie=d5c66a67d3d165d5 in_seq=0 out_seq=0 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.369+0000 7f5cb1ffb640 10 client.16824 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.369+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/3018060922 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644fa2ab8a0 con 0x5644f9f4e9f0 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.369+0000 7f5cb1ffb640 10 client.16824 handle_command_reply: tid=0 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.369+0000 7f5cd73bb740 1 client.16824 shutdown 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 msgr2=0x5644fa343280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa343280 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bad0 tx=0x7f5cc400bed0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 shutdown_connections 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f4e9f0 0x5644fa355060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa34f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa343280 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/3018060922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa342d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 >> 172.21.15.31:0/3018060922 conn(0x5644f9f84460 msgr2=0x5644fa33a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cb1ffb640 0 client.16824 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:58:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 shutdown_connections 2024-09-21T02:58:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 1 -- 172.21.15.31:0/3018060922 wait complete. 2024-09-21T02:58:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.370+0000 7f5cd73bb740 20 client.16824 trim_cache size 0 max 16384 2024-09-21T02:58:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 Processor -- start 2024-09-21T02:58:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 -- start start 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa343200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa34f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa33ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1af6b0 con 0x5644f9f6f390 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1b11f0 con 0x5644f9f671b0 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1afd80 con 0x5644f9f90070 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.383+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa34f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa34f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59458/0 (socket says 172.21.15.31:59458) 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 -- 172.21.15.31:0/2850154189 learned_addr learned my addr 172.21.15.31:0/2850154189 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa33ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa343200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 -- 172.21.15.31:0/2850154189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 msgr2=0x5644fa33ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa33ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 -- 172.21.15.31:0/2850154189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 msgr2=0x5644fa343200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa343200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 -- 172.21.15.31:0/2850154189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fa343de0 con 0x5644f9f6f390 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa343200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cca59c640 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa34f0a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5cc401b940 tx=0x7f5cc4004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6f43a5a5c7f24b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cb3fff640 1 -- 172.21.15.31:0/2850154189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc401b470 con 0x5644f9f6f390 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cb3fff640 1 -- 172.21.15.31:0/2850154189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc4005560 con 0x5644f9f6f390 2024-09-21T02:58:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.384+0000 7f5cb3fff640 1 -- 172.21.15.31:0/2850154189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4024460 con 0x5644f9f6f390 2024-09-21T02:58:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 -- 172.21.15.31:0/2850154189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 msgr2=0x5644fa34f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa34f0a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5cc401b940 tx=0x7f5cc4004460 comp rx=0 tx=0).stop 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 -- 172.21.15.31:0/2850154189 shutdown_connections 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa33ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa34f0a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/2850154189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa343200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 -- 172.21.15.31:0/2850154189 >> 172.21.15.31:0/2850154189 conn(0x5644f9f84460 msgr2=0x5644fa342af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 -- 172.21.15.31:0/2850154189 shutdown_connections 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.385+0000 7f5cd73bb740 1 -- 172.21.15.31:0/2850154189 wait complete. 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 -- start start 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 Processor -- start 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa346b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa347100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1b0b20 con 0x5644f9f6f390 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1b0450 con 0x5644f9f671b0 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1ad690 con 0x5644f9f90070 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cd73bb740 10 client.0 _ll_register_callbacks cb 0x7f5cc40040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa347100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa347100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59472/0 (socket says 172.21.15.31:59472) 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cca59c640 1 -- 172.21.15.31:0/38767808 learned_addr learned my addr 172.21.15.31:0/38767808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.386+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa346b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cca59c640 1 -- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 msgr2=0x5644fa3b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cca59c640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cca59c640 1 -- 172.21.15.31:0/38767808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 msgr2=0x5644fa346b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cca59c640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa346b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cca59c640 1 -- 172.21.15.31:0/38767808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fa3b8230 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa346b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cca59c640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa347100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bad0 tx=0x7f5cc401b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d284265e7c5b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/38767808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40050e0 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/38767808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5cc4017040 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc4005560 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/38767808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5cc4004db0 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cd73bb740 10 client.16830 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644fa3bf4e0 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.387+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/38767808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc4005560 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/38767808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5cc403d020 con 0x5644f9f6f390 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cd73bb740 10 client.16830 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cd73bb740 10 client.16830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cd73bb740 20 client.16830 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:22.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f23da0 0x5644fa3c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cd73bb740 4 client.16830 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:58:22.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644fa2a8cd0 con 0x5644f9f23da0 2024-09-21T02:58:22.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f23da0 0x5644fa3c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cb1ffb640 10 client.16830 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:58:22.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.388+0000 7f5cc9d9b640 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f23da0 0x5644fa3c1bd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89bc7415bfe2e78a server_cookie=5a24048e7cbf9fc0 in_seq=0 out_seq=0 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cb1ffb640 1 -- 172.21.15.31:0/38767808 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644fa2a8cd0 con 0x5644f9f23da0 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cb1ffb640 10 client.16830 handle_command_reply: tid=0 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 client.16830 shutdown 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 msgr2=0x5644fa347100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa347100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400bad0 tx=0x7f5cc401b4e0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 shutdown_connections 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5644f9f23da0 0x5644fa3c1bd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f9f90070 0x5644fa3b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5644f9f6f390 0x5644fa347100 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 --2- 172.21.15.31:0/38767808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f9f671b0 0x5644fa346b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 >> 172.21.15.31:0/38767808 conn(0x5644f9f84460 msgr2=0x5644fa3386d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cb1ffb640 0 client.16830 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:58:22.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 shutdown_connections 2024-09-21T02:58:22.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 1 -- 172.21.15.31:0/38767808 wait complete. 2024-09-21T02:58:22.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.389+0000 7f5cd73bb740 20 client.16830 trim_cache size 0 max 16384 2024-09-21T02:58:22.397 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5cb80b6680 msgr2=0x7f5cb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5cb80b6680 0x7f5cb80b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc106340 tx=0x7f5cbc09b750 comp rx=0 tx=0).stop 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 msgr2=0x7f5ccc19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc19fc80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc005cc0 tx=0x7f5cbc02f380 comp rx=0 tx=0).stop 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 shutdown_connections 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5cb80b6680 0x7f5cb80b8b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ccc1094c0 0x7f5ccc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ccc109e10 0x7f5ccc19fc80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 --2- 172.21.15.31:0/975097143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ccc103b80 0x7f5ccc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 >> 172.21.15.31:0/975097143 conn(0x7f5ccc0ff610 msgr2=0x7f5ccc108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 shutdown_connections 2024-09-21T02:58:22.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:22.390+0000 7f5cd0e47640 1 -- 172.21.15.31:0/975097143 wait complete. 2024-09-21T02:58:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:22 smithi165 ceph-mon[34818]: pgmap v489: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:22 smithi178 ceph-mon[34759]: pgmap v489: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:23.051 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:22 smithi031 ceph-mon[31209]: pgmap v489: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:23.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:58:23.336 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:23.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 -- 172.21.15.31:0/2684069054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 msgr2=0x7fc09c10ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 --2- 172.21.15.31:0/2684069054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c10ecf0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc094009ca0 tx=0x7fc09402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 -- 172.21.15.31:0/2684069054 shutdown_connections 2024-09-21T02:58:23.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 --2- 172.21.15.31:0/2684069054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c10ecf0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 --2- 172.21.15.31:0/2684069054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 0x7fc09c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 --2- 172.21.15.31:0/2684069054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc09c102230 0x7fc09c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 -- 172.21.15.31:0/2684069054 >> 172.21.15.31:0/2684069054 conn(0x7fc09c079e40 msgr2=0x7fc09c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 -- 172.21.15.31:0/2684069054 shutdown_connections 2024-09-21T02:58:23.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.813+0000 7fc0a400f640 1 -- 172.21.15.31:0/2684069054 wait complete. 2024-09-21T02:58:23.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 Processor -- start 2024-09-21T02:58:23.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 -- start start 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc09c102230 0x7fc09c0fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 0x7fc09c0ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c0ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c111900 con 0x7fc09c1084c0 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c111780 con 0x7fc09c107b70 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a400f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c111a80 con 0x7fc09c102230 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a25ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c0ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a25ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c0ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59482/0 (socket says 172.21.15.31:59482) 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a25ac640 1 -- 172.21.15.31:0/3833977324 learned_addr learned my addr 172.21.15.31:0/3833977324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc09c102230 0x7fc09c0fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.814+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 0x7fc09c0ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a25ac640 1 -- 172.21.15.31:0/3833977324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc09c102230 msgr2=0x7fc09c0fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a25ac640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc09c102230 0x7fc09c0fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a25ac640 1 -- 172.21.15.31:0/3833977324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 msgr2=0x7fc09c0ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a25ac640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 0x7fc09c0ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a25ac640 1 -- 172.21.15.31:0/3833977324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c1a8680 con 0x7fc09c1084c0 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 0x7fc09c0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a25ac640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c0ff8f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098d0 tx=0x7fc0940317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc092ffd640 1 -- 172.21.15.31:0/3833977324 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc094003b70 con 0x7fc09c1084c0 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc092ffd640 1 -- 172.21.15.31:0/3833977324 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc094003d10 con 0x7fc09c1084c0 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c1a8910 con 0x7fc09c1084c0 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc092ffd640 1 -- 172.21.15.31:0/3833977324 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09402fa20 con 0x7fc09c1084c0 2024-09-21T02:58:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.815+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c1a8ea0 con 0x7fc09c1084c0 2024-09-21T02:58:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.818+0000 7fc0aa583740 1 Processor -- start 2024-09-21T02:58:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.818+0000 7fc092ffd640 1 -- 172.21.15.31:0/3833977324 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc09402fc20 con 0x7fc09c1084c0 2024-09-21T02:58:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.818+0000 7fc0aa583740 1 -- start start 2024-09-21T02:58:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.818+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.818+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.818+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0aa583740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560a6dba2ea0 con 0x560a6d864390 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0aa583740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560a6db922d0 con 0x560a6d885070 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0aa583740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560a6db960e0 con 0x560a6d85c1b0 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a15aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a15aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc2d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59496/0 (socket says 172.21.15.31:59496) 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a15aa640 1 -- 172.21.15.31:0/3091824372 learned_addr learned my addr 172.21.15.31:0/3091824372 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc092ffd640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0880b66e0 0x7fc0880b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3091824372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 msgr2=0x560a6dc2c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc2c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3091824372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 msgr2=0x560a6dc2d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc2d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3091824372 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6d8f0650 con 0x560a6d85c1b0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc092ffd640 1 -- 172.21.15.31:0/3833977324 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc094108b60 con 0x7fc09c1084c0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.819+0000 7fc0a25ac640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0880b66e0 0x7fc0880b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.820+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc2cbe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc098009bd0 tx=0x7fc098011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9498289e50c73d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.820+0000 7fc087fff640 1 -- 172.21.15.31:0/3091824372 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098004230 con 0x560a6d85c1b0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.820+0000 7fc087fff640 1 -- 172.21.15.31:0/3091824372 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc098009e40 con 0x560a6d85c1b0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.820+0000 7fc087fff640 1 -- 172.21.15.31:0/3091824372 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098004be0 con 0x560a6d85c1b0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 -- 172.21.15.31:0/3091824372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 msgr2=0x560a6dc2cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0a25ac640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0880b66e0 0x7fc0880b8bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc094106380 tx=0x7fc09402a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc2cbe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc098009bd0 tx=0x7fc098011810 comp rx=0 tx=0).stop 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 -- 172.21.15.31:0/3091824372 shutdown_connections 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc2cbe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3091824372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 -- 172.21.15.31:0/3091824372 >> 172.21.15.31:0/3091824372 conn(0x560a6d879460 msgr2=0x560a6dc2a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 -- 172.21.15.31:0/3091824372 shutdown_connections 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 -- 172.21.15.31:0/3091824372 wait complete. 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 -- start start 2024-09-21T02:58:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.821+0000 7fc0aa583740 1 Processor -- start 2024-09-21T02:58:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d85c1b0 0x560a6dc37d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc38250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 0x560a6dc441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560a6d9f3700 con 0x560a6d885070 2024-09-21T02:58:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560a6db2d160 con 0x560a6d85c1b0 2024-09-21T02:58:23.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560a6dc3ae80 con 0x560a6d864390 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0aa583740 10 client.0 _ll_register_callbacks cb 0x560a6db469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 0x560a6dc441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 0x560a6dc441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59502/0 (socket says 172.21.15.31:59502) 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 -- 172.21.15.31:0/1044276237 learned_addr learned my addr 172.21.15.31:0/1044276237 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc38250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d85c1b0 0x560a6dc37d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 -- 172.21.15.31:0/1044276237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 msgr2=0x560a6dc38250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc38250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 -- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d85c1b0 msgr2=0x560a6dc37d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d85c1b0 0x560a6dc37d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.822+0000 7fc0a15aa640 1 -- 172.21.15.31:0/1044276237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6db5ac90 con 0x560a6d885070 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 0x560a6dc441a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc08c005de0 tx=0x7fc08c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2b87f1b72732fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:23.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0857fa640 1 -- 172.21.15.31:0/1044276237 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0131d0 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0857fa640 1 -- 172.21.15.31:0/1044276237 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc08c004030 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a6dba4df0 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0857fa640 1 -- 172.21.15.31:0/1044276237 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc08c004230 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0aa583740 10 client.16842 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a6dc46be0 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0857fa640 1 -- 172.21.15.31:0/1044276237 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560a6dba4df0 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0857fa640 1 -- 172.21.15.31:0/1044276237 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc08c027020 con 0x560a6d885070 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0aa583740 10 client.16842 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0aa583740 10 client.16842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.823+0000 7fc0aa583740 20 client.16842 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.824+0000 7fc0aa583740 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d8439f0 0x560a6dc4a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.824+0000 7fc0aa583740 4 client.16842 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:23.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.824+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6dba08a0 con 0x560a6d8439f0 2024-09-21T02:58:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.824+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d8439f0 0x560a6dc4a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.824+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d8439f0 0x560a6dc4a040 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c5ca1fdb9ac3c94 server_cookie=191f3d583f6c6081 in_seq=0 out_seq=0 2024-09-21T02:58:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.824+0000 7fc0857fa640 10 client.16842 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.825+0000 7fc0857fa640 1 -- 172.21.15.31:0/1044276237 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a6dba08a0 con 0x560a6d8439f0 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.825+0000 7fc0857fa640 10 client.16842 handle_command_reply: tid=0 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.825+0000 7fc0aa583740 1 client.16842 shutdown 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.825+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 msgr2=0x560a6dc441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.825+0000 7fc0aa583740 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 0x560a6dc441a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc08c005de0 tx=0x7fc08c015040 comp rx=0 tx=0).stop 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 shutdown_connections 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d8439f0 0x560a6dc4a040 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc38250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d885070 0x560a6dc441a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0857fa640 0 client.16842 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 --2- 172.21.15.31:0/1044276237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d85c1b0 0x560a6dc37d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 >> 172.21.15.31:0/1044276237 conn(0x560a6d879460 msgr2=0x560a6dc2f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 shutdown_connections 2024-09-21T02:58:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 1 -- 172.21.15.31:0/1044276237 wait complete. 2024-09-21T02:58:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.826+0000 7fc0aa583740 20 client.16842 trim_cache size 0 max 16384 2024-09-21T02:58:23.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 Processor -- start 2024-09-21T02:58:23.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 -- start start 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 0x560a6dc39940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc34f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc35440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b1750 con 0x560a6d85c1b0 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b3290 con 0x560a6d885070 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0aa583740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b1e20 con 0x560a6d864390 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.839+0000 7fc0a15aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 0x560a6dc39940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 0x560a6dc39940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59516/0 (socket says 172.21.15.31:59516) 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 -- 172.21.15.31:0/493415778 learned_addr learned my addr 172.21.15.31:0/493415778 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc35440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc34f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 -- 172.21.15.31:0/493415778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 msgr2=0x560a6dc34f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc34f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 -- 172.21.15.31:0/493415778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 msgr2=0x560a6dc35440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc35440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 -- 172.21.15.31:0/493415778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6dc44850 con 0x560a6d85c1b0 2024-09-21T02:58:23.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 0x560a6dc39940 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000c00 tx=0x7fc08c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ba0fc49ddcd37c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0877fe640 1 -- 172.21.15.31:0/493415778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c02f070 con 0x560a6d85c1b0 2024-09-21T02:58:23.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0877fe640 1 -- 172.21.15.31:0/493415778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c004040 con 0x560a6d85c1b0 2024-09-21T02:58:23.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.840+0000 7fc0877fe640 1 -- 172.21.15.31:0/493415778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00e6f0 con 0x560a6d85c1b0 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 -- 172.21.15.31:0/493415778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 msgr2=0x560a6dc39940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 0x560a6dc39940 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000c00 tx=0x7fc08c001e50 comp rx=0 tx=0).stop 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 -- 172.21.15.31:0/493415778 shutdown_connections 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d864390 0x560a6dc34f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc35440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 --2- 172.21.15.31:0/493415778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d85c1b0 0x560a6dc39940 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 -- 172.21.15.31:0/493415778 >> 172.21.15.31:0/493415778 conn(0x560a6d879460 msgr2=0x560a6dc39230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 -- 172.21.15.31:0/493415778 shutdown_connections 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.841+0000 7fc0aa583740 1 -- 172.21.15.31:0/493415778 wait complete. 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 -- start start 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 Processor -- start 2024-09-21T02:58:23.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc3c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc3c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc3cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b2bc0 con 0x560a6d864390 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1b24f0 con 0x560a6d885070 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1af730 con 0x560a6d85c1b0 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0aa583740 10 client.0 _ll_register_callbacks cb 0x7fc08c013d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0a1dab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc3c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0a1dab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc3c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59520/0 (socket says 172.21.15.31:59520) 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.842+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3484433378 learned_addr learned my addr 172.21.15.31:0/3484433378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc3cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc3c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3484433378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 msgr2=0x560a6dc3c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc3c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 msgr2=0x560a6dc3cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc3cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a1dab640 1 -- 172.21.15.31:0/3484433378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6dcb3bf0 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc3cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0a1dab640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc3c7c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc098009b20 tx=0x7fc098009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5feeb670e298f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0857fa640 1 -- 172.21.15.31:0/3484433378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09801e040 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0857fa640 1 -- 172.21.15.31:0/3484433378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc098004230 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08c004040 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0857fa640 1 -- 172.21.15.31:0/3484433378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc09803b050 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0aa583740 10 client.16848 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.843+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a6dcb4ed0 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0857fa640 1 -- 172.21.15.31:0/3484433378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08c004040 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0857fa640 1 -- 172.21.15.31:0/3484433378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc098018070 con 0x560a6d864390 2024-09-21T02:58:23.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0aa583740 10 client.16848 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0aa583740 10 client.16848 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:58:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0aa583740 20 client.16848 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d818da0 0x560a6dcb7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0aa583740 4 client.16848 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:58:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a6db9dcd0 con 0x560a6d818da0 2024-09-21T02:58:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.844+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d818da0 0x560a6dcb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0857fa640 10 client.16848 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:58:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0a15aa640 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d818da0 0x560a6dcb7530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9700e8611d31011 server_cookie=564d47077db0bdca in_seq=0 out_seq=0 2024-09-21T02:58:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0857fa640 1 -- 172.21.15.31:0/3484433378 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a6db9dcd0 con 0x560a6d818da0 2024-09-21T02:58:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0857fa640 10 client.16848 handle_command_reply: tid=0 2024-09-21T02:58:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 client.16848 shutdown 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 msgr2=0x560a6dc3c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc3c7c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc098009b20 tx=0x7fc098009780 comp rx=0 tx=0).stop 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 shutdown_connections 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x560a6d818da0 0x560a6dcb7530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a6d885070 0x560a6dc3cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560a6d864390 0x560a6dc3c7c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0857fa640 0 client.16848 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 --2- 172.21.15.31:0/3484433378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a6d85c1b0 0x560a6dc3c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 >> 172.21.15.31:0/3484433378 conn(0x560a6d879460 msgr2=0x560a6dc38250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 shutdown_connections 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.845+0000 7fc0aa583740 1 -- 172.21.15.31:0/3484433378 wait complete. 2024-09-21T02:58:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0aa583740 20 client.16848 trim_cache size 0 max 16384 2024-09-21T02:58:23.853 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:58:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0880b66e0 msgr2=0x7fc0880b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0880b66e0 0x7fc0880b8bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc094106380 tx=0x7fc09402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 msgr2=0x7fc09c0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c0ff8f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098d0 tx=0x7fc0940317b0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 shutdown_connections 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc0880b66e0 0x7fc0880b8bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc09c1084c0 0x7fc09c0ff8f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc09c107b70 0x7fc09c0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 --2- 172.21.15.31:0/3833977324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc09c102230 0x7fc09c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 >> 172.21.15.31:0/3833977324 conn(0x7fc09c079e40 msgr2=0x7fc09c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 shutdown_connections 2024-09-21T02:58:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:23.846+0000 7fc0a400f640 1 -- 172.21.15.31:0/3833977324 wait complete. 2024-09-21T02:58:24.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:58:24.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:24 smithi031 ceph-mon[31209]: pgmap v490: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:58:24.850 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:24.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:24 smithi165 ceph-mon[34818]: pgmap v490: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:58:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:24 smithi178 ceph-mon[34759]: pgmap v490: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3835603633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24103b60 msgr2=0x7f9f24103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3835603633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24103b60 0x7f9f24103f70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009ca0 tx=0x7f9f1402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3835603633 shutdown_connections 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3835603633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24109df0 0x7f9f24110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3835603633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f241094a0 0x7f9f241098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3835603633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24103b60 0x7f9f24103f70 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3835603633 >> 172.21.15.31:0/3835603633 conn(0x7f9f2407b850 msgr2=0x7f9f2407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3835603633 shutdown_connections 2024-09-21T02:58:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.321+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3835603633 wait complete. 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.322+0000 7f9f28ed4640 1 Processor -- start 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.322+0000 7f9f28ed4640 1 -- start start 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.322+0000 7f9f28ed4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24103b60 0x7f9f2419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.322+0000 7f9f28ed4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f241094a0 0x7f9f2419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.322+0000 7f9f28ed4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 0x7f9f2419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f28ed4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113ab0 con 0x7f9f24109df0 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f28ed4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113930 con 0x7f9f241094a0 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f28ed4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113c30 con 0x7f9f24103b60 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 0x7f9f2419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 0x7f9f2419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59538/0 (socket says 172.21.15.31:59538) 2024-09-21T02:58:25.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 -- 172.21.15.31:0/3417344734 learned_addr learned my addr 172.21.15.31:0/3417344734 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24103b60 0x7f9f2419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f241094a0 0x7f9f2419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 -- 172.21.15.31:0/3417344734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24103b60 msgr2=0x7f9f2419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24103b60 0x7f9f2419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 -- 172.21.15.31:0/3417344734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f241094a0 msgr2=0x7f9f2419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f241094a0 0x7f9f2419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 -- 172.21.15.31:0/3417344734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a03f0 con 0x7f9f24109df0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f22d9d640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 0x7f9f2419fcc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d750 tx=0x7f9f1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f137fe640 1 -- 172.21.15.31:0/3417344734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c017070 con 0x7f9f24109df0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f137fe640 1 -- 172.21.15.31:0/3417344734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c0054d0 con 0x7f9f24109df0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.323+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a06e0 con 0x7f9f24109df0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.324+0000 7f9f137fe640 1 -- 172.21.15.31:0/3417344734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004c60 con 0x7f9f24109df0 2024-09-21T02:58:25.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.324+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a6e40 con 0x7f9f24109df0 2024-09-21T02:58:25.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.326+0000 7f9f2f448740 1 Processor -- start 2024-09-21T02:58:25.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.326+0000 7f9f137fe640 1 -- 172.21.15.31:0/3417344734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f9f1c004e00 con 0x7f9f24109df0 2024-09-21T02:58:25.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 -- start start 2024-09-21T02:58:25.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56094071bea0 con 0x5609403dd390 2024-09-21T02:58:25.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56094070b2d0 con 0x5609403fe070 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2f448740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56094070f0e0 con 0x5609403d51b0 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407a6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59544/0 (socket says 172.21.15.31:59544) 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f21d9b640 1 -- 172.21.15.31:0/2264025686 learned_addr learned my addr 172.21.15.31:0/2264025686 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f2259c640 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.327+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f21d9b640 1 -- 172.21.15.31:0/2264025686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 msgr2=0x5609407a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f21d9b640 1 -- 172.21.15.31:0/2264025686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 msgr2=0x5609407a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f137fe640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f0c0b6680 0x7f9f0c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f21d9b640 1 -- 172.21.15.31:0/2264025686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560940469650 con 0x5609403dd390 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f22d9d640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f0c0b6680 0x7f9f0c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407a6120 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f18012540 tx=0x7f9f18005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0a72253f6f403d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:25.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f137fe640 1 -- 172.21.15.31:0/3417344734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9f1c0e7010 con 0x7f9f24109df0 2024-09-21T02:58:25.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f10ff9640 1 -- 172.21.15.31:0/2264025686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004230 con 0x5609403dd390 2024-09-21T02:58:25.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f10ff9640 1 -- 172.21.15.31:0/2264025686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f180043d0 con 0x5609403dd390 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.328+0000 7f9f10ff9640 1 -- 172.21.15.31:0/2264025686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004d80 con 0x5609403dd390 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 -- 172.21.15.31:0/2264025686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 msgr2=0x5609407a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f22d9d640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f0c0b6680 0x7f9f0c0b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0e4830 tx=0x7f9f1c094040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407a6120 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f18012540 tx=0x7f9f18005c90 comp rx=0 tx=0).stop 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 -- 172.21.15.31:0/2264025686 shutdown_connections 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407a6120 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 --2- 172.21.15.31:0/2264025686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 -- 172.21.15.31:0/2264025686 >> 172.21.15.31:0/2264025686 conn(0x5609403f2460 msgr2=0x5609407a3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 -- 172.21.15.31:0/2264025686 shutdown_connections 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.329+0000 7f9f2f448740 1 -- 172.21.15.31:0/2264025686 wait complete. 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 -- start start 2024-09-21T02:58:25.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 Processor -- start 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403d51b0 0x5609407b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403fe070 0x5609407c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56094056c700 con 0x5609403dd390 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5609406a6160 con 0x5609403d51b0 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5609407b81e0 con 0x5609403fe070 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2f448740 10 client.0 _ll_register_callbacks cb 0x5609406bf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59548/0 (socket says 172.21.15.31:59548) 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f2259c640 1 -- 172.21.15.31:0/750302061 learned_addr learned my addr 172.21.15.31:0/750302061 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403d51b0 0x5609407b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.330+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403fe070 0x5609407c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2259c640 1 -- 172.21.15.31:0/750302061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403fe070 msgr2=0x5609407c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2259c640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403fe070 0x5609407c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2259c640 1 -- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403d51b0 msgr2=0x5609407b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2259c640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403d51b0 0x5609407b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2259c640 1 -- 172.21.15.31:0/750302061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609406d3c90 con 0x5609403dd390 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2259c640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b56b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f1402ade0 tx=0x7f9f14009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c00e7d2505d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f09ffb640 1 -- 172.21.15.31:0/750302061 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f140317a0 con 0x5609403dd390 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f09ffb640 1 -- 172.21.15.31:0/750302061 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9f14003970 con 0x5609403dd390 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f180043d0 con 0x5609403dd390 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f09ffb640 1 -- 172.21.15.31:0/750302061 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f14003b70 con 0x5609403dd390 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2f448740 10 client.16860 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609407c3f30 con 0x5609403dd390 2024-09-21T02:58:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.331+0000 7f9f09ffb640 1 -- 172.21.15.31:0/750302061 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f180043d0 con 0x5609403dd390 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f09ffb640 1 -- 172.21.15.31:0/750302061 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9f1404c020 con 0x5609403dd390 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f2f448740 10 client.16860 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f2f448740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f2f448740 20 client.16860 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f2f448740 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5609403bc9f0 0x5609407c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f2f448740 4 client.16860 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56094071ddf0 con 0x5609403bc9f0 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.332+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5609403bc9f0 0x5609407c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.333+0000 7f9f09ffb640 10 client.16860 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.333+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5609403bc9f0 0x5609407c7490 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8722d4efe6e6c3 server_cookie=3b10605574085919 in_seq=0 out_seq=0 2024-09-21T02:58:25.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.333+0000 7f9f09ffb640 1 -- 172.21.15.31:0/750302061 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56094071ddf0 con 0x5609403bc9f0 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.333+0000 7f9f09ffb640 10 client.16860 handle_command_reply: tid=0 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 client.16860 shutdown 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 msgr2=0x5609407b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b56b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f1402ade0 tx=0x7f9f14009c20 comp rx=0 tx=0).stop 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 shutdown_connections 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5609403bc9f0 0x5609407c7490 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403fe070 0x5609407c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b56b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 --2- 172.21.15.31:0/750302061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403d51b0 0x5609407b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 >> 172.21.15.31:0/750302061 conn(0x5609403f2460 msgr2=0x5609407a7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f09ffb640 0 client.16860 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 shutdown_connections 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 1 -- 172.21.15.31:0/750302061 wait complete. 2024-09-21T02:58:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.334+0000 7f9f2f448740 20 client.16860 trim_cache size 0 max 16384 2024-09-21T02:58:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 Processor -- start 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 -- start start 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ad400 con 0x5609403dd390 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241aef40 con 0x5609403fe070 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.347+0000 7f9f2f448740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241adad0 con 0x5609403d51b0 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59562/0 (socket says 172.21.15.31:59562) 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 -- 172.21.15.31:0/4224160392 learned_addr learned my addr 172.21.15.31:0/4224160392 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 -- 172.21.15.31:0/4224160392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 msgr2=0x5609407b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 -- 172.21.15.31:0/4224160392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 msgr2=0x5609407b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 -- 172.21.15.31:0/4224160392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609407afde0 con 0x5609403dd390 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f2259c640 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b3f30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009bf0 tx=0x7f9f14003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bbe5c0d0e4f151 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f0bfff640 1 -- 172.21.15.31:0/4224160392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1402fa60 con 0x5609403dd390 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f0bfff640 1 -- 172.21.15.31:0/4224160392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f14036050 con 0x5609403dd390 2024-09-21T02:58:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.348+0000 7f9f0bfff640 1 -- 172.21.15.31:0/4224160392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14030ae0 con 0x5609403dd390 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 -- 172.21.15.31:0/4224160392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 msgr2=0x5609407b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b3f30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009bf0 tx=0x7f9f14003b00 comp rx=0 tx=0).stop 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 -- 172.21.15.31:0/4224160392 shutdown_connections 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403fe070 0x5609407b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403dd390 0x5609407b3f30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 --2- 172.21.15.31:0/4224160392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 -- 172.21.15.31:0/4224160392 >> 172.21.15.31:0/4224160392 conn(0x5609403f2460 msgr2=0x5609407b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 -- 172.21.15.31:0/4224160392 shutdown_connections 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.349+0000 7f9f2f448740 1 -- 172.21.15.31:0/4224160392 wait complete. 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 -- start start 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 Processor -- start 2024-09-21T02:58:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403dd390 0x5609407b9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 0x5609407ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ae870 con 0x5609403fe070 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ae1a0 con 0x5609403dd390 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ab3e0 con 0x5609403d51b0 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2f448740 10 client.0 _ll_register_callbacks cb 0x7f9f1402f3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 0x5609407ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f21d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 0x5609407ba010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59574/0 (socket says 172.21.15.31:59574) 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f21d9b640 1 -- 172.21.15.31:0/1527764120 learned_addr learned my addr 172.21.15.31:0/1527764120 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f2259c640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403dd390 0x5609407b9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.350+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f21d9b640 1 -- 172.21.15.31:0/1527764120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 msgr2=0x5609407b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f21d9b640 1 -- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403dd390 msgr2=0x5609407b9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403dd390 0x5609407b9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f21d9b640 1 -- 172.21.15.31:0/1527764120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560940830fc0 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f2259c640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403dd390 0x5609407b9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f21d9b640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 0x5609407ba010 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800b8a0 tx=0x7f9f14031eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bb503df637b593 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f09ffb640 1 -- 172.21.15.31:0/1527764120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18030aa0 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f09ffb640 1 -- 172.21.15.31:0/1527764120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f9f18030c40 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f14036050 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f09ffb640 1 -- 172.21.15.31:0/1527764120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f18004020 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f2f448740 10 client.16866 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560940832220 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.351+0000 7f9f09ffb640 1 -- 172.21.15.31:0/1527764120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f14036050 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f09ffb640 1 -- 172.21.15.31:0/1527764120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9f18040070 con 0x5609403fe070 2024-09-21T02:58:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2f448740 10 client.16866 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2f448740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2f448740 20 client.16866 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2f448740 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560940391da0 0x5609408346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2f448740 4 client.16866 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609407198a0 con 0x560940391da0 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.352+0000 7f9f2259c640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560940391da0 0x5609408346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2259c640 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560940391da0 0x5609408346a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a35f077eda5a9b2 server_cookie=c1250bedc9d623f in_seq=0 out_seq=0 2024-09-21T02:58:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f09ffb640 10 client.16866 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f09ffb640 1 -- 172.21.15.31:0/1527764120 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609407198a0 con 0x560940391da0 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f09ffb640 10 client.16866 handle_command_reply: tid=0 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 client.16866 shutdown 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 msgr2=0x5609407ba010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 0x5609407ba010 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800b8a0 tx=0x7f9f14031eb0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 shutdown_connections 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560940391da0 0x5609408346a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609403fe070 0x5609407ba010 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609403dd390 0x5609407b9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 --2- 172.21.15.31:0/1527764120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609403d51b0 0x5609407b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 >> 172.21.15.31:0/1527764120 conn(0x5609403f2460 msgr2=0x5609407b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f09ffb640 0 client.16866 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 shutdown_connections 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 1 -- 172.21.15.31:0/1527764120 wait complete. 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.353+0000 7f9f2f448740 20 client.16866 trim_cache size 0 max 16384 2024-09-21T02:58:25.360 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f0c0b6680 msgr2=0x7f9f0c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f0c0b6680 0x7f9f0c0b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0e4830 tx=0x7f9f1c094040 comp rx=0 tx=0).stop 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 msgr2=0x7f9f2419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 0x7f9f2419fcc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d750 tx=0x7f9f1c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 shutdown_connections 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f9f0c0b6680 0x7f9f0c0b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f24109df0 0x7f9f2419fcc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f241094a0 0x7f9f2419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 --2- 172.21.15.31:0/3417344734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24103b60 0x7f9f2419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 >> 172.21.15.31:0/3417344734 conn(0x7f9f2407b850 msgr2=0x7f9f24107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:25.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 shutdown_connections 2024-09-21T02:58:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:25.354+0000 7f9f28ed4640 1 -- 172.21.15.31:0/3417344734 wait complete. 2024-09-21T02:58:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:26 smithi165 ceph-mon[34818]: pgmap v491: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:26 smithi178 ceph-mon[34759]: pgmap v491: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:26 smithi031 ceph-mon[31209]: pgmap v491: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:27.097 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:58:27.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:58:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:27.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 -- 172.21.15.31:0/1475826569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 msgr2=0x7fdbc410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 --2- 172.21.15.31:0/1475826569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc410d8d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02a410 comp rx=0 tx=0).stop 2024-09-21T02:58:27.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 -- 172.21.15.31:0/1475826569 shutdown_connections 2024-09-21T02:58:27.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 --2- 172.21.15.31:0/1475826569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc410de60 0x7fdbc4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 --2- 172.21.15.31:0/1475826569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc410d8d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 --2- 172.21.15.31:0/1475826569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc4104a30 0x7fdbc4104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 -- 172.21.15.31:0/1475826569 >> 172.21.15.31:0/1475826569 conn(0x7fdbc4079760 msgr2=0x7fdbc4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 -- 172.21.15.31:0/1475826569 shutdown_connections 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.852+0000 7fdbca8db640 1 -- 172.21.15.31:0/1475826569 wait complete. 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 Processor -- start 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 -- start start 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc4104a30 0x7fdbc4102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc4102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc410de60 0x7fdbc41031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4113aa0 con 0x7fdbc4105380 2024-09-21T02:58:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4113920 con 0x7fdbc410de60 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbca8db640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4113c20 con 0x7fdbc4104a30 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc4102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc4102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59602/0 (socket says 172.21.15.31:59602) 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbbbfff640 1 -- 172.21.15.31:0/3521021715 learned_addr learned my addr 172.21.15.31:0/3521021715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc4104a30 0x7fdbc4102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc410de60 0x7fdbc41031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbbbfff640 1 -- 172.21.15.31:0/3521021715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc4104a30 msgr2=0x7fdbc4102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.853+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc4104a30 0x7fdbc4102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbbbfff640 1 -- 172.21.15.31:0/3521021715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc410de60 msgr2=0x7fdbc41031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc410de60 0x7fdbc41031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbbbfff640 1 -- 172.21.15.31:0/3521021715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc41a8770 con 0x7fdbc4105380 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc410de60 0x7fdbc41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc4102cb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdbac02a930 tx=0x7fdbac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbc1ffb640 1 -- 172.21.15.31:0/3521021715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac03f030 con 0x7fdbc4105380 2024-09-21T02:58:27.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc41a8a00 con 0x7fdbc4105380 2024-09-21T02:58:27.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbc1ffb640 1 -- 172.21.15.31:0/3521021715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbac003aa0 con 0x7fdbc4105380 2024-09-21T02:58:27.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbc1ffb640 1 -- 172.21.15.31:0/3521021715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac035070 con 0x7fdbc4105380 2024-09-21T02:58:27.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.854+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc41a8f30 con 0x7fdbc4105380 2024-09-21T02:58:27.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.857+0000 7fdbd0e4f740 1 Processor -- start 2024-09-21T02:58:27.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.857+0000 7fdbd0e4f740 1 -- start start 2024-09-21T02:58:27.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f818a350 0x55f4f8531430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8531970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f8531eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc1ffb640 1 -- 172.21.15.31:0/3521021715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fdbac042030 con 0x7fdbc4105380 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f4f84a7e60 con 0x55f4f8161490 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8497290 con 0x55f4f8169670 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4f849b0a0 con 0x55f4f818a350 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8531970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8531970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59612/0 (socket says 172.21.15.31:59612) 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 -- 172.21.15.31:0/4130086733 learned_addr learned my addr 172.21.15.31:0/4130086733 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f818a350 0x55f4f8531430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f8531eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 -- 172.21.15.31:0/4130086733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f818a350 msgr2=0x55f4f8531430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f818a350 0x55f4f8531430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 -- 172.21.15.31:0/4130086733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 msgr2=0x55f4f8531eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f8531eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc3fff640 1 -- 172.21.15.31:0/4130086733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f81f5930 con 0x55f4f8161490 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f818a350 0x55f4f8531430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.858+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f8531eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.859+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8531970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400bc70 tx=0x7fdbb400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5a756e703fdb01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.859+0000 7fdbba7fc640 1 -- 172.21.15.31:0/4130086733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4015070 con 0x55f4f8161490 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.859+0000 7fdbba7fc640 1 -- 172.21.15.31:0/4130086733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbb40035f0 con 0x55f4f8161490 2024-09-21T02:58:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.859+0000 7fdbba7fc640 1 -- 172.21.15.31:0/4130086733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40116a0 con 0x55f4f8161490 2024-09-21T02:58:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.859+0000 7fdbc1ffb640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdbb00b6680 0x7fdbb00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdbb00b6680 0x7fdbb00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbc1ffb640 1 -- 172.21.15.31:0/3521021715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdbac107cd0 con 0x7fdbc4105380 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4130086733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 msgr2=0x55f4f8531970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8531970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400bc70 tx=0x7fdbb400f730 comp rx=0 tx=0).stop 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4130086733 shutdown_connections 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f8531eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8531970 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4130086733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f818a350 0x55f4f8531430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4130086733 >> 172.21.15.31:0/4130086733 conn(0x55f4f817e740 msgr2=0x55f4f852f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4130086733 shutdown_connections 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4130086733 wait complete. 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.860+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdbb00b6680 0x7fdbb00b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005fd0 tx=0x7fdbbc009590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 -- start start 2024-09-21T02:58:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 Processor -- start 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8161490 0x55f4f853cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f853d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 0x55f4f8548f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f4f82f8890 con 0x55f4f818a350 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8431d40 con 0x55f4f8169670 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4f853fc10 con 0x55f4f8161490 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbd0e4f740 10 client.0 _ll_register_callbacks cb 0x55f4f8495c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbc8e78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 0x55f4f8548f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbc8e78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 0x55f4f8548f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59622/0 (socket says 172.21.15.31:59622) 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbc8e78640 1 -- 172.21.15.31:0/2824115389 learned_addr learned my addr 172.21.15.31:0/2824115389 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.861+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f853d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8161490 0x55f4f853cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc8e78640 1 -- 172.21.15.31:0/2824115389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8161490 msgr2=0x55f4f853cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8161490 0x55f4f853cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc8e78640 1 -- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 msgr2=0x55f4f853d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f853d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc8e78640 1 -- 172.21.15.31:0/2824115389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f8549490 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 0x55f4f8548f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc012010 tx=0x7fdbbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca5ed0d3047e139 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdba37fe640 1 -- 172.21.15.31:0/2824115389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00fd60 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdba37fe640 1 -- 172.21.15.31:0/2824115389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdbbc01f420 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.862+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb40035f0 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdba37fe640 1 -- 172.21.15.31:0/2824115389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbbc01f620 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 10 client.16878 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4f854be20 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdba37fe640 1 -- 172.21.15.31:0/2824115389 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbb40035f0 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdba37fe640 1 -- 172.21.15.31:0/2824115389 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdbbc02e020 con 0x55f4f818a350 2024-09-21T02:58:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 10 client.16878 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 20 client.16878 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f8148cd0 0x55f4f854f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 4 client.16878 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f84a9db0 con 0x55f4f8148cd0 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.863+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f8148cd0 0x55f4f854f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.864+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f8148cd0 0x55f4f854f300 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16863e4e34b086f3 server_cookie=521bd0f377f85bb5 in_seq=0 out_seq=0 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.864+0000 7fdba37fe640 10 client.16878 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.864+0000 7fdba37fe640 1 -- 172.21.15.31:0/2824115389 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4f84a9db0 con 0x55f4f8148cd0 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.864+0000 7fdba37fe640 10 client.16878 handle_command_reply: tid=0 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 client.16878 shutdown 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 msgr2=0x55f4f8548f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 0x55f4f8548f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc012010 tx=0x7fdbbc004a40 comp rx=0 tx=0).stop 2024-09-21T02:58:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 shutdown_connections 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f8148cd0 0x55f4f854f300 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f818a350 0x55f4f8548f00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8169670 0x55f4f853d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/2824115389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8161490 0x55f4f853cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdba37fe640 0 client.16878 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 >> 172.21.15.31:0/2824115389 conn(0x55f4f817e740 msgr2=0x55f4f85345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 shutdown_connections 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/2824115389 wait complete. 2024-09-21T02:58:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.865+0000 7fdbd0e4f740 20 client.16878 trim_cache size 0 max 16384 2024-09-21T02:58:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 Processor -- start 2024-09-21T02:58:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 -- start start 2024-09-21T02:58:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8148cd0 0x55f4f8538560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8537550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8169670 0x55f4f8532e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41b1930 con 0x55f4f8161490 2024-09-21T02:58:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41b3470 con 0x55f4f8148cd0 2024-09-21T02:58:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41b2000 con 0x55f4f8169670 2024-09-21T02:58:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8537550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8537550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59634/0 (socket says 172.21.15.31:59634) 2024-09-21T02:58:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 -- 172.21.15.31:0/4005579660 learned_addr learned my addr 172.21.15.31:0/4005579660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8148cd0 0x55f4f8538560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8169670 0x55f4f8532e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 -- 172.21.15.31:0/4005579660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8169670 msgr2=0x55f4f8532e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8169670 0x55f4f8532e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 -- 172.21.15.31:0/4005579660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8148cd0 msgr2=0x55f4f8538560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8148cd0 0x55f4f8538560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 -- 172.21.15.31:0/4005579660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f8534290 con 0x55f4f8161490 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8169670 0x55f4f8532e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8148cd0 0x55f4f8538560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.883+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8537550 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdbac108030 tx=0x7fdbac031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e58163d58d8470e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbb9ffb640 1 -- 172.21.15.31:0/4005579660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac049040 con 0x55f4f8161490 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbb9ffb640 1 -- 172.21.15.31:0/4005579660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbac0cd650 con 0x55f4f8161490 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbb9ffb640 1 -- 172.21.15.31:0/4005579660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0d7560 con 0x55f4f8161490 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4005579660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 msgr2=0x55f4f8537550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8537550 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdbac108030 tx=0x7fdbac031a10 comp rx=0 tx=0).stop 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4005579660 shutdown_connections 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8169670 0x55f4f8532e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8161490 0x55f4f8537550 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/4005579660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8148cd0 0x55f4f8538560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.884+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4005579660 >> 172.21.15.31:0/4005579660 conn(0x55f4f817e740 msgr2=0x55f4f853e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4005579660 shutdown_connections 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/4005579660 wait complete. 2024-09-21T02:58:27.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 -- start start 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 Processor -- start 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8148cd0 0x55f4f8541840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8161490 0x55f4f8541e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 0x55f4f85b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41b2da0 con 0x55f4f8169670 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41b26d0 con 0x55f4f8161490 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.885+0000 7fdbd0e4f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41af910 con 0x55f4f8148cd0 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbd0e4f740 10 client.0 _ll_register_callbacks cb 0x7fdbbc009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 0x55f4f85b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 0x55f4f85b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59640/0 (socket says 172.21.15.31:59640) 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc3fff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8148cd0 0x55f4f8541840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 -- 172.21.15.31:0/3183615511 learned_addr learned my addr 172.21.15.31:0/3183615511 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8161490 0x55f4f8541e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 -- 172.21.15.31:0/3183615511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8148cd0 msgr2=0x55f4f8541840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8148cd0 0x55f4f8541840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 -- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8161490 msgr2=0x55f4f8541e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8161490 0x55f4f8541e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 -- 172.21.15.31:0/3183615511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f85b33b0 con 0x55f4f8169670 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbbbfff640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8161490 0x55f4f8541e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8148cd0 0x55f4f8541840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbc8e78640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 0x55f4f85b2d40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0307b0 tx=0x7fdbbc012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24214335773fc05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdba37fe640 1 -- 172.21.15.31:0/3183615511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc03f060 con 0x55f4f8169670 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdba37fe640 1 -- 172.21.15.31:0/3183615511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fdbbc00fb60 con 0x55f4f8169670 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.886+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbac0cd650 con 0x55f4f8169670 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdba37fe640 1 -- 172.21.15.31:0/3183615511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbbc036070 con 0x55f4f8169670 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 10 client.16884 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:27.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4f85ba730 con 0x55f4f8169670 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdba37fe640 1 -- 172.21.15.31:0/3183615511 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbac0cd650 con 0x55f4f8169670 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdba37fe640 1 -- 172.21.15.31:0/3183615511 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdbbc02e040 con 0x55f4f8169670 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 10 client.16884 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 20 client.16884 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f818a350 0x55f4f85bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:27.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 4 client.16884 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.887+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f4f84a5860 con 0x55f4f818a350 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.888+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f818a350 0x55f4f85bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.888+0000 7fdba37fe640 10 client.16884 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.888+0000 7fdbc3fff640 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f818a350 0x55f4f85bcf50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b8861f71fb3b64 server_cookie=f782416fb3270a65 in_seq=0 out_seq=0 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdba37fe640 1 -- 172.21.15.31:0/3183615511 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f4f84a5860 con 0x55f4f818a350 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdba37fe640 10 client.16884 handle_command_reply: tid=0 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 client.16884 shutdown 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 msgr2=0x55f4f85b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 0x55f4f85b2d40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0307b0 tx=0x7fdbbc012e70 comp rx=0 tx=0).stop 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 shutdown_connections 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55f4f818a350 0x55f4f85bcf50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8169670 0x55f4f85b2d40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4f8161490 0x55f4f8541e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 --2- 172.21.15.31:0/3183615511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f8148cd0 0x55f4f8541840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 >> 172.21.15.31:0/3183615511 conn(0x55f4f817e740 msgr2=0x55f4f8532bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdba37fe640 0 client.16884 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 shutdown_connections 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 1 -- 172.21.15.31:0/3183615511 wait complete. 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.890+0000 7fdbd0e4f740 20 client.16884 trim_cache size 0 max 16384 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "4b117554-5eb7-4b30-bc16-452cb8bcca0d", 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:58:27.895 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdbb00b6680 msgr2=0x7fdbb00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdbb00b6680 0x7fdbb00b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005fd0 tx=0x7fdbbc009590 comp rx=0 tx=0).stop 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 msgr2=0x7fdbc4102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc4102cb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdbac02a930 tx=0x7fdbac004270 comp rx=0 tx=0).stop 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 shutdown_connections 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fdbb00b6680 0x7fdbb00b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdbc410de60 0x7fdbc41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbc4105380 0x7fdbc4102cb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 --2- 172.21.15.31:0/3521021715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc4104a30 0x7fdbc4102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 >> 172.21.15.31:0/3521021715 conn(0x7fdbc4079760 msgr2=0x7fdbc41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 shutdown_connections 2024-09-21T02:58:27.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:27.891+0000 7fdbca8db640 1 -- 172.21.15.31:0/3521021715 wait complete. 2024-09-21T02:58:28.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:58:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:28 smithi031 ceph-mon[31209]: pgmap v492: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:28 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:58:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:28 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:58:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:28 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:58:28.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:28 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:58:28.925 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:28 smithi165 ceph-mon[34818]: pgmap v492: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:28 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:58:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:28 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:58:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:28 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:58:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:28 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:58:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:28 smithi178 ceph-mon[34759]: pgmap v492: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:28 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:58:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:28 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:58:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:28 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:58:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:28 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:58:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 -- 172.21.15.31:0/318258734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb908104880 msgr2=0x7fb908104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 --2- 172.21.15.31:0/318258734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb908104880 0x7fb908104c90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb904009900 tx=0x7fb90402a3f0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 -- 172.21.15.31:0/318258734 shutdown_connections 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 --2- 172.21.15.31:0/318258734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9081052b0 0x7fb90810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 --2- 172.21.15.31:0/318258734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb90810de30 0x7fb908110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 --2- 172.21.15.31:0/318258734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb908104880 0x7fb908104c90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 -- 172.21.15.31:0/318258734 >> 172.21.15.31:0/318258734 conn(0x7fb908079730 msgr2=0x7fb908079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 -- 172.21.15.31:0/318258734 shutdown_connections 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.407+0000 7fb90f457640 1 -- 172.21.15.31:0/318258734 wait complete. 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.408+0000 7fb90f457640 1 Processor -- start 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.408+0000 7fb90f457640 1 -- start start 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb908104880 0x7fb90819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 0x7fb90819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb90810de30 0x7fb90819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb908113ad0 con 0x7fb9081052b0 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb908113950 con 0x7fb90810de30 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb908113c50 con 0x7fb908104880 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 0x7fb90819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 0x7fb90819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44086/0 (socket says 172.21.15.31:44086) 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 -- 172.21.15.31:0/2268885131 learned_addr learned my addr 172.21.15.31:0/2268885131 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb908104880 0x7fb90819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb90810de30 0x7fb90819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 -- 172.21.15.31:0/2268885131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb908104880 msgr2=0x7fb90819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb908104880 0x7fb90819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 -- 172.21.15.31:0/2268885131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb90810de30 msgr2=0x7fb90819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb90810de30 0x7fb90819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 -- 172.21.15.31:0/2268885131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9081a0350 con 0x7fb9081052b0 2024-09-21T02:58:29.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 0x7fb90819aa40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c970 tx=0x7fb8f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb8fe7fc640 1 -- 172.21.15.31:0/2268885131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8014070 con 0x7fb9081052b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9081a0640 con 0x7fb9081052b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb8fe7fc640 1 -- 172.21.15.31:0/2268885131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8f8004440 con 0x7fb9081052b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.409+0000 7fb8fe7fc640 1 -- 172.21.15.31:0/2268885131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8003e50 con 0x7fb9081052b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.410+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9081a6e60 con 0x7fb9081052b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.412+0000 7fb8fe7fc640 1 -- 172.21.15.31:0/2268885131 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb8f8017070 con 0x7fb9081052b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.412+0000 7fb9159cb740 1 Processor -- start 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 -- start start 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 0x56117ad16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56117ac8ce50 con 0x56117a9461b0 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56117ac7c280 con 0x56117a94e390 2024-09-21T02:58:29.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb9159cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56117ac80090 con 0x56117a96f070 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb90d9f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 0x56117ad16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb90d1f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb90d9f4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 0x56117ad16650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:52704/0 (socket says 172.21.15.31:52704) 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb90d9f4640 1 -- 172.21.15.31:0/1232085804 learned_addr learned my addr 172.21.15.31:0/1232085804 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.413+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d9f4640 1 -- 172.21.15.31:0/1232085804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 msgr2=0x56117ad170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d9f4640 1 -- 172.21.15.31:0/1232085804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 msgr2=0x56117ad16b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad16b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d9f4640 1 -- 172.21.15.31:0/1232085804 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117a9da650 con 0x56117a96f070 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb8fe7fc640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8f40b6850 0x7fb8f40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8f40b6850 0x7fb8f40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb8fe7fc640 1 -- 172.21.15.31:0/2268885131 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb8f80e7240 con 0x7fb9081052b0 2024-09-21T02:58:29.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 0x56117ad16650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb900009f20 tx=0x7fb90000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7882c1b7bf24005 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb8f37fe640 1 -- 172.21.15.31:0/1232085804 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90001a070 con 0x56117a96f070 2024-09-21T02:58:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb8f37fe640 1 -- 172.21.15.31:0/1232085804 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9000158c0 con 0x56117a96f070 2024-09-21T02:58:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.414+0000 7fb8f37fe640 1 -- 172.21.15.31:0/1232085804 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90000a6d0 con 0x56117a96f070 2024-09-21T02:58:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 -- 172.21.15.31:0/1232085804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 msgr2=0x56117ad16650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 0x56117ad16650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb900009f20 tx=0x7fb90000c8b0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8f40b6850 0x7fb8f40b8d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb904000c00 tx=0x7fb904031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 -- 172.21.15.31:0/1232085804 shutdown_connections 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad16b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 --2- 172.21.15.31:0/1232085804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a96f070 0x56117ad16650 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 -- 172.21.15.31:0/1232085804 >> 172.21.15.31:0/1232085804 conn(0x56117a963460 msgr2=0x56117ad14540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.415+0000 7fb9159cb740 1 -- 172.21.15.31:0/1232085804 shutdown_connections 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.416+0000 7fb9159cb740 1 -- 172.21.15.31:0/1232085804 wait complete. 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.416+0000 7fb9159cb740 1 -- start start 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.416+0000 7fb9159cb740 1 Processor -- start 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 0x56117ad324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56117aadd700 con 0x56117a96f070 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56117ac17110 con 0x56117a94e390 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56117ad29190 con 0x56117a9461b0 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb9159cb740 10 client.0 _ll_register_callbacks cb 0x56117ac30990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 0x56117ad324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 0x56117ad324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44098/0 (socket says 172.21.15.31:44098) 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 -- 172.21.15.31:0/842105470 learned_addr learned my addr 172.21.15.31:0/842105470 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 -- 172.21.15.31:0/842105470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 msgr2=0x56117ad26110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad26110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 -- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 msgr2=0x56117ad26650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad26650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 -- 172.21.15.31:0/842105470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117ac44c40 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 0x56117ad324f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb900018040 tx=0x7fb900015ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dea7acf22a1156a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.417+0000 7fb8f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/842105470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90001a040 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/842105470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb900009b60 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9000158c0 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/842105470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb90000a470 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 10 client.16896 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56117ad34ed0 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/842105470 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9000158c0 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/842105470 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb9000205e0 con 0x56117a96f070 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 10 client.16896 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:29.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 20 client.16896 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a92d9f0 0x56117ad382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 4 client.16896 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.418+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56117ac8eda0 con 0x56117a92d9f0 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.419+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a92d9f0 0x56117ad382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.419+0000 7fb8f0ff9640 10 client.16896 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.419+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a92d9f0 0x56117ad382e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89450d2583b255b0 server_cookie=4e4b11a48bbb9df5 in_seq=0 out_seq=0 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/842105470 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56117ac8eda0 con 0x56117a92d9f0 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb8f0ff9640 10 client.16896 handle_command_reply: tid=0 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 client.16896 shutdown 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 msgr2=0x56117ad324f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 0x56117ad324f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb900018040 tx=0x7fb900015ca0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 shutdown_connections 2024-09-21T02:58:29.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a92d9f0 0x56117ad382e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a96f070 0x56117ad324f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 --2- 172.21.15.31:0/842105470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb8f0ff9640 0 client.16896 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.420+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 >> 172.21.15.31:0/842105470 conn(0x56117a963460 msgr2=0x56117ad12b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.421+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 shutdown_connections 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.421+0000 7fb9159cb740 1 -- 172.21.15.31:0/842105470 wait complete. 2024-09-21T02:58:29.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.421+0000 7fb9159cb740 20 client.16896 trim_cache size 0 max 16384 2024-09-21T02:58:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 Processor -- start 2024-09-21T02:58:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 -- start start 2024-09-21T02:58:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a92d9f0 0x56117ad265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081ad4e0 con 0x56117a9461b0 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081af020 con 0x56117a94e390 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb9159cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081adbb0 con 0x56117a92d9f0 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad1b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44110/0 (socket says 172.21.15.31:44110) 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 -- 172.21.15.31:0/3634213706 learned_addr learned my addr 172.21.15.31:0/3634213706 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a92d9f0 0x56117ad265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 -- 172.21.15.31:0/3634213706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a92d9f0 msgr2=0x56117ad265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a92d9f0 0x56117ad265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 -- 172.21.15.31:0/3634213706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 msgr2=0x56117ad1bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad1bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90c9f2640 1 -- 172.21.15.31:0/3634213706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117ad27060 con 0x56117a9461b0 2024-09-21T02:58:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.439+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad1bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.440+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad1b7f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80acdb0 tx=0x7fb8f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1efc1b99f73b66c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.440+0000 7fb8f2ffd640 1 -- 172.21.15.31:0/3634213706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80a8f50 con 0x56117a9461b0 2024-09-21T02:58:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.440+0000 7fb8f2ffd640 1 -- 172.21.15.31:0/3634213706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8f80acbc0 con 0x56117a9461b0 2024-09-21T02:58:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.440+0000 7fb8f2ffd640 1 -- 172.21.15.31:0/3634213706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80ac820 con 0x56117a9461b0 2024-09-21T02:58:29.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 -- 172.21.15.31:0/3634213706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 msgr2=0x56117ad1b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad1b7f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80acdb0 tx=0x7fb8f800cf60 comp rx=0 tx=0).stop 2024-09-21T02:58:29.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 -- 172.21.15.31:0/3634213706 shutdown_connections 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad1bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a9461b0 0x56117ad1b7f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 --2- 172.21.15.31:0/3634213706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a92d9f0 0x56117ad265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 -- 172.21.15.31:0/3634213706 >> 172.21.15.31:0/3634213706 conn(0x56117a963460 msgr2=0x56117ad25ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 -- 172.21.15.31:0/3634213706 shutdown_connections 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 -- 172.21.15.31:0/3634213706 wait complete. 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 -- start start 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.441+0000 7fb9159cb740 1 Processor -- start 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 0x56117ad2a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad2a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081ae950 con 0x56117a92d9f0 2024-09-21T02:58:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081ae280 con 0x56117a94e390 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081ab4c0 con 0x56117a9461b0 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb9159cb740 10 client.0 _ll_register_callbacks cb 0x7fb900018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d1f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 0x56117ad2a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d9f4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d1f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 0x56117ad2a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44118/0 (socket says 172.21.15.31:44118) 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d1f3640 1 -- 172.21.15.31:0/2106509224 learned_addr learned my addr 172.21.15.31:0/2106509224 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90c9f2640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad2a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d9f4640 1 -- 172.21.15.31:0/2106509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 msgr2=0x56117ad2a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad2a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d9f4640 1 -- 172.21.15.31:0/2106509224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 msgr2=0x56117ad2a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 0x56117ad2a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d9f4640 1 -- 172.21.15.31:0/2106509224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117ada1ab0 con 0x56117a94e390 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.442+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 0x56117ad2a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb90d9f4640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad2ae30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb900020a20 tx=0x7fb900009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3fad0695f61ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb8f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/2106509224 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900037060 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/2106509224 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb900015720 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f80acbc0 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/2106509224 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb90001a040 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb9159cb740 10 client.26354 fetch_fsmap learned FSMap version 27 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56117ada2c70 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.443+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/2106509224 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8f80acbc0 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/2106509224 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb90000acb0 con 0x56117a94e390 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb9159cb740 10 client.26354 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb9159cb740 10 client.26354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb9159cb740 20 client.26354 populate_metadata read hostname 'smithi031' 2024-09-21T02:58:29.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb9159cb740 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a96f070 0x56117ada5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:29.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb9159cb740 4 client.26354 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:58:29.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56117ac8a850 con 0x56117a96f070 2024-09-21T02:58:29.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a96f070 0x56117ada5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb90d1f3640 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a96f070 0x56117ada5310 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba72957c87abbed server_cookie=d6abfa660edc002c in_seq=0 out_seq=0 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.444+0000 7fb8f0ff9640 10 client.26354 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb8f0ff9640 1 -- 172.21.15.31:0/2106509224 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x56117ac8a850 con 0x56117a96f070 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb8f0ff9640 10 client.26354 handle_command_reply: tid=0 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 client.26354 shutdown 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 msgr2=0x56117ad2ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad2ae30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb900020a20 tx=0x7fb900009ea0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 shutdown_connections 2024-09-21T02:58:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56117a96f070 0x56117ada5310 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56117a9461b0 0x56117ad2a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56117a94e390 0x56117ad2ae30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb8f0ff9640 0 client.26354 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 --2- 172.21.15.31:0/2106509224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56117a92d9f0 0x56117ad2a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 >> 172.21.15.31:0/2106509224 conn(0x56117a963460 msgr2=0x56117ad1b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 shutdown_connections 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 1 -- 172.21.15.31:0/2106509224 wait complete. 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.445+0000 7fb9159cb740 20 client.26354 trim_cache size 0 max 16384 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:58:29.451 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "4b117554-5eb7-4b30-bc16-452cb8bcca0d", 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.452 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T02:58:29.454 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.455 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:58:29.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8f40b6850 msgr2=0x7fb8f40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8f40b6850 0x7fb8f40b8d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb904000c00 tx=0x7fb904031400 comp rx=0 tx=0).stop 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 msgr2=0x7fb90819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 0x7fb90819aa40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c970 tx=0x7fb8f800ce30 comp rx=0 tx=0).stop 2024-09-21T02:58:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 shutdown_connections 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb8f40b6850 0x7fb8f40b8d60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb90810de30 0x7fb90819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9081052b0 0x7fb90819aa40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 --2- 172.21.15.31:0/2268885131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb908104880 0x7fb90819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 >> 172.21.15.31:0/2268885131 conn(0x7fb908079730 msgr2=0x7fb908100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 shutdown_connections 2024-09-21T02:58:29.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:58:29.446+0000 7fb90f457640 1 -- 172.21.15.31:0/2268885131 wait complete. 2024-09-21T02:58:30.231 INFO:tasks.cephfs.filesystem:scrub status for tag:4b117554-5eb7-4b30-bc16-452cb8bcca0d - {'path': '/', 'tag': '4b117554-5eb7-4b30-bc16-452cb8bcca0d', 'options': 'recursive,force'} 2024-09-21T02:58:30.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:30 smithi165 ceph-mon[34818]: pgmap v493: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:30.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:30 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:58:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:30 smithi178 ceph-mon[34759]: pgmap v493: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:30 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:58:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:30 smithi031 ceph-mon[31209]: pgmap v493: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:30 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:58:32.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:32 smithi165 ceph-mon[34818]: pgmap v494: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:32 smithi178 ceph-mon[34759]: pgmap v494: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:32 smithi031 ceph-mon[31209]: pgmap v494: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:34 smithi165 ceph-mon[34818]: pgmap v495: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T02:58:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:34 smithi178 ceph-mon[34759]: pgmap v495: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T02:58:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:34 smithi031 ceph-mon[31209]: pgmap v495: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T02:58:36.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:36 smithi165 ceph-mon[34818]: pgmap v496: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-09-21T02:58:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:36 smithi178 ceph-mon[34759]: pgmap v496: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-09-21T02:58:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:36 smithi031 ceph-mon[31209]: pgmap v496: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-09-21T02:58:38.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:38 smithi165 ceph-mon[34818]: pgmap v497: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-09-21T02:58:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:38 smithi178 ceph-mon[34759]: pgmap v497: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-09-21T02:58:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:38 smithi031 ceph-mon[31209]: pgmap v497: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-09-21T02:58:40.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:40 smithi165 ceph-mon[34818]: pgmap v498: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:40 smithi178 ceph-mon[34759]: pgmap v498: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:40 smithi031 ceph-mon[31209]: pgmap v498: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:42.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:42 smithi165 ceph-mon[34818]: pgmap v499: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-21T02:58:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:42 smithi178 ceph-mon[34759]: pgmap v499: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-21T02:58:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:42 smithi031 ceph-mon[31209]: pgmap v499: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-21T02:58:44.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:44 smithi165 ceph-mon[34818]: pgmap v500: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T02:58:44.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:44 smithi178 ceph-mon[34759]: pgmap v500: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T02:58:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:44 smithi031 ceph-mon[31209]: pgmap v500: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T02:58:46.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:46 smithi165 ceph-mon[34818]: pgmap v501: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:46 smithi178 ceph-mon[34759]: pgmap v501: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:46 smithi031 ceph-mon[31209]: pgmap v501: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:48.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:48 smithi178 ceph-mon[34759]: pgmap v502: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:48 smithi031 ceph-mon[31209]: pgmap v502: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:48 smithi165 ceph-mon[34818]: pgmap v502: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:50.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:50 smithi178 ceph-mon[34759]: pgmap v503: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:50 smithi031 ceph-mon[31209]: pgmap v503: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:50 smithi165 ceph-mon[34818]: pgmap v503: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:52 smithi178 ceph-mon[34759]: pgmap v504: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:52 smithi031 ceph-mon[31209]: pgmap v504: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:52 smithi165 ceph-mon[34818]: pgmap v504: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:58:54.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:54 smithi178 ceph-mon[34759]: pgmap v505: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:58:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:54 smithi031 ceph-mon[31209]: pgmap v505: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:58:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:54 smithi165 ceph-mon[34818]: pgmap v505: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:58:56.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:56 smithi178 ceph-mon[34759]: pgmap v506: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:56 smithi031 ceph-mon[31209]: pgmap v506: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:56 smithi165 ceph-mon[34818]: pgmap v506: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:58:58.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:58:58 smithi178 ceph-mon[34759]: pgmap v507: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:58.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:58:58 smithi165 ceph-mon[34818]: pgmap v507: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:58:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:58:58 smithi031 ceph-mon[31209]: pgmap v507: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:00.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:59:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:00.915 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:00 smithi031 ceph-mon[31209]: pgmap v508: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:00.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:00 smithi178 ceph-mon[34759]: pgmap v508: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 -- 172.21.15.31:0/3367135209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 msgr2=0x7fc7b010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 --2- 172.21.15.31:0/3367135209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b010f520 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009ca0 tx=0x7fc7a802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 -- 172.21.15.31:0/3367135209 shutdown_connections 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 --2- 172.21.15.31:0/3367135209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b01083a0 0x7fc7b01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 --2- 172.21.15.31:0/3367135209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b010f520 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 --2- 172.21.15.31:0/3367135209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b0102a60 0x7fc7b0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 -- 172.21.15.31:0/3367135209 >> 172.21.15.31:0/3367135209 conn(0x7fc7b0079730 msgr2=0x7fc7b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 -- 172.21.15.31:0/3367135209 shutdown_connections 2024-09-21T02:59:01.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.013+0000 7fc7b8114640 1 -- 172.21.15.31:0/3367135209 wait complete. 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 Processor -- start 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 -- start start 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b0102a60 0x7fc7b0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b01083a0 0x7fc7b0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b0112960 con 0x7fc7b0108cf0 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01127e0 con 0x7fc7b0102a60 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b8114640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b0112ae0 con 0x7fc7b01083a0 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b66b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b66b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34916/0 (socket says 172.21.15.31:34916) 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.014+0000 7fc7b66b1640 1 -- 172.21.15.31:0/2585290237 learned_addr learned my addr 172.21.15.31:0/2585290237 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b56af640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b01083a0 0x7fc7b0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b0102a60 0x7fc7b0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b66b1640 1 -- 172.21.15.31:0/2585290237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b01083a0 msgr2=0x7fc7b0199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b66b1640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b01083a0 0x7fc7b0199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b66b1640 1 -- 172.21.15.31:0/2585290237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b0102a60 msgr2=0x7fc7b0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b66b1640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b0102a60 0x7fc7b0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b66b1640 1 -- 172.21.15.31:0/2585290237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b019f300 con 0x7fc7b0108cf0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b66b1640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b019ebd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8005cc0 tx=0x7fc7a802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7a6ffd640 1 -- 172.21.15.31:0/2585290237 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8003bf0 con 0x7fc7b0108cf0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b019f590 con 0x7fc7b0108cf0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7a6ffd640 1 -- 172.21.15.31:0/2585290237 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a8003d90 con 0x7fc7b0108cf0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7a6ffd640 1 -- 172.21.15.31:0/2585290237 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a802fbc0 con 0x7fc7b0108cf0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.015+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b01a5d50 con 0x7fc7b0108cf0 2024-09-21T02:59:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.019+0000 7fc7be688740 1 Processor -- start 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.019+0000 7fc7a6ffd640 1 -- 172.21.15.31:0/2585290237 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc7a802fd60 con 0x7fc7b0108cf0 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.019+0000 7fc7be688740 1 -- start start 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7be688740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce295e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7be688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce258e1b0 0x562ce295e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7be688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce295ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7be688740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562ce28d4e50 con 0x562ce25b7070 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7be688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562ce28c4280 con 0x562ce258e1b0 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7be688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ce28c8090 con 0x562ce2596390 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce295e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce295e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34918/0 (socket says 172.21.15.31:34918) 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 -- 172.21.15.31:0/474840130 learned_addr learned my addr 172.21.15.31:0/474840130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce258e1b0 0x562ce295e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7a6ffd640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc79c0b6680 0x7fc79c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce295ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 -- 172.21.15.31:0/474840130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 msgr2=0x562ce295ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce295ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b66b1640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc79c0b6680 0x7fc79c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 -- 172.21.15.31:0/474840130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce258e1b0 msgr2=0x562ce295e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce258e1b0 0x562ce295e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b56af640 1 -- 172.21.15.31:0/474840130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce2622650 con 0x562ce25b7070 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce258e1b0 0x562ce295e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.020+0000 7fc7a6ffd640 1 -- 172.21.15.31:0/2585290237 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc7a8108b90 con 0x7fc7b0108cf0 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.021+0000 7fc7b56af640 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce295e270 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000bf70 tx=0x7fc7a000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd50d59c56b0483e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.021+0000 7fc79bfff640 1 -- 172.21.15.31:0/474840130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0004030 con 0x562ce25b7070 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.021+0000 7fc79bfff640 1 -- 172.21.15.31:0/474840130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a00054d0 con 0x562ce25b7070 2024-09-21T02:59:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.021+0000 7fc79bfff640 1 -- 172.21.15.31:0/474840130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0004a90 con 0x562ce25b7070 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 -- 172.21.15.31:0/474840130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 msgr2=0x562ce295e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce295e270 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000bf70 tx=0x7fc7a000b780 comp rx=0 tx=0).stop 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7b66b1640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc79c0b6680 0x7fc79c0b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc7a81063b0 tx=0x7fc7a80f2040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 -- 172.21.15.31:0/474840130 shutdown_connections 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce295ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce258e1b0 0x562ce295e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 --2- 172.21.15.31:0/474840130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce295e270 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 -- 172.21.15.31:0/474840130 >> 172.21.15.31:0/474840130 conn(0x562ce25ab460 msgr2=0x562ce295c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 -- 172.21.15.31:0/474840130 shutdown_connections 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.022+0000 7fc7be688740 1 -- 172.21.15.31:0/474840130 wait complete. 2024-09-21T02:59:01.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 -- start start 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 Processor -- start 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce258e1b0 0x562ce296bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce296c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce2977f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562ce2725700 con 0x562ce25b7070 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562ce285f110 con 0x562ce2596390 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.023+0000 7fc7be688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ce296ebb0 con 0x562ce258e1b0 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7be688740 10 client.0 _ll_register_callbacks cb 0x562ce2878990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce2977f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce296c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce2977f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34934/0 (socket says 172.21.15.31:34934) 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b56af640 1 -- 172.21.15.31:0/232192209 learned_addr learned my addr 172.21.15.31:0/232192209 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b56af640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce258e1b0 0x562ce296bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 -- 172.21.15.31:0/232192209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce258e1b0 msgr2=0x562ce296bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce258e1b0 0x562ce296bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 -- 172.21.15.31:0/232192209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 msgr2=0x562ce2977f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce2977f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 -- 172.21.15.31:0/232192209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce288cc40 con 0x562ce2596390 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b56af640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce2977f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce296c070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac015ad0 tx=0x7fc7ac015f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9121a6d2ac1a50c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.024+0000 7fc7997fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7997fa640 1 -- 172.21.15.31:0/232192209 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00be40 con 0x562ce2596390 2024-09-21T02:59:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a00054d0 con 0x562ce2596390 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7997fa640 1 -- 172.21.15.31:0/232192209 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc7ac003800 con 0x562ce2596390 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7997fa640 1 -- 172.21.15.31:0/232192209 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7ac003a00 con 0x562ce2596390 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 10 client.26366 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562ce2964600 con 0x562ce2596390 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7997fa640 1 -- 172.21.15.31:0/232192209 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a00054d0 con 0x562ce2596390 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7997fa640 1 -- 172.21.15.31:0/232192209 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7ac026070 con 0x562ce2596390 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 10 client.26366 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 20 client.26366 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce25759f0 0x562ce2965be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 4 client.26366 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.025+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce28d6da0 con 0x562ce25759f0 2024-09-21T02:59:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.026+0000 7fc7b56af640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce25759f0 0x562ce2965be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.026+0000 7fc7997fa640 10 client.26366 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.026+0000 7fc7b56af640 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce25759f0 0x562ce2965be0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4117bf65f7cf8d34 server_cookie=c8b93041493c942b in_seq=0 out_seq=0 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7997fa640 1 -- 172.21.15.31:0/232192209 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ce28d6da0 con 0x562ce25759f0 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7997fa640 10 client.26366 handle_command_reply: tid=0 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 client.26366 shutdown 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 msgr2=0x562ce296c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce296c070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac015ad0 tx=0x7fc7ac015f90 comp rx=0 tx=0).stop 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 shutdown_connections 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce25759f0 0x562ce2965be0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce25b7070 0x562ce2977f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce296c070 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 --2- 172.21.15.31:0/232192209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce258e1b0 0x562ce296bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 >> 172.21.15.31:0/232192209 conn(0x562ce25ab460 msgr2=0x562ce295a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7997fa640 0 client.26366 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 shutdown_connections 2024-09-21T02:59:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 1 -- 172.21.15.31:0/232192209 wait complete. 2024-09-21T02:59:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.027+0000 7fc7be688740 20 client.26366 trim_cache size 0 max 16384 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 Processor -- start 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 -- start start 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce2963a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce2977e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce25b7070 0x562ce2968d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01ad540 con 0x562ce258e1b0 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01af080 con 0x562ce2596390 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.045+0000 7fc7be688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01adc10 con 0x562ce25b7070 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce2963a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b5eb0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce2977e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce2963a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34944/0 (socket says 172.21.15.31:34944) 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 -- 172.21.15.31:0/197941295 learned_addr learned my addr 172.21.15.31:0/197941295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce25b7070 0x562ce2968d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 -- 172.21.15.31:0/197941295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce25b7070 msgr2=0x562ce2968d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce25b7070 0x562ce2968d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 -- 172.21.15.31:0/197941295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 msgr2=0x562ce2977e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce2977e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 -- 172.21.15.31:0/197941295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce296e270 con 0x562ce258e1b0 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce2977e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc7b56af640 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce2963a50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0004510 tx=0x7fc7a000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30395a1849af4959 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc79b7fe640 1 -- 172.21.15.31:0/197941295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0027040 con 0x562ce258e1b0 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.046+0000 7fc79b7fe640 1 -- 172.21.15.31:0/197941295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a0010b10 con 0x562ce258e1b0 2024-09-21T02:59:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.047+0000 7fc79b7fe640 1 -- 172.21.15.31:0/197941295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a001ea60 con 0x562ce258e1b0 2024-09-21T02:59:01.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 -- 172.21.15.31:0/197941295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 msgr2=0x562ce2963a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce2963a50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0004510 tx=0x7fc7a000bee0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 -- 172.21.15.31:0/197941295 shutdown_connections 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce25b7070 0x562ce2968d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce2596390 0x562ce2977e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 --2- 172.21.15.31:0/197941295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce2963a50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 -- 172.21.15.31:0/197941295 >> 172.21.15.31:0/197941295 conn(0x562ce25ab460 msgr2=0x562ce2963340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 -- 172.21.15.31:0/197941295 shutdown_connections 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 -- 172.21.15.31:0/197941295 wait complete. 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 -- start start 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.048+0000 7fc7be688740 1 Processor -- start 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce297d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce297d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce25b7070 0x562ce29deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01ae9b0 con 0x562ce258e1b0 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01ae2e0 con 0x562ce25b7070 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b01ab520 con 0x562ce2596390 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7be688740 10 client.0 _ll_register_callbacks cb 0x7fc7ac004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce297d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce297d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34958/0 (socket says 172.21.15.31:34958) 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 -- 172.21.15.31:0/2542039156 learned_addr learned my addr 172.21.15.31:0/2542039156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce297d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 -- 172.21.15.31:0/2542039156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 msgr2=0x562ce297d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce297d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 -- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce25b7070 msgr2=0x562ce29deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce25b7070 0x562ce29deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.049+0000 7fc7b56af640 1 -- 172.21.15.31:0/2542039156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce29df170 con 0x562ce258e1b0 2024-09-21T02:59:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7b56af640 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce297d120 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000b510 tx=0x7fc7a0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b03bb96983f6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7997fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7997fa640 1 -- 172.21.15.31:0/2542039156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0010660 con 0x562ce258e1b0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a0010b10 con 0x562ce258e1b0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7997fa640 1 -- 172.21.15.31:0/2542039156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fc7a0010800 con 0x562ce258e1b0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7997fa640 1 -- 172.21.15.31:0/2542039156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7a0035480 con 0x562ce258e1b0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7be688740 10 client.16920 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562ce29e6450 con 0x562ce258e1b0 2024-09-21T02:59:01.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.050+0000 7fc7997fa640 1 -- 172.21.15.31:0/2542039156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a0010b10 con 0x562ce258e1b0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7997fa640 1 -- 172.21.15.31:0/2542039156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7a002d040 con 0x562ce258e1b0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7be688740 10 client.16920 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7be688740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7be688740 20 client.16920 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7be688740 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce254ada0 0x562ce29e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7be688740 4 client.16920 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ce28d2850 con 0x562ce254ada0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce254ada0 0x562ce29e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7b5eb0640 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce254ada0 0x562ce29e88e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b27b54f6b43de0d server_cookie=2b1ddf5abfee8f2f in_seq=0 out_seq=0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.051+0000 7fc7997fa640 10 client.16920 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7997fa640 1 -- 172.21.15.31:0/2542039156 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x562ce28d2850 con 0x562ce254ada0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7997fa640 10 client.16920 handle_command_reply: tid=0 2024-09-21T02:59:01.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 client.16920 shutdown 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 msgr2=0x562ce297d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce297d120 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000b510 tx=0x7fc7a0010d50 comp rx=0 tx=0).stop 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 shutdown_connections 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x562ce254ada0 0x562ce29e88e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ce2596390 0x562ce297d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ce25b7070 0x562ce29deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 --2- 172.21.15.31:0/2542039156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ce258e1b0 0x562ce297d120 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7997fa640 0 client.16920 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 >> 172.21.15.31:0/2542039156 conn(0x562ce25ab460 msgr2=0x562ce2963ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 shutdown_connections 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 1 -- 172.21.15.31:0/2542039156 wait complete. 2024-09-21T02:59:01.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.052+0000 7fc7be688740 20 client.16920 trim_cache size 0 max 16384 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.059 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T02:59:01.060 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.061 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.062 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:59:01.063 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:01.064 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc79c0b6680 msgr2=0x7fc79c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc79c0b6680 0x7fc79c0b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc7a81063b0 tx=0x7fc7a80f2040 comp rx=0 tx=0).stop 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 msgr2=0x7fc7b019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b019ebd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8005cc0 tx=0x7fc7a802f380 comp rx=0 tx=0).stop 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 shutdown_connections 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc79c0b6680 0x7fc79c0b8b90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b01083a0 0x7fc7b0199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b0108cf0 0x7fc7b019ebd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 --2- 172.21.15.31:0/2585290237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b0102a60 0x7fc7b0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 >> 172.21.15.31:0/2585290237 conn(0x7fc7b0079730 msgr2=0x7fc7b0104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:01.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 shutdown_connections 2024-09-21T02:59:01.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:01.053+0000 7fc7b8114640 1 -- 172.21.15.31:0/2585290237 wait complete. 2024-09-21T02:59:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:00 smithi165 ceph-mon[34818]: pgmap v508: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:01.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:59:01.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:02.103 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2032466454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea81049e0 msgr2=0x7f8ea8104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2032466454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea81049e0 0x7f8ea8104df0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009ca0 tx=0x7f8e9802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2032466454 shutdown_connections 2024-09-21T02:59:02.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2032466454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8105330 0x7f8ea810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2032466454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea810de40 0x7f8ea8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2032466454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea81049e0 0x7f8ea8104df0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2032466454 >> 172.21.15.31:0/2032466454 conn(0x7f8ea8079750 msgr2=0x7f8ea8079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2032466454 shutdown_connections 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.625+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2032466454 wait complete. 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 Processor -- start 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 -- start start 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea81049e0 0x7f8ea8102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8105330 0x7f8ea8102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 0x7f8ea81031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8113af0 con 0x7f8ea810de40 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8113970 con 0x7f8ea81049e0 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ead6cd640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8113c70 con 0x7f8ea8105330 2024-09-21T02:59:02.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ea77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 0x7f8ea81031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ea77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 0x7f8ea81031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34966/0 (socket says 172.21.15.31:34966) 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.626+0000 7f8ea77fe640 1 -- 172.21.15.31:0/2773764499 learned_addr learned my addr 172.21.15.31:0/2773764499 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea67fc640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8105330 0x7f8ea8102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea6ffd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea81049e0 0x7f8ea8102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea77fe640 1 -- 172.21.15.31:0/2773764499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8105330 msgr2=0x7f8ea8102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea77fe640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8105330 0x7f8ea8102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea77fe640 1 -- 172.21.15.31:0/2773764499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea81049e0 msgr2=0x7f8ea8102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea77fe640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea81049e0 0x7f8ea8102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea77fe640 1 -- 172.21.15.31:0/2773764499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea81a8760 con 0x7f8ea810de40 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ea77fe640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 0x7f8ea81031a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e970 tx=0x7f8ea000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000ccc0 con 0x7f8ea810de40 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea000ce60 con 0x7f8ea810de40 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea81a8a50 con 0x7f8ea810de40 2024-09-21T02:59:02.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0010430 con 0x7f8ea810de40 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.627+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea81a8f60 con 0x7f8ea810de40 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.628+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea806c280 con 0x7f8ea810de40 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.630+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f8ea0004fb0 con 0x7f8ea810de40 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.631+0000 7f8e8ffff640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8e900b6700 0x7f8e900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.631+0000 7f8ea6ffd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8e900b6700 0x7f8e900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.631+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f8ea00e7ca0 con 0x7f8ea810de40 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.631+0000 7f8ea6ffd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8e900b6700 0x7f8e900b8c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e98004af0 tx=0x7f8e98005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:02.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.634+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ea0056710 con 0x7f8ea810de40 2024-09-21T02:59:02.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:02 smithi031 ceph-mon[31209]: pgmap v509: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:59:02.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.864+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ea8104df0 con 0x7f8ea810de40 2024-09-21T02:59:02.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.864+0000 7f8e8ffff640 1 -- 172.21.15.31:0/2773764499 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f8ea00a9030 con 0x7f8ea810de40 2024-09-21T02:59:02.866 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:59:02.866 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:59:02.866 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8e900b6700 msgr2=0x7f8e900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8e900b6700 0x7f8e900b8c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e98004af0 tx=0x7f8e98005fd0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 msgr2=0x7f8ea81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 0x7f8ea81031a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e970 tx=0x7f8ea000ed70 comp rx=0 tx=0).stop 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 shutdown_connections 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f8e900b6700 0x7f8e900b8c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8105330 0x7f8ea8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea810de40 0x7f8ea81031a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 --2- 172.21.15.31:0/2773764499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea81049e0 0x7f8ea8102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.866+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 >> 172.21.15.31:0/2773764499 conn(0x7f8ea8079750 msgr2=0x7f8ea8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.867+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 shutdown_connections 2024-09-21T02:59:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:02.867+0000 7f8ead6cd640 1 -- 172.21.15.31:0/2773764499 wait complete. 2024-09-21T02:59:02.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:02 smithi178 ceph-mon[34759]: pgmap v509: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:59:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:02 smithi165 ceph-mon[34818]: pgmap v509: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T02:59:03.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:59:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:03.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:03 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2773764499' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:03.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:03 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2773764499' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:03 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2773764499' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 -- 172.21.15.31:0/2588480429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09901048c0 msgr2=0x7f0990104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 --2- 172.21.15.31:0/2588480429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09901048c0 0x7f0990104cd0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0980009900 tx=0x7f098002a3f0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 -- 172.21.15.31:0/2588480429 shutdown_connections 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 --2- 172.21.15.31:0/2588480429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f099010de70 0x7f0990110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 --2- 172.21.15.31:0/2588480429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09901052a0 0x7f099010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 --2- 172.21.15.31:0/2588480429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09901048c0 0x7f0990104cd0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 -- 172.21.15.31:0/2588480429 >> 172.21.15.31:0/2588480429 conn(0x7f0990079760 msgr2=0x7f0990079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 -- 172.21.15.31:0/2588480429 shutdown_connections 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.258+0000 7f09958a9640 1 -- 172.21.15.31:0/2588480429 wait complete. 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 Processor -- start 2024-09-21T02:59:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 -- start start 2024-09-21T02:59:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09901048c0 0x7f099019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09901052a0 0x7f099019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 0x7f099019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113b00 con 0x7f099010de70 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113980 con 0x7f09901048c0 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.259+0000 7f09958a9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0990113c80 con 0x7f09901052a0 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 0x7f099019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098effd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09901048c0 0x7f099019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 0x7f099019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34992/0 (socket says 172.21.15.31:34992) 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 -- 172.21.15.31:0/2611346558 learned_addr learned my addr 172.21.15.31:0/2611346558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09901052a0 0x7f099019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 -- 172.21.15.31:0/2611346558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09901052a0 msgr2=0x7f099019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09901052a0 0x7f099019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 -- 172.21.15.31:0/2611346558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09901048c0 msgr2=0x7f099019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09901048c0 0x7f099019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 -- 172.21.15.31:0/2611346558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09901a03c0 con 0x7f099010de70 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f098f7fe640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 0x7f099019fd50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f098800cce0 tx=0x7f098800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f09948a7640 1 -- 172.21.15.31:0/2611346558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098800f450 con 0x7f099010de70 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f09948a7640 1 -- 172.21.15.31:0/2611346558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098800faa0 con 0x7f099010de70 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f09948a7640 1 -- 172.21.15.31:0/2611346558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988018a80 con 0x7f099010de70 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.260+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09901a06b0 con 0x7f099010de70 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.261+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09901026b0 con 0x7f099010de70 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.263+0000 7f09948a7640 1 -- 172.21.15.31:0/2611346558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f0988018c20 con 0x7f099010de70 2024-09-21T02:59:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.263+0000 7f099be1d740 1 Processor -- start 2024-09-21T02:59:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.263+0000 7f099be1d740 1 -- start start 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f099be1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb00766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f099be1d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb0076be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f099be1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb0077120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f099be1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559baffecea0 con 0x559bafcae390 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f099be1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559baffdc2d0 con 0x559bafccf070 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f099be1d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559baffe00e0 con 0x559bafca61b0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb0077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb0077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34996/0 (socket says 172.21.15.31:34996) 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 -- 172.21.15.31:0/2821116044 learned_addr learned my addr 172.21.15.31:0/2821116044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb00766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098effd640 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb0076be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 -- 172.21.15.31:0/2821116044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 msgr2=0x559bb0076be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb0076be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 -- 172.21.15.31:0/2821116044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 msgr2=0x559bb00766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb00766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 -- 172.21.15.31:0/2821116044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bafd3a650 con 0x559bafcae390 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f09948a7640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f09780b66b0 0x7f09780b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb0077120 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0984011780 tx=0x7f098400a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f45c1049bf070b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.264+0000 7f098effd640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f09780b66b0 0x7f09780b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.265+0000 7f09948a7640 1 -- 172.21.15.31:0/2611346558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f09880e7ab0 con 0x7f099010de70 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.265+0000 7f0975ffb640 1 -- 172.21.15.31:0/2821116044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984005020 con 0x559bafcae390 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.265+0000 7f0975ffb640 1 -- 172.21.15.31:0/2821116044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09840047a0 con 0x559bafcae390 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.265+0000 7f0975ffb640 1 -- 172.21.15.31:0/2821116044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098400fa20 con 0x559bafcae390 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 -- 172.21.15.31:0/2821116044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 msgr2=0x559bb0077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f098effd640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f09780b66b0 0x7f09780b8bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0980031440 tx=0x7f098003c040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb0077120 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0984011780 tx=0x7f098400a700 comp rx=0 tx=0).stop 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 -- 172.21.15.31:0/2821116044 shutdown_connections 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb0076be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb0077120 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 --2- 172.21.15.31:0/2821116044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb00766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 -- 172.21.15.31:0/2821116044 >> 172.21.15.31:0/2821116044 conn(0x559bafcc3460 msgr2=0x559bb0074590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 -- 172.21.15.31:0/2821116044 shutdown_connections 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 -- 172.21.15.31:0/2821116044 wait complete. 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 -- start start 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.266+0000 7f099be1d740 1 Processor -- start 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 0x559bb0081d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafcae390 0x559bb00822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb008e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bafe3d700 con 0x559bafca61b0 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559baff77160 con 0x559bafccf070 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559bb0084e80 con 0x559bafcae390 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f099be1d740 10 client.0 _ll_register_callbacks cb 0x559baff909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 0x559bb0081d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 0x559bb0081d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34998/0 (socket says 172.21.15.31:34998) 2024-09-21T02:59:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 -- 172.21.15.31:0/2395503321 learned_addr learned my addr 172.21.15.31:0/2395503321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafcae390 0x559bb00822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 -- 172.21.15.31:0/2395503321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafcae390 msgr2=0x559bb00822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafcae390 0x559bb00822d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 -- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 msgr2=0x559bb008e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098f7fe640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb008e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb008e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098effd640 1 -- 172.21.15.31:0/2395503321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559baffa4c90 con 0x559bafca61b0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafcae390 0x559bb00822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.267+0000 7f098f7fe640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb008e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f098effd640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 0x559bb0081d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0980005940 tx=0x7f098002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ef72caaaf61a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f096effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f096effd640 1 -- 172.21.15.31:0/2395503321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980055060 con 0x559bafca61b0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f096effd640 1 -- 172.21.15.31:0/2395503321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f0980003c50 con 0x559bafca61b0 2024-09-21T02:59:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09840047a0 con 0x559bafca61b0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f096effd640 1 -- 172.21.15.31:0/2395503321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0980035050 con 0x559bafca61b0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f099be1d740 10 client.16938 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bb0090bc0 con 0x559bafca61b0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f096effd640 1 -- 172.21.15.31:0/2395503321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09840047a0 con 0x559bafca61b0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f096effd640 1 -- 172.21.15.31:0/2395503321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0980059440 con 0x559bafca61b0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.268+0000 7f099be1d740 10 client.16938 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f099be1d740 10 client.16938 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f099be1d740 20 client.16938 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f099be1d740 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc8d9f0 0x559bb00940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f099be1d740 4 client.16938 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559baffeedf0 con 0x559bafc8d9f0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc8d9f0 0x559bb00940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f096effd640 10 client.16938 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.269+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc8d9f0 0x559bb00940f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4e203ad71399f62 server_cookie=c3855dfe43a97e77 in_seq=0 out_seq=0 2024-09-21T02:59:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f096effd640 1 -- 172.21.15.31:0/2395503321 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559baffeedf0 con 0x559bafc8d9f0 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f096effd640 10 client.16938 handle_command_reply: tid=0 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 client.16938 shutdown 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 msgr2=0x559bb0081d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 0x559bb0081d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0980005940 tx=0x7f098002a3f0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 shutdown_connections 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc8d9f0 0x559bb00940f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafcae390 0x559bb00822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafccf070 0x559bb008e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 --2- 172.21.15.31:0/2395503321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafca61b0 0x559bb0081d90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 >> 172.21.15.31:0/2395503321 conn(0x559bafcc3460 msgr2=0x559bb0079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f096effd640 0 client.16938 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.270+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 shutdown_connections 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.271+0000 7f099be1d740 1 -- 172.21.15.31:0/2395503321 wait complete. 2024-09-21T02:59:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.271+0000 7f099be1d740 20 client.16938 trim_cache size 0 max 16384 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 Processor -- start 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 -- start start 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafca61b0 0x559bb0082250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb008e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafccf070 0x559bb0073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f09901ad490 con 0x559bafcae390 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f09901aefd0 con 0x559bafca61b0 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f099be1d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09901adb60 con 0x559bafccf070 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f098e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb008e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f098e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb008e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35014/0 (socket says 172.21.15.31:35014) 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.284+0000 7f098e7fc640 1 -- 172.21.15.31:0/2372975008 learned_addr learned my addr 172.21.15.31:0/2372975008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098effd640 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafca61b0 0x559bb0082250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098f7fe640 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafccf070 0x559bb0073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098e7fc640 1 -- 172.21.15.31:0/2372975008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafccf070 msgr2=0x559bb0073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafccf070 0x559bb0073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098e7fc640 1 -- 172.21.15.31:0/2372975008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafca61b0 msgr2=0x559bb0082250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafca61b0 0x559bb0082250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098e7fc640 1 -- 172.21.15.31:0/2372975008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb00945f0 con 0x559bafcae390 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f098e7fc640 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb008e070 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0984011750 tx=0x7f098401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7c2a3743826255 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f09757fa640 1 -- 172.21.15.31:0/2372975008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098402b070 con 0x559bafcae390 2024-09-21T02:59:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f09757fa640 1 -- 172.21.15.31:0/2372975008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098400fe40 con 0x559bafcae390 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.285+0000 7f09757fa640 1 -- 172.21.15.31:0/2372975008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098401ec50 con 0x559bafcae390 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 -- 172.21.15.31:0/2372975008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 msgr2=0x559bb008e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb008e070 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0984011750 tx=0x7f098401a040 comp rx=0 tx=0).stop 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 -- 172.21.15.31:0/2372975008 shutdown_connections 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafccf070 0x559bb0073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafcae390 0x559bb008e070 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 --2- 172.21.15.31:0/2372975008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafca61b0 0x559bb0082250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 -- 172.21.15.31:0/2372975008 >> 172.21.15.31:0/2372975008 conn(0x559bafcc3460 msgr2=0x559bb0081b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 -- 172.21.15.31:0/2372975008 shutdown_connections 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 -- 172.21.15.31:0/2372975008 wait complete. 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.286+0000 7f099be1d740 1 -- start start 2024-09-21T02:59:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 Processor -- start 2024-09-21T02:59:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb008a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafcae390 0x559bb0085a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 0x559bb0085f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f09901ae900 con 0x559bafccf070 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f09901ae230 con 0x559bafcae390 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09901ab490 con 0x559bafca61b0 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f099be1d740 10 client.0 _ll_register_callbacks cb 0x7f0980005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f098f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 0x559bb0085f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f098f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 0x559bb0085f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35020/0 (socket says 172.21.15.31:35020) 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f098f7fe640 1 -- 172.21.15.31:0/4181085429 learned_addr learned my addr 172.21.15.31:0/4181085429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f098e7fc640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafcae390 0x559bb0085a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.287+0000 7f098effd640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb008a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098f7fe640 1 -- 172.21.15.31:0/4181085429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 msgr2=0x559bb008a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098f7fe640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb008a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098f7fe640 1 -- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafcae390 msgr2=0x559bb0085a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098f7fe640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafcae390 0x559bb0085a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098f7fe640 1 -- 172.21.15.31:0/4181085429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb00864d0 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098effd640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb008a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098e7fc640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafcae390 0x559bb0085a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f098f7fe640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 0x559bb0085f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f09880e5280 tx=0x7f09880ad7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56ffc9520a1f8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f096effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f096effd640 1 -- 172.21.15.31:0/4181085429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09880b2070 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f096effd640 1 -- 172.21.15.31:0/4181085429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f09880a9270 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098400fe40 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f096effd640 1 -- 172.21.15.31:0/4181085429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09880ae020 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f099be1d740 10 client.16944 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bb00fe6d0 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f096effd640 1 -- 172.21.15.31:0/4181085429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098400fe40 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.288+0000 7f096effd640 1 -- 172.21.15.31:0/4181085429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f09880d3060 con 0x559bafccf070 2024-09-21T02:59:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f099be1d740 10 client.16944 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f099be1d740 10 client.16944 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f099be1d740 20 client.16944 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f099be1d740 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc62da0 0x559bb0100e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f099be1d740 4 client.16944 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559baffea8a0 con 0x559bafc62da0 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f098effd640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc62da0 0x559bb0100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f098effd640 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc62da0 0x559bb0100e40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46d2634e21a2edca server_cookie=ff184bb9abc9e991 in_seq=0 out_seq=0 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f096effd640 10 client.16944 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f096effd640 1 -- 172.21.15.31:0/4181085429 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559baffea8a0 con 0x559bafc62da0 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.289+0000 7f096effd640 10 client.16944 handle_command_reply: tid=0 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 client.16944 shutdown 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 msgr2=0x559bb0085f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 0x559bb0085f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f09880e5280 tx=0x7f09880ad7e0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 shutdown_connections 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bafccf070 0x559bb0085f90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559bafcae390 0x559bb0085a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x559bafc62da0 0x559bb0100e40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f096effd640 0 client.16944 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 --2- 172.21.15.31:0/4181085429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559bafca61b0 0x559bb008a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 >> 172.21.15.31:0/4181085429 conn(0x559bafcc3460 msgr2=0x559bb00776d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 shutdown_connections 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 1 -- 172.21.15.31:0/4181085429 wait complete. 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f099be1d740 20 client.16944 trim_cache size 0 max 16384 2024-09-21T02:59:04.296 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f09780b66b0 msgr2=0x7f09780b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f09958a9640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f09780b66b0 0x7f09780b8bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0980031440 tx=0x7f098003c040 comp rx=0 tx=0).stop 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 msgr2=0x7f099019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.290+0000 7f09958a9640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 0x7f099019fd50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f098800cce0 tx=0x7f098800a620 comp rx=0 tx=0).stop 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 shutdown_connections 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f09780b66b0 0x7f09780b8bc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09901052a0 0x7f099019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f099010de70 0x7f099019fd50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 --2- 172.21.15.31:0/2611346558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09901048c0 0x7f099019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 >> 172.21.15.31:0/2611346558 conn(0x7f0990079760 msgr2=0x7f0990107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 shutdown_connections 2024-09-21T02:59:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:04.291+0000 7f09958a9640 1 -- 172.21.15.31:0/2611346558 wait complete. 2024-09-21T02:59:04.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:04 smithi178 ceph-mon[34759]: pgmap v510: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:59:05.005 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:04 smithi031 ceph-mon[31209]: pgmap v510: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:59:05.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:59:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:04 smithi165 ceph-mon[34818]: pgmap v510: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 23 op/s 2024-09-21T02:59:05.295 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.811+0000 7f11f934b640 1 -- 172.21.15.31:0/1626086607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f41094a0 msgr2=0x7f11f41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.811+0000 7f11f934b640 1 --2- 172.21.15.31:0/1626086607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f41094a0 0x7f11f41098b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009900 tx=0x7f11dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 -- 172.21.15.31:0/1626086607 shutdown_connections 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 --2- 172.21.15.31:0/1626086607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f4109df0 0x7f11f4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 --2- 172.21.15.31:0/1626086607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f41094a0 0x7f11f41098b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 --2- 172.21.15.31:0/1626086607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f4103b60 0x7f11f4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 -- 172.21.15.31:0/1626086607 >> 172.21.15.31:0/1626086607 conn(0x7f11f407b850 msgr2=0x7f11f407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 -- 172.21.15.31:0/1626086607 shutdown_connections 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 -- 172.21.15.31:0/1626086607 wait complete. 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.812+0000 7f11f934b640 1 Processor -- start 2024-09-21T02:59:05.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 -- start start 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f4103b60 0x7f11f419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f41094a0 0x7f11f419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 0x7f11f419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4113ab0 con 0x7f11f4109df0 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4113930 con 0x7f11f4103b60 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f934b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4113c30 con 0x7f11f41094a0 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 0x7f11f419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 0x7f11f419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35032/0 (socket says 172.21.15.31:35032) 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 -- 172.21.15.31:0/189091800 learned_addr learned my addr 172.21.15.31:0/189091800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f4103b60 0x7f11f419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f27fc640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f41094a0 0x7f11f419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 -- 172.21.15.31:0/189091800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f41094a0 msgr2=0x7f11f419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f41094a0 0x7f11f419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 -- 172.21.15.31:0/189091800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f4103b60 msgr2=0x7f11f419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f4103b60 0x7f11f419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 -- 172.21.15.31:0/189091800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41a0410 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.813+0000 7f11f37fe640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 0x7f11f419fda0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d750 tx=0x7f11ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.814+0000 7f11ebfff640 1 -- 172.21.15.31:0/189091800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec017070 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.814+0000 7f11ebfff640 1 -- 172.21.15.31:0/189091800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec0054d0 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.814+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41a0700 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.814+0000 7f11ebfff640 1 -- 172.21.15.31:0/189091800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec004c60 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.814+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41a0bc0 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.816+0000 7f11ff8bf740 1 Processor -- start 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ebfff640 1 -- 172.21.15.31:0/189091800 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f11ec010460 con 0x7f11f4109df0 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 -- start start 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f32120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558359ea7ea0 con 0x558359b611b0 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558359e972d0 con 0x558359b8a070 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558359e9b0e0 con 0x558359b69390 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f31be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35036/0 (socket says 172.21.15.31:35036) 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.817+0000 7f11f2ffd640 1 -- 172.21.15.31:0/455883933 learned_addr learned my addr 172.21.15.31:0/455883933 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f27fc640 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f32120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f27fc640 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f2ffd640 1 -- 172.21.15.31:0/455883933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 msgr2=0x558359f32120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f32120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f2ffd640 1 -- 172.21.15.31:0/455883933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 msgr2=0x558359f316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f2ffd640 1 -- 172.21.15.31:0/455883933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558359bf5650 con 0x558359b611b0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f27fc640 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11ebfff640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f11e00b66b0 0x7f11e00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f31be0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007f90 tx=0x7f11e400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d045b035de3312 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11f27fc640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f11e00b66b0 0x7f11e00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11ebfff640 1 -- 172.21.15.31:0/189091800 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f11ec0e7040 con 0x7f11f4109df0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11e97fa640 1 -- 172.21.15.31:0/455883933 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4004d20 con 0x558359b611b0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11e97fa640 1 -- 172.21.15.31:0/455883933 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e4010bb0 con 0x558359b611b0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.818+0000 7f11e97fa640 1 -- 172.21.15.31:0/455883933 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e40056b0 con 0x558359b611b0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 -- 172.21.15.31:0/455883933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 msgr2=0x558359f31be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11f27fc640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f11e00b66b0 0x7f11e00b8bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009230 tx=0x7f11dc02a410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f31be0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007f90 tx=0x7f11e400f850 comp rx=0 tx=0).stop 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 -- 172.21.15.31:0/455883933 shutdown_connections 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f32120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f31be0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/455883933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 -- 172.21.15.31:0/455883933 >> 172.21.15.31:0/455883933 conn(0x558359b7e460 msgr2=0x558359f2f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 -- 172.21.15.31:0/455883933 shutdown_connections 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.819+0000 7f11ff8bf740 1 -- 172.21.15.31:0/455883933 wait complete. 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 -- start start 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 Processor -- start 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b69390 0x558359f3d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f3d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558359cf8700 con 0x558359b611b0 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558359e32160 con 0x558359b69390 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558359f3fe80 con 0x558359b8a070 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11ff8bf740 10 client.0 _ll_register_callbacks cb 0x558359e4b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35046/0 (socket says 172.21.15.31:35046) 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11f2ffd640 1 -- 172.21.15.31:0/3268364282 learned_addr learned my addr 172.21.15.31:0/3268364282 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.820+0000 7f11f37fe640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f3d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f27fc640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b69390 0x558359f3d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f2ffd640 1 -- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 msgr2=0x558359f3d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f3d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f2ffd640 1 -- 172.21.15.31:0/3268364282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b69390 msgr2=0x558359f3d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b69390 0x558359f3d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f2ffd640 1 -- 172.21.15.31:0/3268364282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558359e5fc90 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f27fc640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b69390 0x558359f3d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3cd00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007b60 tx=0x7f11e4010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e71261193314c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11da7fc640 1 -- 172.21.15.31:0/3268364282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4026020 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e4010bb0 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11da7fc640 1 -- 172.21.15.31:0/3268364282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f11e4004d80 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11da7fc640 1 -- 172.21.15.31:0/3268364282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11e4031050 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11ff8bf740 10 client.16956 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558359f4ba40 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.821+0000 7f11da7fc640 1 -- 172.21.15.31:0/3268364282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e4010bb0 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11da7fc640 1 -- 172.21.15.31:0/3268364282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f11e4022830 con 0x558359b611b0 2024-09-21T02:59:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11ff8bf740 10 client.16956 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11ff8bf740 10 client.16956 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11ff8bf740 20 client.16956 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b489f0 0x558359f4ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11ff8bf740 4 client.16956 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558359ea9df0 con 0x558359b489f0 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.822+0000 7f11f27fc640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b489f0 0x558359f4ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.823+0000 7f11f27fc640 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b489f0 0x558359f4ee50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff5637336d2f276f server_cookie=bd1aadfb50c10e1 in_seq=0 out_seq=0 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.823+0000 7f11da7fc640 10 client.16956 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.823+0000 7f11da7fc640 1 -- 172.21.15.31:0/3268364282 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558359ea9df0 con 0x558359b489f0 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.823+0000 7f11da7fc640 10 client.16956 handle_command_reply: tid=0 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 client.16956 shutdown 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 msgr2=0x558359f3cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3cd00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007b60 tx=0x7f11e4010f90 comp rx=0 tx=0).stop 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 shutdown_connections 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b489f0 0x558359f4ee50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f3d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b69390 0x558359f3d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/3268364282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3cd00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11da7fc640 0 client.16956 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 >> 172.21.15.31:0/3268364282 conn(0x558359b7e460 msgr2=0x558359f34880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 shutdown_connections 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 1 -- 172.21.15.31:0/3268364282 wait complete. 2024-09-21T02:59:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.824+0000 7f11ff8bf740 20 client.16956 trim_cache size 0 max 16384 2024-09-21T02:59:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 Processor -- start 2024-09-21T02:59:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 -- start start 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f34eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f2ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41af6e0 con 0x558359b611b0 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41b1220 con 0x558359b8a070 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.838+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41afdb0 con 0x558359b69390 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35048/0 (socket says 172.21.15.31:35048) 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 -- 172.21.15.31:0/1804182187 learned_addr learned my addr 172.21.15.31:0/1804182187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f27fc640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f34eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f37fe640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f2ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 -- 172.21.15.31:0/1804182187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 msgr2=0x558359f34eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f34eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 -- 172.21.15.31:0/1804182187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 msgr2=0x558359f2ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f2ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 -- 172.21.15.31:0/1804182187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558359f3dfc0 con 0x558359b611b0 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f37fe640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f2ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f27fc640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f34eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3ebe0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007eb0 tx=0x7f11e40075b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a744dede98116a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11e8ff9640 1 -- 172.21.15.31:0/1804182187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4026040 con 0x558359b611b0 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11e8ff9640 1 -- 172.21.15.31:0/1804182187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e403a070 con 0x558359b611b0 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.839+0000 7f11e8ff9640 1 -- 172.21.15.31:0/1804182187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e401d040 con 0x558359b611b0 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 -- 172.21.15.31:0/1804182187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 msgr2=0x558359f3ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3ebe0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f11e4007eb0 tx=0x7f11e40075b0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 -- 172.21.15.31:0/1804182187 shutdown_connections 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b69390 0x558359f34eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b8a070 0x558359f2ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/1804182187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b611b0 0x558359f3ebe0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 -- 172.21.15.31:0/1804182187 >> 172.21.15.31:0/1804182187 conn(0x558359b7e460 msgr2=0x558359f347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 -- 172.21.15.31:0/1804182187 shutdown_connections 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.840+0000 7f11ff8bf740 1 -- 172.21.15.31:0/1804182187 wait complete. 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 -- start start 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 Processor -- start 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b611b0 0x558359f52460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 0x558359f4f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f4fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41b0b50 con 0x558359b69390 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41b0480 con 0x558359b611b0 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41ad6c0 con 0x558359b8a070 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11ff8bf740 10 client.0 _ll_register_callbacks cb 0x7f11e4007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11f27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 0x558359f4f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11f27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 0x558359f4f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35058/0 (socket says 172.21.15.31:35058) 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.841+0000 7f11f27fc640 1 -- 172.21.15.31:0/800141361 learned_addr learned my addr 172.21.15.31:0/800141361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f37fe640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f4fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b611b0 0x558359f52460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f27fc640 1 -- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 msgr2=0x558359f4fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f27fc640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f4fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f27fc640 1 -- 172.21.15.31:0/800141361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b611b0 msgr2=0x558359f52460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f27fc640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b611b0 0x558359f52460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f27fc640 1 -- 172.21.15.31:0/800141361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558359f50430 con 0x558359b69390 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b611b0 0x558359f52460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11f27fc640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 0x558359f4f930 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009690 tx=0x7f11dc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6641f3f7cee9dd2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11da7fc640 1 -- 172.21.15.31:0/800141361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc035040 con 0x558359b69390 2024-09-21T02:59:05.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11da7fc640 1 -- 172.21.15.31:0/800141361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f11dc004240 con 0x558359b69390 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e403a070 con 0x558359b69390 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11da7fc640 1 -- 172.21.15.31:0/800141361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11dc0318f0 con 0x558359b69390 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11ff8bf740 10 client.16962 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.842+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558359fb9360 con 0x558359b69390 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11da7fc640 1 -- 172.21.15.31:0/800141361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e403a070 con 0x558359b69390 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11da7fc640 1 -- 172.21.15.31:0/800141361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f11dc03a040 con 0x558359b69390 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11ff8bf740 10 client.16962 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11ff8bf740 10 client.16962 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11ff8bf740 20 client.16962 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b1dda0 0x558359fbbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11ff8bf740 4 client.16962 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558359ea58a0 con 0x558359b1dda0 2024-09-21T02:59:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.843+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b1dda0 0x558359fbbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11f2ffd640 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b1dda0 0x558359fbbb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b46618ef644889f7 server_cookie=9ccadfb72321fdeb in_seq=0 out_seq=0 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11da7fc640 10 client.16962 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11da7fc640 1 -- 172.21.15.31:0/800141361 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558359ea58a0 con 0x558359b1dda0 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11da7fc640 10 client.16962 handle_command_reply: tid=0 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 client.16962 shutdown 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 msgr2=0x558359f4f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 0x558359f4f930 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009690 tx=0x7f11dc031400 comp rx=0 tx=0).stop 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 shutdown_connections 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x558359b1dda0 0x558359fbbb00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558359b8a070 0x558359f4fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558359b69390 0x558359f4f930 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11da7fc640 0 client.16962 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 --2- 172.21.15.31:0/800141361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558359b611b0 0x558359f52460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 >> 172.21.15.31:0/800141361 conn(0x558359b7e460 msgr2=0x558359f35160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 shutdown_connections 2024-09-21T02:59:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 1 -- 172.21.15.31:0/800141361 wait complete. 2024-09-21T02:59:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.844+0000 7f11ff8bf740 20 client.16962 trim_cache size 0 max 16384 2024-09-21T02:59:05.852 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f11e00b66b0 msgr2=0x7f11e00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f11e00b66b0 0x7f11e00b8bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009230 tx=0x7f11dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 msgr2=0x7f11f419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 0x7f11f419fda0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d750 tx=0x7f11ec00dc10 comp rx=0 tx=0).stop 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 shutdown_connections 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f11e00b66b0 0x7f11e00b8bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f41094a0 0x7f11f419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f4109df0 0x7f11f419fda0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 --2- 172.21.15.31:0/189091800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f4103b60 0x7f11f419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 >> 172.21.15.31:0/189091800 conn(0x7f11f407b850 msgr2=0x7f11f41075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 shutdown_connections 2024-09-21T02:59:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:05.845+0000 7f11f934b640 1 -- 172.21.15.31:0/189091800 wait complete. 2024-09-21T02:59:06.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:59:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:06.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:06 smithi031 ceph-mon[31209]: pgmap v511: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:06 smithi165 ceph-mon[34818]: pgmap v511: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:06 smithi178 ceph-mon[34759]: pgmap v511: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 -- 172.21.15.31:0/3655940260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc1094c0 msgr2=0x7f5ffc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 --2- 172.21.15.31:0/3655940260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc1094c0 0x7f5ffc1098d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009900 tx=0x7f5fe802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 -- 172.21.15.31:0/3655940260 shutdown_connections 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 --2- 172.21.15.31:0/3655940260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ffc109e10 0x7f5ffc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 --2- 172.21.15.31:0/3655940260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc1094c0 0x7f5ffc1098d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 --2- 172.21.15.31:0/3655940260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc101fe0 0x7f5ffc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 -- 172.21.15.31:0/3655940260 >> 172.21.15.31:0/3655940260 conn(0x7f5ffc077bc0 msgr2=0x7f5ffc078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 -- 172.21.15.31:0/3655940260 shutdown_connections 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.253+0000 7f6002b23640 1 -- 172.21.15.31:0/3655940260 wait complete. 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 Processor -- start 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 -- start start 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc101fe0 0x7f5ffc102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ffc1094c0 0x7f5ffc1034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 0x7f5ffc103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc113ad0 con 0x7f5ffc109e10 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc113950 con 0x7f5ffc1094c0 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f6002b23640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc113c50 con 0x7f5ffc101fe0 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f60010c0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 0x7f5ffc103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f60010c0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 0x7f5ffc103a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35076/0 (socket says 172.21.15.31:35076) 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f60010c0640 1 -- 172.21.15.31:0/4165800169 learned_addr learned my addr 172.21.15.31:0/4165800169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f60008bf640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc101fe0 0x7f5ffc102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.254+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ffc1094c0 0x7f5ffc1034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f60010c0640 1 -- 172.21.15.31:0/4165800169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc101fe0 msgr2=0x7f5ffc102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f60010c0640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc101fe0 0x7f5ffc102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f60010c0640 1 -- 172.21.15.31:0/4165800169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ffc1094c0 msgr2=0x7f5ffc1034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f60010c0640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ffc1094c0 0x7f5ffc1034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f60010c0640 1 -- 172.21.15.31:0/4165800169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc1a8800 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f60010c0640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 0x7f5ffc103a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400eaa0 tx=0x7f5ff400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f5ff9ffb640 1 -- 172.21.15.31:0/4165800169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400cdb0 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f5ff9ffb640 1 -- 172.21.15.31:0/4165800169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff4005020 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc1a8af0 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f5ff9ffb640 1 -- 172.21.15.31:0/4165800169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40116d0 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.255+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc1a9000 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 Processor -- start 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f5ff9ffb640 1 -- 172.21.15.31:0/4165800169 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5ff40051d0 con 0x7f5ffc109e10 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 -- start start 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 0x56499a2682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e981b0 0x56499a268800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a268d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56499a1deea0 con 0x564999ec1070 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56499a1ce2d0 con 0x564999e981b0 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f6009097740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56499a1d20e0 con 0x564999ea0390 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 0x56499a2682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 0x56499a2682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35088/0 (socket says 172.21.15.31:35088) 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.258+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1842850564 learned_addr learned my addr 172.21.15.31:0/1842850564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f60008bf640 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e981b0 0x56499a268800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ff9ffb640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5fec0b6700 0x7f5fec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1842850564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 msgr2=0x56499a268d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a268d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1842850564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e981b0 msgr2=0x56499a268800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e981b0 0x56499a268800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1842850564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564999f2c650 con 0x564999ec1070 2024-09-21T02:59:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f60010c0640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5fec0b6700 0x7f5fec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 0x56499a2682c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006010 tx=0x7f5fe8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29332227134281d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5ff9ffb640 1 -- 172.21.15.31:0/4165800169 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5ff40e77e0 con 0x7f5ffc109e10 2024-09-21T02:59:07.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.259+0000 7f5fe6ffd640 1 -- 172.21.15.31:0/1842850564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8035030 con 0x564999ec1070 2024-09-21T02:59:07.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f5fe6ffd640 1 -- 172.21.15.31:0/1842850564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe80054f0 con 0x564999ec1070 2024-09-21T02:59:07.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f5fe6ffd640 1 -- 172.21.15.31:0/1842850564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80035f0 con 0x564999ec1070 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 -- 172.21.15.31:0/1842850564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 msgr2=0x56499a2682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f60010c0640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5fec0b6700 0x7f5fec0b8c10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40e5000 tx=0x7f5ff407a930 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 0x56499a2682c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006010 tx=0x7f5fe8004180 comp rx=0 tx=0).stop 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 -- 172.21.15.31:0/1842850564 shutdown_connections 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a268d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e981b0 0x56499a268800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 --2- 172.21.15.31:0/1842850564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999ec1070 0x56499a2682c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.260+0000 7f6009097740 1 -- 172.21.15.31:0/1842850564 >> 172.21.15.31:0/1842850564 conn(0x564999eb5460 msgr2=0x56499a2661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.261+0000 7f6009097740 1 -- 172.21.15.31:0/1842850564 shutdown_connections 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.261+0000 7f6009097740 1 -- 172.21.15.31:0/1842850564 wait complete. 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.261+0000 7f6009097740 1 -- start start 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.261+0000 7f6009097740 1 Processor -- start 2024-09-21T02:59:07.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a273a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a273f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ec1070 0x56499a27fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56499a02f700 con 0x564999e981b0 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56499a169160 con 0x564999ea0390 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56499a276a40 con 0x564999ec1070 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f6009097740 10 client.0 _ll_register_callbacks cb 0x56499a1829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a273a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a273a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35100/0 (socket says 172.21.15.31:35100) 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1363408960 learned_addr learned my addr 172.21.15.31:0/1363408960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f60008bf640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a273f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ec1070 0x56499a27fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ec1070 msgr2=0x56499a27fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ec1070 0x56499a27fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1363408960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 msgr2=0x56499a273f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a273f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1363408960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499a196c90 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a273a40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8004c20 tx=0x7f5fe80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910c3aab2f90d6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.262+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1363408960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8035070 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1363408960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5fe80035f0 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe80054f0 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1363408960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5fe8058050 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 10 client.16974 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56499a2827e0 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1363408960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe80054f0 con 0x564999e981b0 2024-09-21T02:59:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1363408960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5fe8050020 con 0x564999e981b0 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 10 client.16974 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 20 client.16974 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999e7f9f0 0x56499a285b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 4 client.16974 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.263+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499a1e0df0 con 0x564999e7f9f0 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.264+0000 7f60008bf640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999e7f9f0 0x56499a285b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.264+0000 7f5fdbfff640 10 client.16974 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.264+0000 7f60008bf640 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999e7f9f0 0x56499a285b40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=758f8326eb9a9f1b server_cookie=d300d11a69406a16 in_seq=0 out_seq=0 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1363408960 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56499a1e0df0 con 0x564999e7f9f0 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f5fdbfff640 10 client.16974 handle_command_reply: tid=0 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 client.16974 shutdown 2024-09-21T02:59:07.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 msgr2=0x56499a273a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a273a40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8004c20 tx=0x7f5fe80058b0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 shutdown_connections 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999e7f9f0 0x56499a285b40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ec1070 0x56499a27fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a273f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 --2- 172.21.15.31:0/1363408960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a273a40 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f5fdbfff640 0 client.16974 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 >> 172.21.15.31:0/1363408960 conn(0x564999eb5460 msgr2=0x56499a26b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 shutdown_connections 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 1 -- 172.21.15.31:0/1363408960 wait complete. 2024-09-21T02:59:07.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.265+0000 7f6009097740 20 client.16974 trim_cache size 0 max 16384 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 Processor -- start 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 -- start start 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 0x56499a275670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999e981b0 0x56499a270c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a271170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b1830 con 0x564999e7f9f0 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b3370 con 0x564999ea0390 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f6009097740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b1f00 con 0x564999e981b0 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 0x56499a275670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 0x56499a275670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35104/0 (socket says 172.21.15.31:35104) 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1511715261 learned_addr learned my addr 172.21.15.31:0/1511715261 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f60008bf640 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999e981b0 0x56499a270c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1511715261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999e981b0 msgr2=0x56499a270c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999e981b0 0x56499a270c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1511715261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 msgr2=0x56499a271170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a271170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.279+0000 7f5ffbfff640 1 -- 172.21.15.31:0/1511715261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499a26aa10 con 0x564999e7f9f0 2024-09-21T02:59:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.280+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 0x56499a275670 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80092a0 tx=0x7f5fe8005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115fe2a816b3d009 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.280+0000 7f5fe67fc640 1 -- 172.21.15.31:0/1511715261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8035040 con 0x564999e7f9f0 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.280+0000 7f5fe67fc640 1 -- 172.21.15.31:0/1511715261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe8047020 con 0x564999e7f9f0 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.280+0000 7f5fe67fc640 1 -- 172.21.15.31:0/1511715261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8031700 con 0x564999e7f9f0 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.280+0000 7f6009097740 1 -- 172.21.15.31:0/1511715261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 msgr2=0x56499a275670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.280+0000 7f6009097740 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 0x56499a275670 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80092a0 tx=0x7f5fe8005980 comp rx=0 tx=0).stop 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 -- 172.21.15.31:0/1511715261 shutdown_connections 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999e981b0 0x56499a270c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999ea0390 0x56499a271170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 --2- 172.21.15.31:0/1511715261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e7f9f0 0x56499a275670 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 -- 172.21.15.31:0/1511715261 >> 172.21.15.31:0/1511715261 conn(0x564999eb5460 msgr2=0x56499a274f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 -- 172.21.15.31:0/1511715261 shutdown_connections 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 -- 172.21.15.31:0/1511715261 wait complete. 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 -- start start 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.281+0000 7f6009097740 1 Processor -- start 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e7f9f0 0x56499a288bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a2891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a2e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b2ca0 con 0x564999e981b0 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1b25d0 con 0x564999e7f9f0 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc1af810 con 0x564999ea0390 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f6009097740 10 client.0 _ll_register_callbacks cb 0x7f5ff0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a2891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a2891d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35110/0 (socket says 172.21.15.31:35110) 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 -- 172.21.15.31:0/1166932323 learned_addr learned my addr 172.21.15.31:0/1166932323 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e7f9f0 0x56499a288bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a2e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 -- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 msgr2=0x56499a2e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a2e8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 -- 172.21.15.31:0/1166932323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e7f9f0 msgr2=0x56499a288bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e7f9f0 0x56499a288bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 -- 172.21.15.31:0/1166932323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499a2e8ff0 con 0x564999e981b0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e7f9f0 0x56499a288bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f60008bf640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a2891d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc1049d0 tx=0x7f5ff00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eee0b72a0cedcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f5fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.282+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1166932323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0029060 con 0x564999e981b0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1166932323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f5ff001d870 con 0x564999e981b0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe8047020 con 0x564999e981b0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1166932323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ff000be30 con 0x564999e981b0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 10 client.16980 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56499a2f0450 con 0x564999e981b0 2024-09-21T02:59:07.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1166932323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe8047020 con 0x564999e981b0 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1166932323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5ff0011bf0 con 0x564999e981b0 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 10 client.16980 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 10 client.16980 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 20 client.16980 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999ec1070 0x56499a2f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 4 client.16980 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56499a1dc8a0 con 0x564999ec1070 2024-09-21T02:59:07.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.283+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999ec1070 0x56499a2f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f5ffbfff640 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999ec1070 0x56499a2f2bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a24af5bf45879b7 server_cookie=11599a695deaf272 in_seq=0 out_seq=0 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f5fdbfff640 10 client.16980 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f5fdbfff640 1 -- 172.21.15.31:0/1166932323 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56499a1dc8a0 con 0x564999ec1070 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f5fdbfff640 10 client.16980 handle_command_reply: tid=0 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f6009097740 1 client.16980 shutdown 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 msgr2=0x56499a2891d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f6009097740 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a2891d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc1049d0 tx=0x7f5ff00039c0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 shutdown_connections 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f6009097740 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x564999ec1070 0x56499a2f2bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.284+0000 7f6009097740 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564999ea0390 0x56499a2e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6009097740 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564999e981b0 0x56499a2891d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6009097740 1 --2- 172.21.15.31:0/1166932323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564999e7f9f0 0x56499a288bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f5fdbfff640 0 client.16980 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 >> 172.21.15.31:0/1166932323 conn(0x564999eb5460 msgr2=0x56499a274e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 shutdown_connections 2024-09-21T02:59:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6009097740 1 -- 172.21.15.31:0/1166932323 wait complete. 2024-09-21T02:59:07.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6009097740 20 client.16980 trim_cache size 0 max 16384 2024-09-21T02:59:07.295 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5fec0b6700 msgr2=0x7f5fec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6002b23640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5fec0b6700 0x7f5fec0b8c10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40e5000 tx=0x7f5ff407a930 comp rx=0 tx=0).stop 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 msgr2=0x7f5ffc103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.285+0000 7f6002b23640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 0x7f5ffc103a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400eaa0 tx=0x7f5ff400ef60 comp rx=0 tx=0).stop 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 shutdown_connections 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5fec0b6700 0x7f5fec0b8c10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ffc109e10 0x7f5ffc103a00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ffc1094c0 0x7f5ffc1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 --2- 172.21.15.31:0/4165800169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ffc101fe0 0x7f5ffc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:07.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 >> 172.21.15.31:0/4165800169 conn(0x7f5ffc077bc0 msgr2=0x7f5ffc107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 shutdown_connections 2024-09-21T02:59:07.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:07.286+0000 7f6002b23640 1 -- 172.21.15.31:0/4165800169 wait complete. 2024-09-21T02:59:08.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:59:08.346 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 -- 172.21.15.31:0/922928318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64103b60 msgr2=0x7f3d64103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 --2- 172.21.15.31:0/922928318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64103b60 0x7f3d64103f70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009900 tx=0x7f3d6002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 -- 172.21.15.31:0/922928318 shutdown_connections 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 --2- 172.21.15.31:0/922928318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d641094a0 0x7f3d641098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 --2- 172.21.15.31:0/922928318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d64109df0 0x7f3d64110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 --2- 172.21.15.31:0/922928318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64103b60 0x7f3d64103f70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 -- 172.21.15.31:0/922928318 >> 172.21.15.31:0/922928318 conn(0x7f3d6407b850 msgr2=0x7f3d6407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 -- 172.21.15.31:0/922928318 shutdown_connections 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 -- 172.21.15.31:0/922928318 wait complete. 2024-09-21T02:59:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.875+0000 7f3d6b942640 1 Processor -- start 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 -- start start 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d64103b60 0x7f3d64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d641094a0 0x7f3d64075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 0x7f3d64076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3d64113ab0 con 0x7f3d64109df0 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d64113930 con 0x7f3d641094a0 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d6b942640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d64113c30 con 0x7f3d64103b60 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 0x7f3d64076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 0x7f3d64076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35132/0 (socket says 172.21.15.31:35132) 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 -- 172.21.15.31:0/659886240 learned_addr learned my addr 172.21.15.31:0/659886240 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d696de640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d64103b60 0x7f3d64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d68edd640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d641094a0 0x7f3d64075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 -- 172.21.15.31:0/659886240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d64103b60 msgr2=0x7f3d64075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d64103b60 0x7f3d64075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 -- 172.21.15.31:0/659886240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d641094a0 msgr2=0x7f3d64075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d641094a0 0x7f3d64075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d69edf640 1 -- 172.21.15.31:0/659886240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d641a8760 con 0x7f3d64109df0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.876+0000 7f3d68edd640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d641094a0 0x7f3d64075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.877+0000 7f3d69edf640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 0x7f3d64076190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00eaa0 tx=0x7f3d5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.877+0000 7f3d5a7fc640 1 -- 172.21.15.31:0/659886240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00cdb0 con 0x7f3d64109df0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.877+0000 7f3d5a7fc640 1 -- 172.21.15.31:0/659886240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5c005020 con 0x7f3d64109df0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.877+0000 7f3d5a7fc640 1 -- 172.21.15.31:0/659886240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0116d0 con 0x7f3d64109df0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.877+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d641a8a50 con 0x7f3d64109df0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.877+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d641a8f40 con 0x7f3d64109df0 2024-09-21T02:59:08.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d5a7fc640 1 -- 172.21.15.31:0/659886240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3d5c0051d0 con 0x7f3d64109df0 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 Processor -- start 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 -- start start 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0f8070 0x55884e49f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e49fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55884e415ea0 con 0x55884e0d7390 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55884e4052d0 con 0x55884e0f8070 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55884e4090e0 con 0x55884e0cf1b0 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35140/0 (socket says 172.21.15.31:35140) 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d68edd640 1 -- 172.21.15.31:0/1774885750 learned_addr learned my addr 172.21.15.31:0/1774885750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d68edd640 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0f8070 0x55884e49f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.880+0000 7f3d5a7fc640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d500b6700 0x7f3d500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d696de640 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e49fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d69edf640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d500b6700 0x7f3d500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d68edd640 1 -- 172.21.15.31:0/1774885750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 msgr2=0x55884e49fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d68edd640 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e49fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d68edd640 1 -- 172.21.15.31:0/1774885750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0f8070 msgr2=0x55884e49f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d68edd640 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0f8070 0x55884e49f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d68edd640 1 -- 172.21.15.31:0/1774885750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884e163650 con 0x55884e0d7390 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d5a7fc640 1 -- 172.21.15.31:0/659886240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3d5c0e7890 con 0x7f3d64109df0 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d68edd640 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4a0120 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009fc0 tx=0x7f3d54009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e316f07661d4c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1774885750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540150f0 con 0x55884e0d7390 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d69edf640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d500b6700 0x7f3d500b8c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0ad400 tx=0x7f3d5c07a3b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1774885750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d540054d0 con 0x55884e0d7390 2024-09-21T02:59:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.881+0000 7f3d4f7fe640 1 -- 172.21.15.31:0/1774885750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54004e10 con 0x55884e0d7390 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 -- 172.21.15.31:0/1774885750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 msgr2=0x55884e4a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4a0120 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009fc0 tx=0x7f3d54009670 comp rx=0 tx=0).stop 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 -- 172.21.15.31:0/1774885750 shutdown_connections 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e49fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4a0120 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/1774885750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0f8070 0x55884e49f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 -- 172.21.15.31:0/1774885750 >> 172.21.15.31:0/1774885750 conn(0x55884e0ec460 msgr2=0x55884e49d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 -- 172.21.15.31:0/1774885750 shutdown_connections 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.882+0000 7f3d71eb6740 1 -- 172.21.15.31:0/1774885750 wait complete. 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 -- start start 2024-09-21T02:59:08.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 Processor -- start 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4aae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4ab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0f8070 0x55884e4b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55884e266700 con 0x55884e0d7390 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55884e3a0160 con 0x55884e0cf1b0 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55884e4ade80 con 0x55884e0f8070 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d71eb6740 10 client.0 _ll_register_callbacks cb 0x55884e3b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d696de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d696de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4ab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35150/0 (socket says 172.21.15.31:35150) 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.883+0000 7f3d696de640 1 -- 172.21.15.31:0/927863205 learned_addr learned my addr 172.21.15.31:0/927863205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d68edd640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0f8070 0x55884e4b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d68edd640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d696de640 1 -- 172.21.15.31:0/927863205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0f8070 msgr2=0x55884e4b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d696de640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0f8070 0x55884e4b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d696de640 1 -- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 msgr2=0x55884e4aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d696de640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d696de640 1 -- 172.21.15.31:0/927863205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884e3cdc90 con 0x55884e0d7390 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d68edd640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d696de640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4ab370 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005ab0 tx=0x7f3d60031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59c70eb4d04922d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d4cff9640 1 -- 172.21.15.31:0/927863205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60035040 con 0x55884e0d7390 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d4cff9640 1 -- 172.21.15.31:0/927863205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3d60004240 con 0x55884e0d7390 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d540054d0 con 0x55884e0d7390 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d4cff9640 1 -- 172.21.15.31:0/927863205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d60004440 con 0x55884e0d7390 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d71eb6740 10 client.16992 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.884+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55884e4b9c60 con 0x55884e0d7390 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d4cff9640 1 -- 172.21.15.31:0/927863205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d540054d0 con 0x55884e0d7390 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d4cff9640 1 -- 172.21.15.31:0/927863205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d60004b80 con 0x55884e0d7390 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d71eb6740 10 client.16992 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d71eb6740 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d71eb6740 20 client.16992 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e0b69f0 0x55884e4bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d71eb6740 4 client.16992 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55884e417df0 con 0x55884e0b69f0 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.885+0000 7f3d68edd640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e0b69f0 0x55884e4bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.886+0000 7f3d68edd640 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e0b69f0 0x55884e4bd0e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aad0a247235bf9ed server_cookie=2c3be5596fadc90b in_seq=0 out_seq=0 2024-09-21T02:59:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.886+0000 7f3d4cff9640 10 client.16992 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d4cff9640 1 -- 172.21.15.31:0/927863205 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55884e417df0 con 0x55884e0b69f0 2024-09-21T02:59:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d4cff9640 10 client.16992 handle_command_reply: tid=0 2024-09-21T02:59:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 client.16992 shutdown 2024-09-21T02:59:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 msgr2=0x55884e4ab370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4ab370 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005ab0 tx=0x7f3d60031740 comp rx=0 tx=0).stop 2024-09-21T02:59:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 shutdown_connections 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e0b69f0 0x55884e4bd0e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0f8070 0x55884e4b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0d7390 0x55884e4ab370 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/927863205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 >> 172.21.15.31:0/927863205 conn(0x55884e0ec460 msgr2=0x55884e4a2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d4cff9640 0 client.16992 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.887+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 shutdown_connections 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.888+0000 7f3d71eb6740 1 -- 172.21.15.31:0/927863205 wait complete. 2024-09-21T02:59:08.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.888+0000 7f3d71eb6740 20 client.16992 trim_cache size 0 max 16384 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 Processor -- start 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 -- start start 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0d7390 0x55884e4a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e4a8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b1920 con 0x55884e0f8070 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b3460 con 0x55884e0cf1b0 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.901+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b1ff0 con 0x55884e0d7390 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e4a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e4a8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35154/0 (socket says 172.21.15.31:35154) 2024-09-21T02:59:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 -- 172.21.15.31:0/3126157499 learned_addr learned my addr 172.21.15.31:0/3126157499 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d696de640 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0d7390 0x55884e4a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 -- 172.21.15.31:0/3126157499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0d7390 msgr2=0x55884e4a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0d7390 0x55884e4a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 -- 172.21.15.31:0/3126157499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 msgr2=0x55884e4a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 -- 172.21.15.31:0/3126157499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884e4a1f30 con 0x55884e0f8070 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d696de640 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0d7390 0x55884e4a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e4a8560 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3d54015f90 tx=0x7f3d54017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c27d0a5c35f176 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d4effd640 1 -- 172.21.15.31:0/3126157499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54004680 con 0x55884e0f8070 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d4effd640 1 -- 172.21.15.31:0/3126157499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d540150f0 con 0x55884e0f8070 2024-09-21T02:59:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.902+0000 7f3d4effd640 1 -- 172.21.15.31:0/3126157499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54003910 con 0x55884e0f8070 2024-09-21T02:59:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3126157499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 msgr2=0x55884e4a8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e4a8560 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3d54015f90 tx=0x7f3d54017040 comp rx=0 tx=0).stop 2024-09-21T02:59:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3126157499 shutdown_connections 2024-09-21T02:59:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0d7390 0x55884e4a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e4a8560 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3126157499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0cf1b0 0x55884e4a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3126157499 >> 172.21.15.31:0/3126157499 conn(0x55884e0ec460 msgr2=0x55884e4a06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3126157499 shutdown_connections 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.903+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3126157499 wait complete. 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 -- start start 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 Processor -- start 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e4b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0d7390 0x55884e4b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e51fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b2d90 con 0x55884e0f8070 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641b26c0 con 0x55884e0d7390 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d641af900 con 0x55884e0cf1b0 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.904+0000 7f3d71eb6740 10 client.0 _ll_register_callbacks cb 0x7f3d60005e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e51fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e4b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e4b0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:53146/0 (socket says 172.21.15.31:53146) 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 -- 172.21.15.31:0/3429803159 learned_addr learned my addr 172.21.15.31:0/3429803159 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d696de640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0d7390 0x55884e4b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 -- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0d7390 msgr2=0x55884e4b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0d7390 0x55884e4b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 -- 172.21.15.31:0/3429803159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 msgr2=0x55884e51fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e51fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 -- 172.21.15.31:0/3429803159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55884e520150 con 0x55884e0cf1b0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d696de640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0d7390 0x55884e4b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d68edd640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e4b0a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009790 tx=0x7f3d54000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7db9d3be71ffc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.905+0000 7f3d4cff9640 1 -- 172.21.15.31:0/3429803159 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5403f060 con 0x55884e0cf1b0 2024-09-21T02:59:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d4cff9640 1 -- 172.21.15.31:0/3429803159 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3d540104d0 con 0x55884e0cf1b0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d540150f0 con 0x55884e0cf1b0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d4cff9640 1 -- 172.21.15.31:0/3429803159 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d540047e0 con 0x55884e0cf1b0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 10 client.26569 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55884e527490 con 0x55884e0cf1b0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d4cff9640 1 -- 172.21.15.31:0/3429803159 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d540150f0 con 0x55884e0cf1b0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d4cff9640 1 -- 172.21.15.31:0/3429803159 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d5400c070 con 0x55884e0cf1b0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 10 client.26569 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 10 client.26569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 20 client.26569 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e08bda0 0x55884e529a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 4 client.26569 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.906+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55884e4138a0 con 0x55884e08bda0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d696de640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e08bda0 0x55884e529a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d696de640 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e08bda0 0x55884e529a30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ad0acac48aff596 server_cookie=81fc1d1395c3b303 in_seq=0 out_seq=0 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d4cff9640 10 client.26569 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d4cff9640 1 -- 172.21.15.31:0/3429803159 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55884e4138a0 con 0x55884e08bda0 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d4cff9640 10 client.26569 handle_command_reply: tid=0 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d71eb6740 1 client.26569 shutdown 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 msgr2=0x55884e4b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e4b0a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009790 tx=0x7f3d54000bc0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.907+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 shutdown_connections 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55884e08bda0 0x55884e529a30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55884e0f8070 0x55884e51fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d4cff9640 0 client.26569 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55884e0d7390 0x55884e4b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 --2- 172.21.15.31:0/3429803159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55884e0cf1b0 0x55884e4b0a90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 >> 172.21.15.31:0/3429803159 conn(0x55884e0ec460 msgr2=0x55884e4a2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 shutdown_connections 2024-09-21T02:59:08.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 1 -- 172.21.15.31:0/3429803159 wait complete. 2024-09-21T02:59:08.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d71eb6740 20 client.26569 trim_cache size 0 max 16384 2024-09-21T02:59:08.914 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d500b6700 msgr2=0x7f3d500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d500b6700 0x7f3d500b8c10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0ad400 tx=0x7f3d5c07a3b0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 msgr2=0x7f3d64076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 0x7f3d64076190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00eaa0 tx=0x7f3d5c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 shutdown_connections 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3d500b6700 0x7f3d500b8c10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d64109df0 0x7f3d64076190 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d641094a0 0x7f3d64075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 --2- 172.21.15.31:0/659886240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d64103b60 0x7f3d64075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.908+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 >> 172.21.15.31:0/659886240 conn(0x7f3d6407b850 msgr2=0x7f3d64107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.909+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 shutdown_connections 2024-09-21T02:59:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:08.909+0000 7f3d6b942640 1 -- 172.21.15.31:0/659886240 wait complete. 2024-09-21T02:59:09.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:08 smithi031 ceph-mon[31209]: pgmap v512: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:59:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:08 smithi165 ceph-mon[34818]: pgmap v512: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:59:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:08 smithi178 ceph-mon[34759]: pgmap v512: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T02:59:09.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:59:09.815 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:10.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 -- 172.21.15.31:0/3956378460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67741083a0 msgr2=0x7f67741087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 --2- 172.21.15.31:0/3956378460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67741083a0 0x7f67741087b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f675c009900 tx=0x7f675c02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:10.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 -- 172.21.15.31:0/3956378460 shutdown_connections 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 --2- 172.21.15.31:0/3956378460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6774108cf0 0x7f677410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 --2- 172.21.15.31:0/3956378460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67741083a0 0x7f67741087b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 --2- 172.21.15.31:0/3956378460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774102a60 0x7f6774102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 -- 172.21.15.31:0/3956378460 >> 172.21.15.31:0/3956378460 conn(0x7f6774079730 msgr2=0x7f6774079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 -- 172.21.15.31:0/3956378460 shutdown_connections 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.313+0000 7f677b0af640 1 -- 172.21.15.31:0/3956378460 wait complete. 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 Processor -- start 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 -- start start 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 0x7f677410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67741083a0 0x7f67741104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774108cf0 0x7f6774110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113a80 con 0x7f6774102a60 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113900 con 0x7f6774108cf0 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677b0af640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113c00 con 0x7f67741083a0 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 0x7f677410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 0x7f677410ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40968/0 (socket says 172.21.15.31:40968) 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6778e4b640 1 -- 172.21.15.31:0/3564199113 learned_addr learned my addr 172.21.15.31:0/3564199113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6778e4b640 1 -- 172.21.15.31:0/3564199113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67741083a0 msgr2=0x7f67741104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6773fff640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67741083a0 0x7f67741104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6778e4b640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67741083a0 0x7f67741104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f6778e4b640 1 -- 172.21.15.31:0/3564199113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774108cf0 msgr2=0x7f6774110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.314+0000 7f677964c640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774108cf0 0x7f6774110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f6778e4b640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774108cf0 0x7f6774110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f6778e4b640 1 -- 172.21.15.31:0/3564199113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67741a8770 con 0x7f6774102a60 2024-09-21T02:59:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f677964c640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774108cf0 0x7f6774110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f6778e4b640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 0x7f677410ffb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f676400c890 tx=0x7f676400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f6771ffb640 1 -- 172.21.15.31:0/3564199113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004220 con 0x7f6774102a60 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67741a8a60 con 0x7f6774102a60 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f6771ffb640 1 -- 172.21.15.31:0/3564199113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67640043c0 con 0x7f6774102a60 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f6771ffb640 1 -- 172.21.15.31:0/3564199113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004a00 con 0x7f6774102a60 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.315+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a8fa0 con 0x7f6774102a60 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.318+0000 7f6771ffb640 1 -- 172.21.15.31:0/3564199113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f67640079b0 con 0x7f6774102a60 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.318+0000 7f6781623740 1 Processor -- start 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.318+0000 7f6781623740 1 -- start start 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.318+0000 7f6781623740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.318+0000 7f6781623740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x55556245bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.318+0000 7f6781623740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6781623740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5555623d1ea0 con 0x55556208b1b0 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6781623740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5555623c12d0 con 0x5555620b4070 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6781623740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5555623c50e0 con 0x555562093390 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x55556245bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:51080/0 (socket says 172.21.15.31:51080) 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x55556245bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40976/0 (socket says 172.21.15.31:40976) 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 -- 172.21.15.31:0/1053009118 learned_addr learned my addr 172.21.15.31:0/1053009118 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6771ffb640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67600b6680 0x7f67600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6773fff640 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67600b6680 0x7f67600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 -- 172.21.15.31:0/1053009118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 msgr2=0x55556245c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 -- 172.21.15.31:0/1053009118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 msgr2=0x55556245bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6771ffb640 1 -- 172.21.15.31:0/3564199113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f67640e7590 con 0x7f6774102a60 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x55556245bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f677964c640 1 -- 172.21.15.31:0/1053009118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55556211f650 con 0x5555620b4070 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.319+0000 7f6778e4b640 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x55556245bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.320+0000 7f677964c640 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245b6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f676c013910 tx=0x7f676c013dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71a3d23fae27520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.320+0000 7f677964c640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67600b6680 0x7f67600b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f676c00d000 tx=0x7f676c01d000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.320+0000 7f676affd640 1 -- 172.21.15.31:0/1053009118 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c004d20 con 0x5555620b4070 2024-09-21T02:59:10.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.320+0000 7f676affd640 1 -- 172.21.15.31:0/1053009118 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c021030 con 0x5555620b4070 2024-09-21T02:59:10.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.320+0000 7f676affd640 1 -- 172.21.15.31:0/1053009118 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c01d640 con 0x5555620b4070 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 -- 172.21.15.31:0/1053009118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 msgr2=0x55556245b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245b6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f676c013910 tx=0x7f676c013dd0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 -- 172.21.15.31:0/1053009118 shutdown_connections 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x55556245bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 --2- 172.21.15.31:0/1053009118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245b6a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 -- 172.21.15.31:0/1053009118 >> 172.21.15.31:0/1053009118 conn(0x5555620a8460 msgr2=0x555562459590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 -- 172.21.15.31:0/1053009118 shutdown_connections 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.321+0000 7f6781623740 1 -- 172.21.15.31:0/1053009118 wait complete. 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 -- start start 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 Processor -- start 2024-09-21T02:59:10.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x555562466de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555562093390 0x555562467320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x555562473110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555562222700 con 0x55556208b1b0 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55556235c160 con 0x555562093390 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555562469e20 con 0x5555620b4070 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6781623740 10 client.0 _ll_register_callbacks cb 0x5555623759e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x555562466de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6773fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555562093390 0x555562467320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x555562466de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40986/0 (socket says 172.21.15.31:40986) 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f6778e4b640 1 -- 172.21.15.31:0/2416117627 learned_addr learned my addr 172.21.15.31:0/2416117627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.322+0000 7f677964c640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x555562473110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6778e4b640 1 -- 172.21.15.31:0/2416117627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 msgr2=0x555562473110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6778e4b640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x555562473110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6778e4b640 1 -- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555562093390 msgr2=0x555562467320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6778e4b640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555562093390 0x555562467320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6778e4b640 1 -- 172.21.15.31:0/2416117627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555562389c90 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6773fff640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555562093390 0x555562467320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6778e4b640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x555562466de0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6764005fd0 tx=0x7f67640e5260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd265f024fb88d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6753fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6753fff640 1 -- 172.21.15.31:0/2416117627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676401d040 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6753fff640 1 -- 172.21.15.31:0/2416117627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f6764094ae0 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676c021030 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6753fff640 1 -- 172.21.15.31:0/2416117627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6764094040 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6781623740 10 client.17010 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.323+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555562475c60 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6753fff640 1 -- 172.21.15.31:0/2416117627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c021030 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6753fff640 1 -- 172.21.15.31:0/2416117627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6764094ce0 con 0x55556208b1b0 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6781623740 10 client.17010 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:10.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6781623740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6781623740 20 client.17010 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6781623740 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5555620729f0 0x555562479130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6781623740 4 client.17010 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555623d3df0 con 0x5555620729f0 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.324+0000 7f6773fff640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5555620729f0 0x555562479130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.325+0000 7f6773fff640 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5555620729f0 0x555562479130 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975600c9c145ff2b server_cookie=796094ff822b951a in_seq=0 out_seq=0 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.325+0000 7f6753fff640 10 client.17010 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.325+0000 7f6753fff640 1 -- 172.21.15.31:0/2416117627 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555623d3df0 con 0x5555620729f0 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.325+0000 7f6753fff640 10 client.17010 handle_command_reply: tid=0 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.325+0000 7f6781623740 1 client.17010 shutdown 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 msgr2=0x555562466de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x555562466de0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6764005fd0 tx=0x7f67640e5260 comp rx=0 tx=0).stop 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 shutdown_connections 2024-09-21T02:59:10.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5555620729f0 0x555562479130 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x555562473110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555562093390 0x555562467320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 --2- 172.21.15.31:0/2416117627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x555562466de0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 >> 172.21.15.31:0/2416117627 conn(0x5555620a8460 msgr2=0x55556245e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6753fff640 0 client.17010 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 shutdown_connections 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 1 -- 172.21.15.31:0/2416117627 wait complete. 2024-09-21T02:59:10.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.326+0000 7f6781623740 20 client.17010 trim_cache size 0 max 16384 2024-09-21T02:59:10.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 Processor -- start 2024-09-21T02:59:10.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 -- start start 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x5555624672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b1a40 con 0x55556208b1b0 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b3580 con 0x5555620b4070 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6781623740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b2110 con 0x555562093390 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x5555624672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6778e4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x5555624672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40992/0 (socket says 172.21.15.31:40992) 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.339+0000 7f6778e4b640 1 -- 172.21.15.31:0/975775546 learned_addr learned my addr 172.21.15.31:0/975775546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6773fff640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f677964c640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6778e4b640 1 -- 172.21.15.31:0/975775546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 msgr2=0x55556245e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6778e4b640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6778e4b640 1 -- 172.21.15.31:0/975775546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 msgr2=0x55556245ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6778e4b640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6778e4b640 1 -- 172.21.15.31:0/975775546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555624652e0 con 0x55556208b1b0 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6773fff640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f677964c640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f6778e4b640 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x5555624672a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f67640ad0a0 tx=0x7f67640acbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c774bf6a65b6731 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f676a7fc640 1 -- 172.21.15.31:0/975775546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676401d070 con 0x55556208b1b0 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f676a7fc640 1 -- 172.21.15.31:0/975775546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6764005290 con 0x55556208b1b0 2024-09-21T02:59:10.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.340+0000 7f676a7fc640 1 -- 172.21.15.31:0/975775546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764094d40 con 0x55556208b1b0 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 -- 172.21.15.31:0/975775546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 msgr2=0x5555624672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x5555624672a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f67640ad0a0 tx=0x7f67640acbd0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 -- 172.21.15.31:0/975775546 shutdown_connections 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555562093390 0x55556245e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555620b4070 0x55556245ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 --2- 172.21.15.31:0/975775546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55556208b1b0 0x5555624672a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 -- 172.21.15.31:0/975775546 >> 172.21.15.31:0/975775546 conn(0x5555620a8460 msgr2=0x555562466b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 -- 172.21.15.31:0/975775546 shutdown_connections 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.341+0000 7f6781623740 1 -- 172.21.15.31:0/975775546 wait complete. 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 -- start start 2024-09-21T02:59:10.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 Processor -- start 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55556208b1b0 0x55556246b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 0x55556246b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x55556246bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b2eb0 con 0x555562093390 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b27e0 con 0x55556208b1b0 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67741afa20 con 0x5555620b4070 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6781623740 10 client.0 _ll_register_callbacks cb 0x7f6764004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6773fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 0x55556246b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6773fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 0x55556246b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40994/0 (socket says 172.21.15.31:40994) 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.342+0000 7f6773fff640 1 -- 172.21.15.31:0/4263318734 learned_addr learned my addr 172.21.15.31:0/4263318734 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f677964c640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x55556246bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6778e4b640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55556208b1b0 0x55556246b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6773fff640 1 -- 172.21.15.31:0/4263318734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 msgr2=0x55556246bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6773fff640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x55556246bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6773fff640 1 -- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55556208b1b0 msgr2=0x55556246b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6773fff640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55556208b1b0 0x55556246b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6773fff640 1 -- 172.21.15.31:0/4263318734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555624e2ae0 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f677964c640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x55556246bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6778e4b640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55556208b1b0 0x55556246b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6773fff640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 0x55556246b640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f675c009340 tx=0x7f675c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d38a6722fb5fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6753fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6753fff640 1 -- 172.21.15.31:0/4263318734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c035040 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555623cf8a0 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6753fff640 1 -- 172.21.15.31:0/4263318734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f675c005240 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6753fff640 1 -- 172.21.15.31:0/4263318734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f675c051050 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6781623740 10 client.17016 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.343+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5555624e3df0 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6753fff640 1 -- 172.21.15.31:0/4263318734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5555623cf8a0 con 0x555562093390 2024-09-21T02:59:10.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6753fff640 1 -- 172.21.15.31:0/4263318734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f675c031620 con 0x555562093390 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6781623740 10 client.17016 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6781623740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6781623740 20 client.17016 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6781623740 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x555562047da0 0x5555624e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6781623740 4 client.17016 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555623cccd0 con 0x555562047da0 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.344+0000 7f6778e4b640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x555562047da0 0x5555624e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6778e4b640 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x555562047da0 0x5555624e65b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cd5c51219b73b0 server_cookie=a720426bd152ee31 in_seq=0 out_seq=0 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6753fff640 10 client.17016 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6753fff640 1 -- 172.21.15.31:0/4263318734 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555623cccd0 con 0x555562047da0 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6753fff640 10 client.17016 handle_command_reply: tid=0 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 client.17016 shutdown 2024-09-21T02:59:10.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 msgr2=0x55556246b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 0x55556246b640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f675c009340 tx=0x7f675c004270 comp rx=0 tx=0).stop 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 shutdown_connections 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x555562047da0 0x5555624e65b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5555620b4070 0x55556246bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555562093390 0x55556246b640 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6753fff640 0 client.17016 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 --2- 172.21.15.31:0/4263318734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55556208b1b0 0x55556246b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 >> 172.21.15.31:0/4263318734 conn(0x5555620a8460 msgr2=0x55556245e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 shutdown_connections 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 1 -- 172.21.15.31:0/4263318734 wait complete. 2024-09-21T02:59:10.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.345+0000 7f6781623740 20 client.17016 trim_cache size 0 max 16384 2024-09-21T02:59:10.352 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67600b6680 msgr2=0x7f67600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67600b6680 0x7f67600b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f676c00d000 tx=0x7f676c01d000 comp rx=0 tx=0).stop 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 msgr2=0x7f677410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 0x7f677410ffb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f676400c890 tx=0x7f676400cd50 comp rx=0 tx=0).stop 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 shutdown_connections 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67600b6680 0x7f67600b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67741083a0 0x7f67741104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6774108cf0 0x7f6774110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 --2- 172.21.15.31:0/3564199113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6774102a60 0x7f677410ffb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 >> 172.21.15.31:0/3564199113 conn(0x7f6774079730 msgr2=0x7f67741044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 shutdown_connections 2024-09-21T02:59:10.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:10.346+0000 7f677b0af640 1 -- 172.21.15.31:0/3564199113 wait complete. 2024-09-21T02:59:11.034 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:10 smithi031 ceph-mon[31209]: pgmap v513: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:10 smithi165 ceph-mon[34818]: pgmap v513: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:11.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:10 smithi178 ceph-mon[34759]: pgmap v513: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:12.068 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:59:12.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:59:12.343 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 -- 172.21.15.31:0/2887300997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c103b60 msgr2=0x7ff00c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 --2- 172.21.15.31:0/2887300997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c103b60 0x7ff00c103f70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff008009900 tx=0x7ff00802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 -- 172.21.15.31:0/2887300997 shutdown_connections 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 --2- 172.21.15.31:0/2887300997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c1094a0 0x7ff00c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 --2- 172.21.15.31:0/2887300997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c109df0 0x7ff00c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 --2- 172.21.15.31:0/2887300997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c103b60 0x7ff00c103f70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 -- 172.21.15.31:0/2887300997 >> 172.21.15.31:0/2887300997 conn(0x7ff00c07b850 msgr2=0x7ff00c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 -- 172.21.15.31:0/2887300997 shutdown_connections 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.859+0000 7ff014554640 1 -- 172.21.15.31:0/2887300997 wait complete. 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 Processor -- start 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 -- start start 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c103b60 0x7ff00c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c1094a0 0x7ff00c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 0x7ff00c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c113ab0 con 0x7ff00c109df0 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c113930 con 0x7ff00c1094a0 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff014554640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c113c30 con 0x7ff00c103b60 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff012af1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 0x7ff00c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff012af1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 0x7ff00c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41016/0 (socket says 172.21.15.31:41016) 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff012af1640 1 -- 172.21.15.31:0/3962987158 learned_addr learned my addr 172.21.15.31:0/3962987158 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff0122f0640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c103b60 0x7ff00c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff011aef640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c1094a0 0x7ff00c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.860+0000 7ff012af1640 1 -- 172.21.15.31:0/3962987158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c103b60 msgr2=0x7ff00c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff012af1640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c103b60 0x7ff00c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff012af1640 1 -- 172.21.15.31:0/3962987158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c1094a0 msgr2=0x7ff00c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff012af1640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c1094a0 0x7ff00c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff012af1640 1 -- 172.21.15.31:0/3962987158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c1a03d0 con 0x7ff00c109df0 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff0122f0640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c103b60 0x7ff00c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff011aef640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c1094a0 0x7ff00c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff012af1640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 0x7ff00c19fca0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff004009fd0 tx=0x7ff00400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff0037fe640 1 -- 172.21.15.31:0/3962987158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004017070 con 0x7ff00c109df0 2024-09-21T02:59:12.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c1a06c0 con 0x7ff00c109df0 2024-09-21T02:59:12.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff0037fe640 1 -- 172.21.15.31:0/3962987158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff004003e20 con 0x7ff00c109df0 2024-09-21T02:59:12.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff0037fe640 1 -- 172.21.15.31:0/3962987158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004010460 con 0x7ff00c109df0 2024-09-21T02:59:12.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.861+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c1a6e20 con 0x7ff00c109df0 2024-09-21T02:59:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.863+0000 7ff01aac8740 1 Processor -- start 2024-09-21T02:59:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff0037fe640 1 -- 172.21.15.31:0/3962987158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff004003970 con 0x7ff00c109df0 2024-09-21T02:59:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 -- start start 2024-09-21T02:59:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cf4490 0x55c8a70c4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c8a703ae60 con 0x55c8a6d1d350 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c8a702a290 con 0x55c8a6cfc670 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff01aac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c8a702e0a0 con 0x55c8a6cf4490 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.864+0000 7ff011aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41020/0 (socket says 172.21.15.31:41020) 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 -- 172.21.15.31:0/3539813659 learned_addr learned my addr 172.21.15.31:0/3539813659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff0122f0640 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cf4490 0x55c8a70c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff0037fe640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff80b6680 0x7feff80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 -- 172.21.15.31:0/3539813659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cf4490 msgr2=0x55c8a70c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cf4490 0x55c8a70c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 -- 172.21.15.31:0/3539813659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 msgr2=0x55c8a70c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff012af1640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff80b6680 0x7feff80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 -- 172.21.15.31:0/3539813659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a6d88930 con 0x55c8a6d1d350 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff0122f0640 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cf4490 0x55c8a70c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff0037fe640 1 -- 172.21.15.31:0/3962987158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff0040e74a0 con 0x7ff00c109df0 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff011aef640 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c4430 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feffc0159b0 tx=0x7feffc009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a5645b982d94b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.865+0000 7ff000ff9640 1 -- 172.21.15.31:0/3539813659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc015270 con 0x55c8a6d1d350 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff000ff9640 1 -- 172.21.15.31:0/3539813659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc0054d0 con 0x55c8a6d1d350 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff000ff9640 1 -- 172.21.15.31:0/3539813659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc004d90 con 0x55c8a6d1d350 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 -- 172.21.15.31:0/3539813659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 msgr2=0x55c8a70c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff012af1640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff80b6680 0x7feff80b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0040e4cc0 tx=0x7ff0040dddb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c4430 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feffc0159b0 tx=0x7feffc009670 comp rx=0 tx=0).stop 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 -- 172.21.15.31:0/3539813659 shutdown_connections 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cf4490 0x55c8a70c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 --2- 172.21.15.31:0/3539813659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c4430 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.866+0000 7ff01aac8740 1 -- 172.21.15.31:0/3539813659 >> 172.21.15.31:0/3539813659 conn(0x55c8a6d11740 msgr2=0x55c8a70c2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 -- 172.21.15.31:0/3539813659 shutdown_connections 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 -- 172.21.15.31:0/3539813659 wait complete. 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 -- start start 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 Processor -- start 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 0x55c8a70d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6e8b890 con 0x55c8a6cf4490 2024-09-21T02:59:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c8a6fc4d40 con 0x55c8a6cfc670 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.867+0000 7ff01aac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c8a70d6f10 con 0x55c8a6d1d350 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 0x55c8a70d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 0x55c8a70d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41024/0 (socket says 172.21.15.31:41024) 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 -- 172.21.15.31:0/4140828382 learned_addr learned my addr 172.21.15.31:0/4140828382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff01aac8740 10 client.0 _ll_register_callbacks cb 0x55c8a7028c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff0122f0640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 -- 172.21.15.31:0/4140828382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 msgr2=0x55c8a70e0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 -- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 msgr2=0x55c8a70d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70d4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 -- 172.21.15.31:0/4140828382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a70e07b0 con 0x55c8a6cf4490 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff011aef640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 0x55c8a70d4350 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feffc004ab0 tx=0x7feffc009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79af9b6a894c3159 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7feff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7feff5ffb640 1 -- 172.21.15.31:0/4140828382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc004d60 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7feff5ffb640 1 -- 172.21.15.31:0/4140828382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7feffc015110 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.868+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feffc0054d0 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7feff5ffb640 1 -- 172.21.15.31:0/4140828382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feffc030050 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 10 client.17028 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a70e3070 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7feff5ffb640 1 -- 172.21.15.31:0/4140828382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc0054d0 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7feff5ffb640 1 -- 172.21.15.31:0/4140828382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feffc013400 con 0x55c8a6cf4490 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 10 client.17028 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 20 client.17028 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cdbcd0 0x55c8a70e6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 4 client.17028 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8a703cdb0 con 0x55c8a6cdbcd0 2024-09-21T02:59:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.869+0000 7ff0122f0640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cdbcd0 0x55c8a70e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.870+0000 7ff0122f0640 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cdbcd0 0x55c8a70e6420 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d1dad8e9a85cc9 server_cookie=d91c2beeaca36f47 in_seq=0 out_seq=0 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.870+0000 7feff5ffb640 10 client.17028 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7feff5ffb640 1 -- 172.21.15.31:0/4140828382 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c8a703cdb0 con 0x55c8a6cdbcd0 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7feff5ffb640 10 client.17028 handle_command_reply: tid=0 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 client.17028 shutdown 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 msgr2=0x55c8a70d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 0x55c8a70d4350 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feffc004ab0 tx=0x7feffc009c60 comp rx=0 tx=0).stop 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 shutdown_connections 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cdbcd0 0x55c8a70e6420 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cfc670 0x55c8a70d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 --2- 172.21.15.31:0/4140828382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cf4490 0x55c8a70d4350 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 >> 172.21.15.31:0/4140828382 conn(0x55c8a6d11740 msgr2=0x55c8a70c0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7feff5ffb640 0 client.17028 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 shutdown_connections 2024-09-21T02:59:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 1 -- 172.21.15.31:0/4140828382 wait complete. 2024-09-21T02:59:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.871+0000 7ff01aac8740 20 client.17028 trim_cache size 0 max 16384 2024-09-21T02:59:12.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 Processor -- start 2024-09-21T02:59:12.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 -- start start 2024-09-21T02:59:12.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cfc670 0x55c8a70d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c1ad580 con 0x55c8a6d1d350 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c1af0c0 con 0x55c8a6cf4490 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff01aac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c1adc50 con 0x55c8a6cfc670 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41030/0 (socket says 172.21.15.31:41030) 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 -- 172.21.15.31:0/1434914786 learned_addr learned my addr 172.21.15.31:0/1434914786 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff0122f0640 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cfc670 0x55c8a70d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 -- 172.21.15.31:0/1434914786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cfc670 msgr2=0x55c8a70d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cfc670 0x55c8a70d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 -- 172.21.15.31:0/1434914786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 msgr2=0x55c8a70c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.889+0000 7ff011aef640 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.890+0000 7ff011aef640 1 -- 172.21.15.31:0/1434914786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a6cdbf00 con 0x55c8a6d1d350 2024-09-21T02:59:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.890+0000 7ff0122f0640 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cfc670 0x55c8a70d39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.890+0000 7ff011aef640 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c0ec0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feffc00a010 tx=0x7feffc017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf2d60477b8652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.890+0000 7feff7fff640 1 -- 172.21.15.31:0/1434914786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc0155b0 con 0x55c8a6d1d350 2024-09-21T02:59:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.890+0000 7feff7fff640 1 -- 172.21.15.31:0/1434914786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc015110 con 0x55c8a6d1d350 2024-09-21T02:59:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.890+0000 7feff7fff640 1 -- 172.21.15.31:0/1434914786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc010940 con 0x55c8a6d1d350 2024-09-21T02:59:12.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- 172.21.15.31:0/1434914786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 msgr2=0x55c8a70c0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c0ec0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feffc00a010 tx=0x7feffc017040 comp rx=0 tx=0).stop 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- 172.21.15.31:0/1434914786 shutdown_connections 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6cfc670 0x55c8a70d39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6d1d350 0x55c8a70c0ec0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1434914786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- 172.21.15.31:0/1434914786 >> 172.21.15.31:0/1434914786 conn(0x55c8a6d11740 msgr2=0x55c8a70d5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- 172.21.15.31:0/1434914786 shutdown_connections 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- 172.21.15.31:0/1434914786 wait complete. 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- start start 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 Processor -- start 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 0x55c8a70e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.891+0000 7ff01aac8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c1ae9f0 con 0x55c8a6cfc670 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff01aac8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c1ae320 con 0x55c8a6cf4490 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff01aac8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c1ab560 con 0x55c8a6d1d350 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff01aac8740 10 client.0 _ll_register_callbacks cb 0x7feffc009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 0x55c8a70e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 0x55c8a70e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41036/0 (socket says 172.21.15.31:41036) 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 -- 172.21.15.31:0/1330307377 learned_addr learned my addr 172.21.15.31:0/1330307377 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff011aef640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff011aef640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 -- 172.21.15.31:0/1330307377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 msgr2=0x55c8a70e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 -- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 msgr2=0x55c8a70e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70e7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 -- 172.21.15.31:0/1330307377 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a714fbd0 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff0122f0640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 0x55c8a70e79c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff008005f70 tx=0x7ff0080042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de72f79921f9801 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7feff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7feff5ffb640 1 -- 172.21.15.31:0/1330307377 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008035040 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7feff5ffb640 1 -- 172.21.15.31:0/1330307377 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7ff008005040 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.892+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feffc015110 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7feff5ffb640 1 -- 172.21.15.31:0/1330307377 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff008051050 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 10 client.17034 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c8a7150e00 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7feff5ffb640 1 -- 172.21.15.31:0/1330307377 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc015110 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7feff5ffb640 1 -- 172.21.15.31:0/1330307377 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0080035f0 con 0x55c8a6cfc670 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 10 client.17034 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 20 client.17034 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cb1080 0x55c8a7153640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 4 client.17034 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:59:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c8a7038860 con 0x55c8a6cb1080 2024-09-21T02:59:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.893+0000 7ff011aef640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cb1080 0x55c8a7153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.894+0000 7feff5ffb640 10 client.17034 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.894+0000 7ff011aef640 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cb1080 0x55c8a7153640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8c5ba5717ba422 server_cookie=85e1de3cc80587dc in_seq=0 out_seq=0 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.895+0000 7feff5ffb640 1 -- 172.21.15.31:0/1330307377 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c8a7038860 con 0x55c8a6cb1080 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.895+0000 7feff5ffb640 10 client.17034 handle_command_reply: tid=0 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.895+0000 7ff01aac8740 1 client.17034 shutdown 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 msgr2=0x55c8a70e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 0x55c8a70e79c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff008005f70 tx=0x7ff0080042c0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 shutdown_connections 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c8a6cb1080 0x55c8a7153640 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8a6d1d350 0x55c8a70e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c8a6cfc670 0x55c8a70e79c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 --2- 172.21.15.31:0/1330307377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8a6cf4490 0x55c8a70e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 >> 172.21.15.31:0/1330307377 conn(0x55c8a6d11740 msgr2=0x55c8a70c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7feff5ffb640 0 client.17034 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 shutdown_connections 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 1 -- 172.21.15.31:0/1330307377 wait complete. 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.896+0000 7ff01aac8740 20 client.17034 trim_cache size 0 max 16384 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:59:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "18df0cce-9122-4996-a67b-d5d7fb77b7fb", 2024-09-21T02:59:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:59:12.902 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:59:12.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff80b6680 msgr2=0x7feff80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff80b6680 0x7feff80b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0040e4cc0 tx=0x7ff0040dddb0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 msgr2=0x7ff00c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 0x7ff00c19fca0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff004009fd0 tx=0x7ff00400b260 comp rx=0 tx=0).stop 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 shutdown_connections 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7feff80b6680 0x7feff80b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c109df0 0x7ff00c19fca0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff00c1094a0 0x7ff00c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 --2- 172.21.15.31:0/3962987158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff00c103b60 0x7ff00c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 >> 172.21.15.31:0/3962987158 conn(0x7ff00c07b850 msgr2=0x7ff00c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 shutdown_connections 2024-09-21T02:59:12.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:12.897+0000 7ff014554640 1 -- 172.21.15.31:0/3962987158 wait complete. 2024-09-21T02:59:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:12 smithi031 ceph-mon[31209]: pgmap v514: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:12 smithi165 ceph-mon[34818]: pgmap v514: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:12 smithi178 ceph-mon[34759]: pgmap v514: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T02:59:13.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:59:13.898 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:14.021 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:13 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T02:59:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:13 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:59:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:13 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T02:59:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:13 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T02:59:14.022 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:13 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:59:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:13 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T02:59:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:13 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:59:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:13 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T02:59:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:13 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T02:59:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:13 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:59:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:13 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T02:59:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:13 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:59:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:13 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T02:59:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:13 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T02:59:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:13 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 -- 172.21.15.31:0/3761138825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48441052b0 msgr2=0x7f484410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 --2- 172.21.15.31:0/3761138825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48441052b0 0x7f484410d8f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4834009900 tx=0x7f483402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 -- 172.21.15.31:0/3761138825 shutdown_connections 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 --2- 172.21.15.31:0/3761138825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f484410de30 0x7f4844110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 --2- 172.21.15.31:0/3761138825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48441052b0 0x7f484410d8f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 --2- 172.21.15.31:0/3761138825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4844104880 0x7f4844104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 -- 172.21.15.31:0/3761138825 >> 172.21.15.31:0/3761138825 conn(0x7f4844079730 msgr2=0x7f4844079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 -- 172.21.15.31:0/3761138825 shutdown_connections 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.426+0000 7f484c386640 1 -- 172.21.15.31:0/3761138825 wait complete. 2024-09-21T02:59:14.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 Processor -- start 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 -- start start 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 0x7f4844102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48441052b0 0x7f4844102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484410de30 0x7f4844103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4844113a70 con 0x7f4844104880 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48441138f0 con 0x7f484410de30 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484c386640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4844113bf0 con 0x7f48441052b0 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484a122640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 0x7f4844102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f4849921640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48441052b0 0x7f4844102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484a122640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 0x7f4844102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41060/0 (socket says 172.21.15.31:41060) 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484a122640 1 -- 172.21.15.31:0/2996032692 learned_addr learned my addr 172.21.15.31:0/2996032692 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.427+0000 7f484a923640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484410de30 0x7f4844103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a122640 1 -- 172.21.15.31:0/2996032692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48441052b0 msgr2=0x7f4844102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a122640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48441052b0 0x7f4844102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a122640 1 -- 172.21.15.31:0/2996032692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484410de30 msgr2=0x7f4844103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a122640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484410de30 0x7f4844103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a122640 1 -- 172.21.15.31:0/2996032692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48441a8740 con 0x7f4844104880 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f4849921640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48441052b0 0x7f4844102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a923640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484410de30 0x7f4844103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484a122640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 0x7f4844102790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f484000e9c0 tx=0x7f484000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f483b7fe640 1 -- 172.21.15.31:0/2996032692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484000cd80 con 0x7f4844104880 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f483b7fe640 1 -- 172.21.15.31:0/2996032692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48400035f0 con 0x7f4844104880 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f483b7fe640 1 -- 172.21.15.31:0/2996032692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840010690 con 0x7f4844104880 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48441a8a30 con 0x7f4844104880 2024-09-21T02:59:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.428+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48441a8f90 con 0x7f4844104880 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 Processor -- start 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 -- start start 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddbd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddbdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dd33e50 con 0x55cc7da16070 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dd23280 con 0x55cc7d9ed1b0 2024-09-21T02:59:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f48528fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dd27090 con 0x55cc7d9f5390 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f483b7fe640 1 -- 172.21.15.31:0/2996032692 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f48400037a0 con 0x7f4844104880 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddbd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddbd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41072/0 (socket says 172.21.15.31:41072) 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f484a923640 1 -- 172.21.15.31:0/3074257413 learned_addr learned my addr 172.21.15.31:0/3074257413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f4849921640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddbdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.431+0000 7f484a122640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a923640 1 -- 172.21.15.31:0/3074257413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 msgr2=0x55cc7ddbdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a923640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddbdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a923640 1 -- 172.21.15.31:0/3074257413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 msgr2=0x55cc7ddbd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a923640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a923640 1 -- 172.21.15.31:0/3074257413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7da81650 con 0x55cc7da16070 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a122640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f4849921640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f484a923640 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddbd270 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f483c009fd0 tx=0x7f483c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b1ae9751b56d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f4838d9a640 1 -- 172.21.15.31:0/3074257413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c011760 con 0x55cc7da16070 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f4838d9a640 1 -- 172.21.15.31:0/3074257413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483c011db0 con 0x55cc7da16070 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f4838d9a640 1 -- 172.21.15.31:0/3074257413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00e3c0 con 0x55cc7da16070 2024-09-21T02:59:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.432+0000 7f483b7fe640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f48300b6700 0x7f48300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f484a122640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f48300b6700 0x7f48300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f483b7fe640 1 -- 172.21.15.31:0/2996032692 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f48400e7af0 con 0x7f4844104880 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 -- 172.21.15.31:0/3074257413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 msgr2=0x55cc7ddbd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddbd270 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f483c009fd0 tx=0x7f483c0108e0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f484a122640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f48300b6700 0x7f48300b8c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f48400e5310 tx=0x7f48400aa000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 -- 172.21.15.31:0/3074257413 shutdown_connections 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 --2- 172.21.15.31:0/3074257413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddbd270 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.433+0000 7f48528fa740 1 -- 172.21.15.31:0/3074257413 >> 172.21.15.31:0/3074257413 conn(0x55cc7da0a460 msgr2=0x55cc7ddbb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 -- 172.21.15.31:0/3074257413 shutdown_connections 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 -- 172.21.15.31:0/3074257413 wait complete. 2024-09-21T02:59:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 -- start start 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 Processor -- start 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddc8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55cc7db84700 con 0x55cc7da16070 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dcbe110 con 0x55cc7d9ed1b0 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cc7ddcba50 con 0x55cc7d9f5390 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.434+0000 7f48528fa740 10 client.0 _ll_register_callbacks cb 0x55cc7dcd7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41074/0 (socket says 172.21.15.31:41074) 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:60956/0 (socket says 172.21.15.31:60956) 2024-09-21T02:59:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f484a923640 1 -- 172.21.15.31:0/1607769019 learned_addr learned my addr 172.21.15.31:0/1607769019 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f484a923640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddc8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 -- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 msgr2=0x55cc7ddc8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddc8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 -- 172.21.15.31:0/1607769019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 msgr2=0x55cc7ddc93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 -- 172.21.15.31:0/1607769019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7dcebc40 con 0x55cc7d9f5390 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f484a923640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f4849921640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc8e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4834006010 tx=0x7f4834031740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1312153dc49c3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f482dffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.435+0000 7f482dffb640 1 -- 172.21.15.31:0/1607769019 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834035040 con 0x55cc7d9f5390 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483c011db0 con 0x55cc7d9f5390 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f482dffb640 1 -- 172.21.15.31:0/1607769019 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f4834004040 con 0x55cc7d9f5390 2024-09-21T02:59:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f482dffb640 1 -- 172.21.15.31:0/1607769019 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4834004240 con 0x55cc7d9f5390 2024-09-21T02:59:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 10 client.26602 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc7ddd7740 con 0x55cc7d9f5390 2024-09-21T02:59:14.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f482dffb640 1 -- 172.21.15.31:0/1607769019 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483c011db0 con 0x55cc7d9f5390 2024-09-21T02:59:14.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f482dffb640 1 -- 172.21.15.31:0/1607769019 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4834004bb0 con 0x55cc7d9f5390 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 10 client.26602 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 10 client.26602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 20 client.26602 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9d49f0 0x55cc7dddabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 4 client.26602 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.436+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc7dd35da0 con 0x55cc7d9d49f0 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.437+0000 7f484a923640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9d49f0 0x55cc7dddabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.437+0000 7f484a923640 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9d49f0 0x55cc7dddabb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795c6f3fe1a7ad3b server_cookie=42b1e8a4d17ab80f in_seq=0 out_seq=0 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.437+0000 7f482dffb640 10 client.26602 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f482dffb640 1 -- 172.21.15.31:0/1607769019 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc7dd35da0 con 0x55cc7d9d49f0 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f482dffb640 10 client.26602 handle_command_reply: tid=0 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 client.26602 shutdown 2024-09-21T02:59:14.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 msgr2=0x55cc7ddc8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc8e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4834006010 tx=0x7f4834031740 comp rx=0 tx=0).stop 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 shutdown_connections 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9d49f0 0x55cc7dddabb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc8e50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f482dffb640 0 client.26602 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 --2- 172.21.15.31:0/1607769019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddc8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 >> 172.21.15.31:0/1607769019 conn(0x55cc7da0a460 msgr2=0x55cc7ddc0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 shutdown_connections 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 1 -- 172.21.15.31:0/1607769019 wait complete. 2024-09-21T02:59:14.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.438+0000 7f48528fa740 20 client.26602 trim_cache size 0 max 16384 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 Processor -- start 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 -- start start 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48441b1960 con 0x55cc7da16070 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48441b34a0 con 0x55cc7d9f5390 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f48528fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48441b2030 con 0x55cc7d9ed1b0 2024-09-21T02:59:14.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41082/0 (socket says 172.21.15.31:41082) 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 -- 172.21.15.31:0/655044692 learned_addr learned my addr 172.21.15.31:0/655044692 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f4849921640 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 -- 172.21.15.31:0/655044692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 msgr2=0x55cc7ddbe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 -- 172.21.15.31:0/655044692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 msgr2=0x55cc7ddc5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.456+0000 7f484a923640 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.457+0000 7f484a923640 1 -- 172.21.15.31:0/655044692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7ddb5410 con 0x55cc7da16070 2024-09-21T02:59:14.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.457+0000 7f484a923640 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc6040 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f483c01ba30 tx=0x7f483c01ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3174508cd10a68f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.457+0000 7f482ffff640 1 -- 172.21.15.31:0/655044692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c0049e0 con 0x55cc7da16070 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.457+0000 7f482ffff640 1 -- 172.21.15.31:0/655044692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483c004b80 con 0x55cc7da16070 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.457+0000 7f482ffff640 1 -- 172.21.15.31:0/655044692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c005290 con 0x55cc7da16070 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- 172.21.15.31:0/655044692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 msgr2=0x55cc7ddc6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc6040 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f483c01ba30 tx=0x7f483c01ba60 comp rx=0 tx=0).stop 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- 172.21.15.31:0/655044692 shutdown_connections 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddc6040 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddc5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- 172.21.15.31:0/655044692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddbe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- 172.21.15.31:0/655044692 >> 172.21.15.31:0/655044692 conn(0x55cc7da0a460 msgr2=0x55cc7ddbe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- 172.21.15.31:0/655044692 shutdown_connections 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- 172.21.15.31:0/655044692 wait complete. 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- start start 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 Processor -- start 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddcceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddcd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddcd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48441b2dd0 con 0x55cc7da16070 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.458+0000 7f48528fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48441b2700 con 0x55cc7d9f5390 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f48528fa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48441af940 con 0x55cc7d9ed1b0 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f48528fa740 10 client.0 _ll_register_callbacks cb 0x7f4834009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddcd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddcd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41096/0 (socket says 172.21.15.31:41096) 2024-09-21T02:59:14.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 -- 172.21.15.31:0/3855164759 learned_addr learned my addr 172.21.15.31:0/3855164759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f4849921640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddcd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddcceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 -- 172.21.15.31:0/3855164759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 msgr2=0x55cc7ddcceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddcceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 -- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 msgr2=0x55cc7ddcd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddcd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 -- 172.21.15.31:0/3855164759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7de44770 con 0x55cc7da16070 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f4849921640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddcd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f484a923640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddcd930 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f483c009b20 tx=0x7f483c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364b5d0542636f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f482dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f482dffb640 1 -- 172.21.15.31:0/3855164759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c0117f0 con 0x55cc7da16070 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f483c004b80 con 0x55cc7da16070 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.459+0000 7f482dffb640 1 -- 172.21.15.31:0/3855164759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f483c011990 con 0x55cc7da16070 2024-09-21T02:59:14.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f482dffb640 1 -- 172.21.15.31:0/3855164759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f483c045050 con 0x55cc7da16070 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 10 client.17052 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cc7de459e0 con 0x55cc7da16070 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f482dffb640 1 -- 172.21.15.31:0/3855164759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483c004b80 con 0x55cc7da16070 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f482dffb640 1 -- 172.21.15.31:0/3855164759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f483c02a070 con 0x55cc7da16070 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 10 client.17052 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 20 client.17052 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9a9da0 0x55cc7de47ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 4 client.17052 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc7dd31850 con 0x55cc7d9a9da0 2024-09-21T02:59:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.460+0000 7f4849921640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9a9da0 0x55cc7de47ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f482dffb640 10 client.17052 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f4849921640 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9a9da0 0x55cc7de47ea0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64d9f900b1a2b86 server_cookie=288b60a30ac0124 in_seq=0 out_seq=0 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f482dffb640 1 -- 172.21.15.31:0/3855164759 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x55cc7dd31850 con 0x55cc7d9a9da0 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f482dffb640 10 client.17052 handle_command_reply: tid=0 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 client.17052 shutdown 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 msgr2=0x55cc7ddcd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddcd930 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f483c009b20 tx=0x7f483c0096c0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 shutdown_connections 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55cc7d9a9da0 0x55cc7de47ea0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc7da16070 0x55cc7ddcd930 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc7d9f5390 0x55cc7ddcd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f482dffb640 0 client.17052 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 --2- 172.21.15.31:0/3855164759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cc7d9ed1b0 0x55cc7ddcceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 >> 172.21.15.31:0/3855164759 conn(0x55cc7da0a460 msgr2=0x55cc7ddc9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 shutdown_connections 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.461+0000 7f48528fa740 1 -- 172.21.15.31:0/3855164759 wait complete. 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f48528fa740 20 client.17052 trim_cache size 0 max 16384 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "18df0cce-9122-4996-a67b-d5d7fb77b7fb", 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:59:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:59:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:14.475 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f48300b6700 msgr2=0x7f48300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f48300b6700 0x7f48300b8c10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f48400e5310 tx=0x7f48400aa000 comp rx=0 tx=0).stop 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 msgr2=0x7f4844102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 0x7f4844102790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f484000e9c0 tx=0x7f484000ee80 comp rx=0 tx=0).stop 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 shutdown_connections 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f48300b6700 0x7f48300b8c10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48441052b0 0x7f4844102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484410de30 0x7f4844103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 --2- 172.21.15.31:0/2996032692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4844104880 0x7f4844102790 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:14.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 >> 172.21.15.31:0/2996032692 conn(0x7f4844079730 msgr2=0x7f48441008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:14.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 shutdown_connections 2024-09-21T02:59:14.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:14.462+0000 7f484c386640 1 -- 172.21.15.31:0/2996032692 wait complete. 2024-09-21T02:59:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: pgmap v515: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:59:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:59:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:59:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: pgmap v515: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:59:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:59:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:59:15.207 INFO:tasks.cephfs.filesystem:scrub status for tag:18df0cce-9122-4996-a67b-d5d7fb77b7fb - {'path': '/', 'tag': '18df0cce-9122-4996-a67b-d5d7fb77b7fb', 'options': 'recursive,force'} 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: pgmap v515: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:59:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T02:59:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:15 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-21T02:59:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:15 smithi031 ceph-mon[31209]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T02:59:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:15 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T02:59:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:15 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-21T02:59:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:15 smithi165 ceph-mon[34818]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T02:59:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:15 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T02:59:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:15 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-21T02:59:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:15 smithi178 ceph-mon[34759]: Unable to set osd_memory_target on smithi031 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T02:59:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:15 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T02:59:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:16 smithi031 ceph-mon[31209]: pgmap v516: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:59:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:16 smithi165 ceph-mon[34818]: pgmap v516: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:59:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:16 smithi178 ceph-mon[34759]: pgmap v516: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-21T02:59:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:18 smithi031 ceph-mon[31209]: pgmap v517: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:59:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:18 smithi165 ceph-mon[34818]: pgmap v517: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:59:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:18 smithi178 ceph-mon[34759]: pgmap v517: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T02:59:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:20 smithi031 ceph-mon[31209]: pgmap v518: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:59:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:20 smithi165 ceph-mon[34818]: pgmap v518: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:59:21.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:20 smithi178 ceph-mon[34759]: pgmap v518: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:59:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:22 smithi031 ceph-mon[31209]: pgmap v519: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:59:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:22 smithi165 ceph-mon[34818]: pgmap v519: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:59:23.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:22 smithi178 ceph-mon[34759]: pgmap v519: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T02:59:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:24 smithi031 ceph-mon[31209]: pgmap v520: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:59:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:24 smithi165 ceph-mon[34818]: pgmap v520: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:59:25.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:24 smithi178 ceph-mon[34759]: pgmap v520: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T02:59:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:26 smithi031 ceph-mon[31209]: pgmap v521: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T02:59:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:26 smithi165 ceph-mon[34818]: pgmap v521: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T02:59:27.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:26 smithi178 ceph-mon[34759]: pgmap v521: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T02:59:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:28 smithi031 ceph-mon[31209]: pgmap v522: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:59:29.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:28 smithi165 ceph-mon[34818]: pgmap v522: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:59:29.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:28 smithi178 ceph-mon[34759]: pgmap v522: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T02:59:31.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:30 smithi178 ceph-mon[34759]: pgmap v523: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:30 smithi031 ceph-mon[31209]: pgmap v523: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:30 smithi165 ceph-mon[34818]: pgmap v523: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:32 smithi031 ceph-mon[31209]: pgmap v524: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:32 smithi165 ceph-mon[34818]: pgmap v524: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:33.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:32 smithi178 ceph-mon[34759]: pgmap v524: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:34 smithi031 ceph-mon[31209]: pgmap v525: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:34 smithi165 ceph-mon[34818]: pgmap v525: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:35.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:34 smithi178 ceph-mon[34759]: pgmap v525: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:36 smithi031 ceph-mon[31209]: pgmap v526: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:36 smithi165 ceph-mon[34818]: pgmap v526: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:37.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:36 smithi178 ceph-mon[34759]: pgmap v526: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:38 smithi031 ceph-mon[31209]: pgmap v527: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:38 smithi165 ceph-mon[34818]: pgmap v527: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:39.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:38 smithi178 ceph-mon[34759]: pgmap v527: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:40 smithi031 ceph-mon[31209]: pgmap v528: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:40 smithi165 ceph-mon[34818]: pgmap v528: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:40 smithi178 ceph-mon[34759]: pgmap v528: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:42 smithi031 ceph-mon[31209]: pgmap v529: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:42 smithi165 ceph-mon[34818]: pgmap v529: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:42 smithi178 ceph-mon[34759]: pgmap v529: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:44 smithi031 ceph-mon[31209]: pgmap v530: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:44 smithi165 ceph-mon[34818]: pgmap v530: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:45.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:59:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:44 smithi178 ceph-mon[34759]: pgmap v530: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:45.485 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 -- 172.21.15.31:0/2720922749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f410de60 msgr2=0x7f50f4110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 --2- 172.21.15.31:0/2720922749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f410de60 0x7f50f4110330 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009ca0 tx=0x7f50ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 -- 172.21.15.31:0/2720922749 shutdown_connections 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 --2- 172.21.15.31:0/2720922749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f4105380 0x7f50f410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 --2- 172.21.15.31:0/2720922749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f410de60 0x7f50f4110330 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 --2- 172.21.15.31:0/2720922749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50f4104a30 0x7f50f4104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 -- 172.21.15.31:0/2720922749 >> 172.21.15.31:0/2720922749 conn(0x7f50f4079760 msgr2=0x7f50f4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 -- 172.21.15.31:0/2720922749 shutdown_connections 2024-09-21T02:59:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 -- 172.21.15.31:0/2720922749 wait complete. 2024-09-21T02:59:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.994+0000 7f50fa13c640 1 Processor -- start 2024-09-21T02:59:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 -- start start 2024-09-21T02:59:45.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 0x7f50f419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:45.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50f4105380 0x7f50f419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:45.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f410de60 0x7f50f419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4113ac0 con 0x7f50f4104a30 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4113940 con 0x7f50f4105380 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50fa13c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4113c40 con 0x7f50f410de60 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 0x7f50f419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 0x7f50f419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42690/0 (socket says 172.21.15.31:42690) 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f37fe640 1 -- 172.21.15.31:0/293568045 learned_addr learned my addr 172.21.15.31:0/293568045 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50f4105380 0x7f50f419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f3fff640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f410de60 0x7f50f419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f37fe640 1 -- 172.21.15.31:0/293568045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f410de60 msgr2=0x7f50f419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f37fe640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f410de60 0x7f50f419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.995+0000 7f50f37fe640 1 -- 172.21.15.31:0/293568045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50f4105380 msgr2=0x7f50f419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f37fe640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50f4105380 0x7f50f419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f37fe640 1 -- 172.21.15.31:0/293568045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41a03e0 con 0x7f50f4104a30 2024-09-21T02:59:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f3fff640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f410de60 0x7f50f419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f37fe640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 0x7f50f419a470 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f50e400d770 tx=0x7f50e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f0ff9640 1 -- 172.21.15.31:0/293568045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4017070 con 0x7f50f4104a30 2024-09-21T02:59:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f0ff9640 1 -- 172.21.15.31:0/293568045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e4009c30 con 0x7f50f4104a30 2024-09-21T02:59:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50f0ff9640 1 -- 172.21.15.31:0/293568045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003cc0 con 0x7f50f4104a30 2024-09-21T02:59:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41a06d0 con 0x7f50f4104a30 2024-09-21T02:59:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.996+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a6e30 con 0x7f50f4104a30 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 Processor -- start 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f0ff9640 1 -- 172.21.15.31:0/293568045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f50e4003e60 con 0x7f50f4104a30 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 -- start start 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a7e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a7eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a7f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c3089f4e50 con 0x55c3086b6390 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c3089e4280 con 0x55c3086d7070 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f51006b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c3089e8090 con 0x55c3086ae1b0 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a7f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a7f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42696/0 (socket says 172.21.15.31:42696) 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f2ffd640 1 -- 172.21.15.31:0/3353275572 learned_addr learned my addr 172.21.15.31:0/3353275572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f37fe640 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a7eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f3fff640 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a7e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:45.999+0000 7f50f2ffd640 1 -- 172.21.15.31:0/3353275572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 msgr2=0x55c308a7eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a7eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f2ffd640 1 -- 172.21.15.31:0/3353275572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 msgr2=0x55c308a7e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a7e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f2ffd640 1 -- 172.21.15.31:0/3353275572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c308742650 con 0x55c3086b6390 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a7f0d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009c20 tx=0x7f50e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b940dbe5bd52491 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f0ff9640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50dc0b6680 0x7f50dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50d9ffb640 1 -- 172.21.15.31:0/3353275572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8004230 con 0x55c3086b6390 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f0ff9640 1 -- 172.21.15.31:0/293568045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f50e40e7070 con 0x7f50f4104a30 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50f3fff640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50dc0b6680 0x7f50dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50d9ffb640 1 -- 172.21.15.31:0/3353275572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e80043d0 con 0x55c3086b6390 2024-09-21T02:59:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.000+0000 7f50d9ffb640 1 -- 172.21.15.31:0/3353275572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8004d00 con 0x55c3086b6390 2024-09-21T02:59:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f50f3fff640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50dc0b6680 0x7f50dc0b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02f500 tx=0x7f50ec02f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 -- 172.21.15.31:0/3353275572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 msgr2=0x55c308a7f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a7f0d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009c20 tx=0x7f50e8011810 comp rx=0 tx=0).stop 2024-09-21T02:59:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 -- 172.21.15.31:0/3353275572 shutdown_connections 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a7eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a7f0d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 --2- 172.21.15.31:0/3353275572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a7e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 -- 172.21.15.31:0/3353275572 >> 172.21.15.31:0/3353275572 conn(0x55c3086cb460 msgr2=0x55c308a7c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 -- 172.21.15.31:0/3353275572 shutdown_connections 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.001+0000 7f51006b0740 1 -- 172.21.15.31:0/3353275572 wait complete. 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 -- start start 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 Processor -- start 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a8a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a8a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c308845700 con 0x55c3086b6390 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c30897f110 con 0x55c3086d7070 2024-09-21T02:59:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c308a91240 con 0x55c3086ae1b0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f50f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a8a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f51006b0740 10 client.0 _ll_register_callbacks cb 0x55c308998990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.002+0000 7f50f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a8a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42700/0 (socket says 172.21.15.31:42700) 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 -- 172.21.15.31:0/883302621 learned_addr learned my addr 172.21.15.31:0/883302621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f37fe640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f3fff640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a8a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 -- 172.21.15.31:0/883302621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 msgr2=0x55c308a89c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a89c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 -- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 msgr2=0x55c308a8a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a8a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 -- 172.21.15.31:0/883302621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3089acc40 con 0x55c3086b6390 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f37fe640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a89c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a8a150 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f50e8019c10 tx=0x7f50e8019c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6789079b0e2b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50d2ffd640 1 -- 172.21.15.31:0/883302621 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8028020 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50d2ffd640 1 -- 172.21.15.31:0/883302621 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f50e8024830 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e80043d0 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f50d2ffd640 1 -- 172.21.15.31:0/883302621 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50e8033050 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f51006b0740 10 client.17064 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.003+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c308a9ce50 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f50d2ffd640 1 -- 172.21.15.31:0/883302621 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e80043d0 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f50d2ffd640 1 -- 172.21.15.31:0/883302621 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f50e8018070 con 0x55c3086b6390 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f51006b0740 10 client.17064 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f51006b0740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f51006b0740 20 client.17064 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f51006b0740 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c3086959f0 0x55c308aa01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f51006b0740 4 client.17064 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3089f6da0 con 0x55c3086959f0 2024-09-21T02:59:46.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.004+0000 7f50f37fe640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c3086959f0 0x55c308aa01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.005+0000 7f50f37fe640 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c3086959f0 0x55c308aa01b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5f2764d8ab692f server_cookie=d39d084806a4a65d in_seq=0 out_seq=0 2024-09-21T02:59:46.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.005+0000 7f50d2ffd640 10 client.17064 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f50d2ffd640 1 -- 172.21.15.31:0/883302621 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f50e4086e50 con 0x55c3086959f0 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f50d2ffd640 10 client.17064 handle_command_reply: tid=0 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 client.17064 shutdown 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 msgr2=0x55c308a8a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a8a150 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f50e8019c10 tx=0x7f50e8019c40 comp rx=0 tx=0).stop 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 shutdown_connections 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c3086959f0 0x55c308aa01b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a8a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086b6390 0x55c308a8a150 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 --2- 172.21.15.31:0/883302621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086ae1b0 0x55c308a89c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 >> 172.21.15.31:0/883302621 conn(0x55c3086cb460 msgr2=0x55c308a817e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f50d2ffd640 0 client.17064 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 shutdown_connections 2024-09-21T02:59:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 1 -- 172.21.15.31:0/883302621 wait complete. 2024-09-21T02:59:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.006+0000 7f51006b0740 20 client.17064 trim_cache size 0 max 16384 2024-09-21T02:59:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 Processor -- start 2024-09-21T02:59:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 -- start start 2024-09-21T02:59:46.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a80150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a8baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41ad490 con 0x55c3086ae1b0 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41aefd0 con 0x55c3086d7070 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f51006b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41adb60 con 0x55c3086b6390 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a80150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a80150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42706/0 (socket says 172.21.15.31:42706) 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.024+0000 7f50f37fe640 1 -- 172.21.15.31:0/1532343697 learned_addr learned my addr 172.21.15.31:0/1532343697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f37fe640 1 -- 172.21.15.31:0/1532343697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 msgr2=0x55c308a8baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f3fff640 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f37fe640 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a8baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f37fe640 1 -- 172.21.15.31:0/1532343697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 msgr2=0x55c308a99b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f37fe640 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a99b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f37fe640 1 -- 172.21.15.31:0/1532343697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c308a8c350 con 0x55c3086ae1b0 2024-09-21T02:59:46.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f3fff640 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a99b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:46.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50f37fe640 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a80150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f50e40e4de0 tx=0x7f50e4003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8377817ca7d075e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50d97fa640 1 -- 172.21.15.31:0/1532343697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4020040 con 0x55c3086ae1b0 2024-09-21T02:59:46.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50d97fa640 1 -- 172.21.15.31:0/1532343697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c3089f6da0 con 0x55c3086ae1b0 2024-09-21T02:59:46.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.025+0000 7f50d97fa640 1 -- 172.21.15.31:0/1532343697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40940b0 con 0x55c3086ae1b0 2024-09-21T02:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 -- 172.21.15.31:0/1532343697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 msgr2=0x55c308a80150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a80150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f50e40e4de0 tx=0x7f50e4003760 comp rx=0 tx=0).stop 2024-09-21T02:59:46.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 -- 172.21.15.31:0/1532343697 shutdown_connections 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a8baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a99b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 --2- 172.21.15.31:0/1532343697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a80150 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 -- 172.21.15.31:0/1532343697 >> 172.21.15.31:0/1532343697 conn(0x55c3086cb460 msgr2=0x55c308a99450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 -- 172.21.15.31:0/1532343697 shutdown_connections 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 -- 172.21.15.31:0/1532343697 wait complete. 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 -- start start 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.026+0000 7f51006b0740 1 Processor -- start 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a92430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a92970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a92eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41ae900 con 0x55c3086ae1b0 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41ae230 con 0x55c3086d7070 2024-09-21T02:59:46.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41ab470 con 0x55c3086b6390 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f51006b0740 10 client.0 _ll_register_callbacks cb 0x7f50e8009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a92430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a92430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42720/0 (socket says 172.21.15.31:42720) 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 -- 172.21.15.31:0/1909043661 learned_addr learned my addr 172.21.15.31:0/1909043661 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a92970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f3fff640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a92eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 -- 172.21.15.31:0/1909043661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 msgr2=0x55c308a92970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a92970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 -- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 msgr2=0x55c308a92eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a92eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f37fe640 1 -- 172.21.15.31:0/1909043661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c308b09a40 con 0x55c3086ae1b0 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.027+0000 7f50f3fff640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a92eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50f37fe640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a92430 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f50e40ace70 tx=0x7f50e40e4c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14df8612ea38d18e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50d2ffd640 1 -- 172.21.15.31:0/1909043661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4020070 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50d2ffd640 1 -- 172.21.15.31:0/1909043661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f50e40942f0 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c3089f6da0 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50d2ffd640 1 -- 172.21.15.31:0/1909043661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50e40b19c0 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 10 client.17070 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c308b0abd0 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50d2ffd640 1 -- 172.21.15.31:0/1909043661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c3089f6da0 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f50d2ffd640 1 -- 172.21.15.31:0/1909043661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f50e405c020 con 0x55c3086ae1b0 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 10 client.17070 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 20 client.17070 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:46.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c30866ada0 0x55c308b0d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 4 client.17070 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.028+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3089f2850 con 0x55c30866ada0 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.029+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c30866ada0 0x55c308b0d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.029+0000 7f50d2ffd640 10 client.17070 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.029+0000 7f50f2ffd640 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c30866ada0 0x55c308b0d280 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a685fdf8b72110 server_cookie=d0a033bcc8cc121f in_seq=0 out_seq=0 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.029+0000 7f50d2ffd640 1 -- 172.21.15.31:0/1909043661 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x55c3089f2850 con 0x55c30866ada0 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.029+0000 7f50d2ffd640 10 client.17070 handle_command_reply: tid=0 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.029+0000 7f51006b0740 1 client.17070 shutdown 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 msgr2=0x55c308a92430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a92430 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f50e40ace70 tx=0x7f50e40e4c90 comp rx=0 tx=0).stop 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 shutdown_connections 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55c30866ada0 0x55c308b0d280 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50d2ffd640 0 client.17070 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:46.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c3086b6390 0x55c308a92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c3086d7070 0x55c308a92eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 --2- 172.21.15.31:0/1909043661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c3086ae1b0 0x55c308a92430 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 >> 172.21.15.31:0/1909043661 conn(0x55c3086cb460 msgr2=0x55c308a8b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 shutdown_connections 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 1 -- 172.21.15.31:0/1909043661 wait complete. 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f51006b0740 20 client.17070 trim_cache size 0 max 16384 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T02:59:46.036 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T02:59:46.042 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50dc0b6680 msgr2=0x7f50dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50dc0b6680 0x7f50dc0b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02f500 tx=0x7f50ec02f310 comp rx=0 tx=0).stop 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 msgr2=0x7f50f419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 0x7f50f419a470 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f50e400d770 tx=0x7f50e400dc30 comp rx=0 tx=0).stop 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 shutdown_connections 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f50dc0b6680 0x7f50dc0b8b90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f410de60 0x7f50f419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50f4105380 0x7f50f419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 --2- 172.21.15.31:0/293568045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50f4104a30 0x7f50f419a470 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.030+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 >> 172.21.15.31:0/293568045 conn(0x7f50f4079760 msgr2=0x7f50f41007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.031+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 shutdown_connections 2024-09-21T02:59:46.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:46.031+0000 7f50fa13c640 1 -- 172.21.15.31:0/293568045 wait complete. 2024-09-21T02:59:46.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:46 smithi031 ceph-mon[31209]: pgmap v531: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:46.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T02:59:46.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:47.181 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:46 smithi165 ceph-mon[34818]: pgmap v531: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:46 smithi178 ceph-mon[34759]: pgmap v531: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.666+0000 7fd0211c6640 1 -- 172.21.15.31:0/410688188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 msgr2=0x7fd01c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.666+0000 7fd0211c6640 1 --2- 172.21.15.31:0/410688188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c1087e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd010009900 tx=0x7fd01002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 -- 172.21.15.31:0/410688188 shutdown_connections 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 --2- 172.21.15.31:0/410688188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01c108d20 0x7fd01c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 --2- 172.21.15.31:0/410688188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c1087e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 --2- 172.21.15.31:0/410688188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 0x7fd01c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 -- 172.21.15.31:0/410688188 >> 172.21.15.31:0/410688188 conn(0x7fd01c079750 msgr2=0x7fd01c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 -- 172.21.15.31:0/410688188 shutdown_connections 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.667+0000 7fd0211c6640 1 -- 172.21.15.31:0/410688188 wait complete. 2024-09-21T02:59:47.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 Processor -- start 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 -- start start 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 0x7fd01c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01c108d20 0x7fd01c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c113ab0 con 0x7fd01c1083d0 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c113930 con 0x7fd01c102a90 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd0211c6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c113c30 con 0x7fd01c108d20 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd01a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd01a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42730/0 (socket says 172.21.15.31:42730) 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.668+0000 7fd01a59c640 1 -- 172.21.15.31:0/1945652474 learned_addr learned my addr 172.21.15.31:0/1945652474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01b59e640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01c108d20 0x7fd01c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01ad9d640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 0x7fd01c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:47.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01a59c640 1 -- 172.21.15.31:0/1945652474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01c108d20 msgr2=0x7fd01c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01a59c640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01c108d20 0x7fd01c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01a59c640 1 -- 172.21.15.31:0/1945652474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 msgr2=0x7fd01c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01a59c640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 0x7fd01c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01a59c640 1 -- 172.21.15.31:0/1945652474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c1a0410 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01ad9d640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 0x7fd01c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd01a59c640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c19aa40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd010032fd0 tx=0x7fd010004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01003f030 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c1a06a0 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd010004490 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010035070 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.669+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1a6e60 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.670+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01c103d10 con 0x7fd01c1083d0 2024-09-21T02:59:47.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.673+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd010042030 con 0x7fd01c1083d0 2024-09-21T02:59:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.674+0000 7fd003fff640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0040b6680 0x7fd0040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.674+0000 7fd01ad9d640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0040b6680 0x7fd0040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.674+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fd010107ff0 con 0x7fd01c1083d0 2024-09-21T02:59:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.674+0000 7fd01ad9d640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0040b6680 0x7fd0040b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd00c005c40 tx=0x7fd00c00a430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:47.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.677+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0100ce560 con 0x7fd01c1083d0 2024-09-21T02:59:47.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.904+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd01c102ea0 con 0x7fd01c1083d0 2024-09-21T02:59:47.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.904+0000 7fd003fff640 1 -- 172.21.15.31:0/1945652474 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fd0100d2940 con 0x7fd01c1083d0 2024-09-21T02:59:47.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T02:59:47.906 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T02:59:47.906 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0040b6680 msgr2=0x7fd0040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0040b6680 0x7fd0040b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd00c005c40 tx=0x7fd00c00a430 comp rx=0 tx=0).stop 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 msgr2=0x7fd01c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c19aa40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd010032fd0 tx=0x7fd010004270 comp rx=0 tx=0).stop 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 shutdown_connections 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0040b6680 0x7fd0040b8b90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01c108d20 0x7fd01c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c1083d0 0x7fd01c19aa40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 --2- 172.21.15.31:0/1945652474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd01c102a90 0x7fd01c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 >> 172.21.15.31:0/1945652474 conn(0x7fd01c079750 msgr2=0x7fd01c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 shutdown_connections 2024-09-21T02:59:47.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:47.907+0000 7fd0211c6640 1 -- 172.21.15.31:0/1945652474 wait complete. 2024-09-21T02:59:48.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T02:59:48.964 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:48 smithi031 ceph-mon[31209]: pgmap v532: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1945652474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:48 smithi165 ceph-mon[34818]: pgmap v532: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1945652474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:48 smithi178 ceph-mon[34759]: pgmap v532: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1945652474' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:49.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 -- 172.21.15.31:0/593520535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578410de10 msgr2=0x7f5784110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 --2- 172.21.15.31:0/593520535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578410de10 0x7f5784110300 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f577c009ca0 tx=0x7f577c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 -- 172.21.15.31:0/593520535 shutdown_connections 2024-09-21T02:59:49.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 --2- 172.21.15.31:0/593520535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5784105300 0x7f578410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 --2- 172.21.15.31:0/593520535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f578410de10 0x7f5784110300 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 --2- 172.21.15.31:0/593520535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57841049b0 0x7f5784104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 -- 172.21.15.31:0/593520535 >> 172.21.15.31:0/593520535 conn(0x7f5784079730 msgr2=0x7f5784079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 -- 172.21.15.31:0/593520535 shutdown_connections 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.443+0000 7f5789730640 1 -- 172.21.15.31:0/593520535 wait complete. 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 Processor -- start 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 -- start start 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57841049b0 0x7f57841026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 0x7f5784102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578410de10 0x7f5784103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5784113a90 con 0x7f5784105300 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.444+0000 7f5789730640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5784113910 con 0x7f57841049b0 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5789730640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5784113c10 con 0x7f578410de10 2024-09-21T02:59:49.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 0x7f5784102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 0x7f5784102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50110/0 (socket says 172.21.15.31:50110) 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 -- 172.21.15.31:0/3220299875 learned_addr learned my addr 172.21.15.31:0/3220299875 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57841049b0 0x7f57841026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f57837fe640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578410de10 0x7f5784103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 -- 172.21.15.31:0/3220299875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578410de10 msgr2=0x7f5784103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578410de10 0x7f5784103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 -- 172.21.15.31:0/3220299875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57841049b0 msgr2=0x7f57841026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57841049b0 0x7f57841026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 -- 172.21.15.31:0/3220299875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57841a8860 con 0x7f5784105300 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f577bfff640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 0x7f5784102c30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d750 tx=0x7f576c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5780ff9640 1 -- 172.21.15.31:0/3220299875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c017070 con 0x7f5784105300 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57841a8b50 con 0x7f5784105300 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5780ff9640 1 -- 172.21.15.31:0/3220299875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576c0054d0 con 0x7f5784105300 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5780ff9640 1 -- 172.21.15.31:0/3220299875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c004c30 con 0x7f5784105300 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.445+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a9040 con 0x7f5784105300 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 Processor -- start 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 -- start start 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd60602070 0x55bd609a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 0x55bd609a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bd6091fea0 con 0x55bd605e1390 2024-09-21T02:59:49.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd6090f2d0 con 0x55bd60602070 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f578fca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd609130e0 con 0x55bd605d91b0 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.449+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 0x55bd609a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5780ff9640 1 -- 172.21.15.31:0/3220299875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f576c004dd0 con 0x7f5784105300 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 0x55bd609a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54354/0 (socket says 172.21.15.31:54354) 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 -- 172.21.15.31:0/3276203545 learned_addr learned my addr 172.21.15.31:0/3276203545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f57837fe640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd60602070 0x55bd609a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f577bfff640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 -- 172.21.15.31:0/3276203545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd60602070 msgr2=0x55bd609a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd60602070 0x55bd609a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 -- 172.21.15.31:0/3276203545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 msgr2=0x55bd609a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 -- 172.21.15.31:0/3276203545 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6066d650 con 0x55bd605d91b0 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f57837fe640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd60602070 0x55bd609a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:49.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f577bfff640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.450+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 0x55bd609a9800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57740045e0 tx=0x7f5774004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3032279418c0e184 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.451+0000 7f57797fa640 1 -- 172.21.15.31:0/3276203545 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774003dd0 con 0x55bd605d91b0 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.451+0000 7f57797fa640 1 -- 172.21.15.31:0/3276203545 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5774015070 con 0x55bd605d91b0 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.451+0000 7f57797fa640 1 -- 172.21.15.31:0/3276203545 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774011640 con 0x55bd605d91b0 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.451+0000 7f5780ff9640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f57700b6680 0x7f57700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.451+0000 7f57837fe640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f57700b6680 0x7f57700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.451+0000 7f5780ff9640 1 -- 172.21.15.31:0/3220299875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f576c0e7090 con 0x7f5784105300 2024-09-21T02:59:49.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 -- 172.21.15.31:0/3276203545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 msgr2=0x55bd609a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f57837fe640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f57700b6680 0x7f57700b8b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f577c02f500 tx=0x7f577c009820 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 0x55bd609a9800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57740045e0 tx=0x7f5774004610 comp rx=0 tx=0).stop 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 -- 172.21.15.31:0/3276203545 shutdown_connections 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605d91b0 0x55bd609a9800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 --2- 172.21.15.31:0/3276203545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd60602070 0x55bd609a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 -- 172.21.15.31:0/3276203545 >> 172.21.15.31:0/3276203545 conn(0x55bd605f6460 msgr2=0x55bd609a71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 -- 172.21.15.31:0/3276203545 shutdown_connections 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 -- 172.21.15.31:0/3276203545 wait complete. 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 -- start start 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.452+0000 7f578fca4740 1 Processor -- start 2024-09-21T02:59:49.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd60602070 0x55bd609c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bd60770700 con 0x55bd605e1390 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd608aa160 con 0x55bd605d91b0 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd609b7aa0 con 0x55bd60602070 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f578fca4740 10 client.0 _ll_register_callbacks cb 0x55bd608c39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50118/0 (socket says 172.21.15.31:50118) 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 -- 172.21.15.31:0/4047097215 learned_addr learned my addr 172.21.15.31:0/4047097215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f5782ffd640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f57837fe640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd60602070 0x55bd609c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 -- 172.21.15.31:0/4047097215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd60602070 msgr2=0x55bd609c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd60602070 0x55bd609c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 -- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 msgr2=0x55bd609b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.453+0000 7f577bfff640 1 -- 172.21.15.31:0/4047097215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd608d7c90 con 0x55bd605e1390 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f577bfff640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609b4e80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f576c0aa1c0 tx=0x7f576c0acde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dfaaaa46ffbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f576a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f576a7fc640 1 -- 172.21.15.31:0/4047097215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c020070 con 0x55bd605e1390 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f576a7fc640 1 -- 172.21.15.31:0/4047097215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f576c0e90d0 con 0x55bd605e1390 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5774015070 con 0x55bd605e1390 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f576a7fc640 1 -- 172.21.15.31:0/4047097215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f576c0e4c20 con 0x55bd605e1390 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 10 client.17088 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd609c3800 con 0x55bd605e1390 2024-09-21T02:59:49.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f576a7fc640 1 -- 172.21.15.31:0/4047097215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5774015070 con 0x55bd605e1390 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f576a7fc640 1 -- 172.21.15.31:0/4047097215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f576c0940f0 con 0x55bd605e1390 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 10 client.17088 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 10 client.17088 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 20 client.17088 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd605c09f0 0x55bd609c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 4 client.17088 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.454+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd60921df0 con 0x55bd605c09f0 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.455+0000 7f5782ffd640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd605c09f0 0x55bd609c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.455+0000 7f576a7fc640 10 client.17088 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.455+0000 7f5782ffd640 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd605c09f0 0x55bd609c6d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=60ef5bc801298b7d server_cookie=4521a557b356df4d in_seq=0 out_seq=0 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f576a7fc640 1 -- 172.21.15.31:0/4047097215 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd60921df0 con 0x55bd605c09f0 2024-09-21T02:59:49.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f576a7fc640 10 client.17088 handle_command_reply: tid=0 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 client.17088 shutdown 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 msgr2=0x55bd609b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609b4e80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f576c0aa1c0 tx=0x7f576c0acde0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 shutdown_connections 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd605c09f0 0x55bd609c6d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f576a7fc640 0 client.17088 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd60602070 0x55bd609c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd609b4e80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 --2- 172.21.15.31:0/4047097215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 >> 172.21.15.31:0/4047097215 conn(0x55bd605f6460 msgr2=0x55bd609ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 shutdown_connections 2024-09-21T02:59:49.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 1 -- 172.21.15.31:0/4047097215 wait complete. 2024-09-21T02:59:49.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.456+0000 7f578fca4740 20 client.17088 trim_cache size 0 max 16384 2024-09-21T02:59:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 Processor -- start 2024-09-21T02:59:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 -- start start 2024-09-21T02:59:49.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 0x55bd609aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605e1390 0x55bd609c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b1940 con 0x55bd605d91b0 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b3480 con 0x55bd605e1390 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f578fca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b2010 con 0x55bd605c09f0 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f577bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 0x55bd609aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f577bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 0x55bd609aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50132/0 (socket says 172.21.15.31:50132) 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f577bfff640 1 -- 172.21.15.31:0/2254544477 learned_addr learned my addr 172.21.15.31:0/2254544477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f5782ffd640 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f577bfff640 1 -- 172.21.15.31:0/2254544477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 msgr2=0x55bd609b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f577bfff640 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f577bfff640 1 -- 172.21.15.31:0/2254544477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605e1390 msgr2=0x55bd609c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.470+0000 7f57837fe640 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605e1390 0x55bd609c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f577bfff640 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605e1390 0x55bd609c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f577bfff640 1 -- 172.21.15.31:0/2254544477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd609a1460 con 0x55bd605d91b0 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f57837fe640 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605e1390 0x55bd609c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f577bfff640 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 0x55bd609aa9b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f576c009f40 tx=0x7f576c0a8df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e104bfee79acb62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f5778ff9640 1 -- 172.21.15.31:0/2254544477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0b68c0 con 0x55bd605d91b0 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f5778ff9640 1 -- 172.21.15.31:0/2254544477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576c0b1980 con 0x55bd605d91b0 2024-09-21T02:59:49.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.471+0000 7f5778ff9640 1 -- 172.21.15.31:0/2254544477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0a8970 con 0x55bd605d91b0 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 -- 172.21.15.31:0/2254544477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 msgr2=0x55bd609aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 0x55bd609aa9b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f576c009f40 tx=0x7f576c0a8df0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 -- 172.21.15.31:0/2254544477 shutdown_connections 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605e1390 0x55bd609c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605d91b0 0x55bd609aa9b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 --2- 172.21.15.31:0/2254544477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 -- 172.21.15.31:0/2254544477 >> 172.21.15.31:0/2254544477 conn(0x55bd605f6460 msgr2=0x55bd609aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 -- 172.21.15.31:0/2254544477 shutdown_connections 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 -- 172.21.15.31:0/2254544477 wait complete. 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 -- start start 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.472+0000 7f578fca4740 1 Processor -- start 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd60a30700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b2db0 con 0x55bd605e1390 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b26e0 con 0x55bd605d91b0 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57841af920 con 0x55bd605c09f0 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f578fca4740 10 client.0 _ll_register_callbacks cb 0x7f576c00d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd60a30700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd60a30700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50140/0 (socket says 172.21.15.31:50140) 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 -- 172.21.15.31:0/3082513148 learned_addr learned my addr 172.21.15.31:0/3082513148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f577bfff640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 -- 172.21.15.31:0/3082513148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 msgr2=0x55bd609b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 -- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 msgr2=0x55bd609b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 -- 172.21.15.31:0/3082513148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd609a25d0 con 0x55bd605e1390 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f57837fe640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd60a30700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f577c0316f0 tx=0x7f577c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f0c5ff4eeb5fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.473+0000 7f576a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f576a7fc640 1 -- 172.21.15.31:0/3082513148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c004040 con 0x55bd605e1390 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f576a7fc640 1 -- 172.21.15.31:0/3082513148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f577c042030 con 0x55bd605e1390 2024-09-21T02:59:49.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c0b1980 con 0x55bd605e1390 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f576a7fc640 1 -- 172.21.15.31:0/3082513148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f577c0041e0 con 0x55bd605e1390 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 10 client.17094 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd60a31c30 con 0x55bd605e1390 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f576a7fc640 1 -- 172.21.15.31:0/3082513148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f576c0b1980 con 0x55bd605e1390 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f576a7fc640 1 -- 172.21.15.31:0/3082513148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f577c05b070 con 0x55bd605e1390 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 10 client.17094 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 10 client.17094 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 20 client.17094 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd60602070 0x55bd60a34450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 4 client.17094 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.474+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd6091d8a0 con 0x55bd60602070 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd60602070 0x55bd60a34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f5782ffd640 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd60602070 0x55bd60a34450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b122b42a763c96ac server_cookie=6bd43a52c532ca32 in_seq=0 out_seq=0 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f576a7fc640 10 client.17094 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f576a7fc640 1 -- 172.21.15.31:0/3082513148 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd6091d8a0 con 0x55bd60602070 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f576a7fc640 10 client.17094 handle_command_reply: tid=0 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 client.17094 shutdown 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 msgr2=0x55bd60a30700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd60a30700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f577c0316f0 tx=0x7f577c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 shutdown_connections 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd605e1390 0x55bd60a30700 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd605d91b0 0x55bd609b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55bd60602070 0x55bd60a34450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 --2- 172.21.15.31:0/3082513148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd605c09f0 0x55bd609b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f576a7fc640 0 client.17094 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T02:59:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.475+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 >> 172.21.15.31:0/3082513148 conn(0x55bd605f6460 msgr2=0x55bd609aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 shutdown_connections 2024-09-21T02:59:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f578fca4740 1 -- 172.21.15.31:0/3082513148 wait complete. 2024-09-21T02:59:49.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f578fca4740 20 client.17094 trim_cache size 0 max 16384 2024-09-21T02:59:49.483 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f57700b6680 msgr2=0x7f57700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f57700b6680 0x7f57700b8b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f577c02f500 tx=0x7f577c009820 comp rx=0 tx=0).stop 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 msgr2=0x7f5784102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 0x7f5784102c30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d750 tx=0x7f576c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 shutdown_connections 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f57700b6680 0x7f57700b8b90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f578410de10 0x7f5784103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5784105300 0x7f5784102c30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 --2- 172.21.15.31:0/3220299875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57841049b0 0x7f57841026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 >> 172.21.15.31:0/3220299875 conn(0x7f5784079730 msgr2=0x7f57840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 shutdown_connections 2024-09-21T02:59:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:49.476+0000 7f5789730640 1 -- 172.21.15.31:0/3220299875 wait complete. 2024-09-21T02:59:50.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T02:59:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:50.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:50 smithi031 ceph-mon[31209]: pgmap v533: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 -- 172.21.15.31:0/2009486934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4600107b70 msgr2=0x7f4600107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 --2- 172.21.15.31:0/2009486934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4600107b70 0x7f4600107f80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009900 tx=0x7f45f002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 -- 172.21.15.31:0/2009486934 shutdown_connections 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 --2- 172.21.15.31:0/2009486934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46001084c0 0x7f460010ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 --2- 172.21.15.31:0/2009486934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4600107b70 0x7f4600107f80 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 --2- 172.21.15.31:0/2009486934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4600102230 0x7f4600102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 -- 172.21.15.31:0/2009486934 >> 172.21.15.31:0/2009486934 conn(0x7f4600079e40 msgr2=0x7f460007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 -- 172.21.15.31:0/2009486934 shutdown_connections 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.061+0000 7f4607bd6640 1 -- 172.21.15.31:0/2009486934 wait complete. 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 Processor -- start 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 -- start start 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4600102230 0x7f46000fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4600107b70 0x7f46000ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 0x7f46000ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4600112150 con 0x7f46001084c0 2024-09-21T02:59:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4600111fd0 con 0x7f4600102230 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4607bd6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46001122d0 con 0x7f4600107b70 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4606173640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 0x7f46000ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4606173640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 0x7f46000ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50146/0 (socket says 172.21.15.31:50146) 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.062+0000 7f4606173640 1 -- 172.21.15.31:0/1884845627 learned_addr learned my addr 172.21.15.31:0/1884845627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4605972640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4600102230 0x7f46000fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4605171640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4600107b70 0x7f46000ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4606173640 1 -- 172.21.15.31:0/1884845627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4600107b70 msgr2=0x7f46000ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4606173640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4600107b70 0x7f46000ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4606173640 1 -- 172.21.15.31:0/1884845627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4600102230 msgr2=0x7f46000fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4606173640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4600102230 0x7f46000fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4606173640 1 -- 172.21.15.31:0/1884845627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001a86e0 con 0x7f46001084c0 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4606173640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 0x7f46000ff8f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f45f800cce0 tx=0x7f45f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f45f6ffd640 1 -- 172.21.15.31:0/1884845627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800f450 con 0x7f46001084c0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f45f6ffd640 1 -- 172.21.15.31:0/1884845627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f800faa0 con 0x7f46001084c0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f45f6ffd640 1 -- 172.21.15.31:0/1884845627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8018b70 con 0x7f46001084c0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001a89d0 con 0x7f46001084c0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.063+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a8e90 con 0x7f46001084c0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.066+0000 7f460e14a740 1 Processor -- start 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.066+0000 7f45f6ffd640 1 -- 172.21.15.31:0/1884845627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f45f8018d10 con 0x7f46001084c0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.066+0000 7f460e14a740 1 -- start start 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f460e14a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 0x55a34e6e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f460e14a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6e5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f460e14a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f460e14a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a34e65bea0 con 0x55a34e33e070 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f460e14a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a34e64b2d0 con 0x55a34e3151b0 2024-09-21T02:59:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f460e14a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a34e64f0e0 con 0x55a34e31d390 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 0x55a34e6e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 0x55a34e6e56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50148/0 (socket says 172.21.15.31:50148) 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 -- 172.21.15.31:0/359390514 learned_addr learned my addr 172.21.15.31:0/359390514 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f45f6ffd640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45ec0b6680 0x7f45ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605972640 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6e5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 -- 172.21.15.31:0/359390514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 msgr2=0x55a34e6e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6e6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 -- 172.21.15.31:0/359390514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 msgr2=0x55a34e6e5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6e5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605171640 1 -- 172.21.15.31:0/359390514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34e3a9650 con 0x55a34e33e070 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4606173640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45ec0b6680 0x7f45ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f4605972640 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6e5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.067+0000 7f45f6ffd640 1 -- 172.21.15.31:0/1884845627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f45f80e7ae0 con 0x7f46001084c0 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.068+0000 7f4605171640 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 0x55a34e6e56a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009900 tx=0x7f45f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7deaa041780ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.068+0000 7f45ebfff640 1 -- 172.21.15.31:0/359390514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0035040 con 0x55a34e33e070 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.068+0000 7f45ebfff640 1 -- 172.21.15.31:0/359390514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f0042070 con 0x55a34e33e070 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.068+0000 7f45ebfff640 1 -- 172.21.15.31:0/359390514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00047a0 con 0x55a34e33e070 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 -- 172.21.15.31:0/359390514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 msgr2=0x55a34e6e56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 0x55a34e6e56a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009900 tx=0x7f45f0000bc0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f4606173640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45ec0b6680 0x7f45ec0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f45f80e5850 tx=0x7f45f807ac30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 -- 172.21.15.31:0/359390514 shutdown_connections 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6e5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 --2- 172.21.15.31:0/359390514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e33e070 0x55a34e6e56a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 -- 172.21.15.31:0/359390514 >> 172.21.15.31:0/359390514 conn(0x55a34e332460 msgr2=0x55a34e6e3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 -- 172.21.15.31:0/359390514 shutdown_connections 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 -- 172.21.15.31:0/359390514 wait complete. 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 -- start start 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.069+0000 7f460e14a740 1 Processor -- start 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e33e070 0x55a34e6fd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a34e4ac700 con 0x55a34e31d390 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a34e5e6160 con 0x55a34e3151b0 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a34e6f3e20 con 0x55a34e33e070 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f460e14a740 10 client.0 _ll_register_callbacks cb 0x55a34e5ff9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50160/0 (socket says 172.21.15.31:50160) 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 -- 172.21.15.31:0/1269542226 learned_addr learned my addr 172.21.15.31:0/1269542226 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605171640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605171640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e33e070 0x55a34e6fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 -- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e33e070 msgr2=0x55a34e6fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e33e070 0x55a34e6fd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 -- 172.21.15.31:0/1269542226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 msgr2=0x55a34e6f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605972640 1 -- 172.21.15.31:0/1269542226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34e613c90 con 0x55a34e31d390 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.070+0000 7f4605171640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f4605972640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f12b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0116b0 tx=0x7f45fc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a900db70ca8cba90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f45e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f45e97fa640 1 -- 172.21.15.31:0/1269542226 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc01f060 con 0x55a34e31d390 2024-09-21T02:59:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f45e97fa640 1 -- 172.21.15.31:0/1269542226 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f45fc00fb40 con 0x55a34e31d390 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f0042070 con 0x55a34e31d390 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f45e97fa640 1 -- 172.21.15.31:0/1269542226 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45fc00fd40 con 0x55a34e31d390 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 10 client.17106 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a34e6ffb20 con 0x55a34e31d390 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f45e97fa640 1 -- 172.21.15.31:0/1269542226 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f0042070 con 0x55a34e31d390 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f45e97fa640 1 -- 172.21.15.31:0/1269542226 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f45fc0224b0 con 0x55a34e31d390 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 10 client.17106 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 10 client.17106 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 20 client.17106 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e2fc9f0 0x55a34e703050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 4 client.17106 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.071+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a34e65ddf0 con 0x55a34e2fc9f0 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.072+0000 7f4605171640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e2fc9f0 0x55a34e703050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.072+0000 7f45e97fa640 10 client.17106 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.072+0000 7f4605171640 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e2fc9f0 0x55a34e703050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba8d8f5c5d4a513 server_cookie=bb6f6f10b40e9b5e in_seq=0 out_seq=0 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f45e97fa640 1 -- 172.21.15.31:0/1269542226 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a34e65ddf0 con 0x55a34e2fc9f0 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f45e97fa640 10 client.17106 handle_command_reply: tid=0 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 client.17106 shutdown 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 msgr2=0x55a34e6f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f12b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0116b0 tx=0x7f45fc011b70 comp rx=0 tx=0).stop 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 shutdown_connections 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e2fc9f0 0x55a34e703050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e33e070 0x55a34e6fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f12b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f45e97fa640 0 client.17106 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 --2- 172.21.15.31:0/1269542226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 >> 172.21.15.31:0/1269542226 conn(0x55a34e332460 msgr2=0x55a34e6e8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 shutdown_connections 2024-09-21T02:59:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 1 -- 172.21.15.31:0/1269542226 wait complete. 2024-09-21T02:59:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.073+0000 7f460e14a740 20 client.17106 trim_cache size 0 max 16384 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 Processor -- start 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 -- start start 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 0x55a34e6e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b16b0 con 0x55a34e2fc9f0 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b31f0 con 0x55a34e3151b0 2024-09-21T02:59:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.087+0000 7f460e14a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b1d80 con 0x55a34e31d390 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 0x55a34e6e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 0x55a34e6e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50166/0 (socket says 172.21.15.31:50166) 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 -- 172.21.15.31:0/441778298 learned_addr learned my addr 172.21.15.31:0/441778298 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605972640 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 -- 172.21.15.31:0/441778298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 msgr2=0x55a34e6fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 -- 172.21.15.31:0/441778298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 msgr2=0x55a34e6f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 -- 172.21.15.31:0/441778298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34e6e8500 con 0x55a34e2fc9f0 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f4605171640 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 0x55a34e6e7090 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005280 tx=0x7f45f00052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ab0466bab3166d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f45eb7fe640 1 -- 172.21.15.31:0/441778298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0035040 con 0x55a34e2fc9f0 2024-09-21T02:59:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f45eb7fe640 1 -- 172.21.15.31:0/441778298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f0042040 con 0x55a34e2fc9f0 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.088+0000 7f45eb7fe640 1 -- 172.21.15.31:0/441778298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0046670 con 0x55a34e2fc9f0 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 -- 172.21.15.31:0/441778298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 msgr2=0x55a34e6e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 0x55a34e6e7090 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f45f0005280 tx=0x7f45f00052b0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 -- 172.21.15.31:0/441778298 shutdown_connections 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e31d390 0x55a34e6fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 --2- 172.21.15.31:0/441778298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e2fc9f0 0x55a34e6e7090 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 -- 172.21.15.31:0/441778298 >> 172.21.15.31:0/441778298 conn(0x55a34e332460 msgr2=0x55a34e6f2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 -- 172.21.15.31:0/441778298 shutdown_connections 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.089+0000 7f460e14a740 1 -- 172.21.15.31:0/441778298 wait complete. 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 -- start start 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 Processor -- start 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e2fc9f0 0x55a34e6f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b2b20 con 0x55a34e31d390 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b2450 con 0x55a34e3151b0 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46001af690 con 0x55a34e2fc9f0 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f460e14a740 10 client.0 _ll_register_callbacks cb 0x7f45fc00a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f4605171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f4605171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50172/0 (socket says 172.21.15.31:50172) 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.090+0000 7f4605171640 1 -- 172.21.15.31:0/2955354577 learned_addr learned my addr 172.21.15.31:0/2955354577 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e2fc9f0 0x55a34e6f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605972640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 -- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e2fc9f0 msgr2=0x55a34e6f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e2fc9f0 0x55a34e6f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 -- 172.21.15.31:0/2955354577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 msgr2=0x55a34e6f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 -- 172.21.15.31:0/2955354577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34e76cb90 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f4605171640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f5ca0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55a34e6d49f0 tx=0x7f45f0004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6399a3bfe9360243 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f45e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f45e97fa640 1 -- 172.21.15.31:0/2955354577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0035070 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f45e97fa640 1 -- 172.21.15.31:0/2955354577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f45f0004a60 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f0042040 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f45e97fa640 1 -- 172.21.15.31:0/2955354577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45f0004020 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f460e14a740 10 client.17112 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a34e76de10 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.091+0000 7f45e97fa640 1 -- 172.21.15.31:0/2955354577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f0042040 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f45e97fa640 1 -- 172.21.15.31:0/2955354577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f45f0059020 con 0x55a34e31d390 2024-09-21T02:59:51.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f460e14a740 10 client.17112 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f460e14a740 10 client.17112 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f460e14a740 20 client.17112 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f460e14a740 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e33e070 0x55a34e76fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f460e14a740 4 client.17112 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a34e6598a0 con 0x55a34e33e070 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.092+0000 7f4605972640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e33e070 0x55a34e76fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f4605972640 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e33e070 0x55a34e76fff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9b82523f23c4afea server_cookie=91df6c097ee0786b in_seq=0 out_seq=0 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f45e97fa640 10 client.17112 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f45e97fa640 1 -- 172.21.15.31:0/2955354577 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a34e6598a0 con 0x55a34e33e070 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f45e97fa640 10 client.17112 handle_command_reply: tid=0 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 client.17112 shutdown 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 msgr2=0x55a34e6f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f5ca0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55a34e6d49f0 tx=0x7f45f0004df0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 shutdown_connections 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a34e33e070 0x55a34e76fff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a34e31d390 0x55a34e6f5ca0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a34e3151b0 0x55a34e6f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f45e97fa640 0 client.17112 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 --2- 172.21.15.31:0/2955354577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a34e2fc9f0 0x55a34e6f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 >> 172.21.15.31:0/2955354577 conn(0x55a34e332460 msgr2=0x55a34e6f2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 shutdown_connections 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 1 -- 172.21.15.31:0/2955354577 wait complete. 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.093+0000 7f460e14a740 20 client.17112 trim_cache size 0 max 16384 2024-09-21T02:59:51.107 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45ec0b6680 msgr2=0x7f45ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45ec0b6680 0x7f45ec0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f45f80e5850 tx=0x7f45f807ac30 comp rx=0 tx=0).stop 2024-09-21T02:59:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 msgr2=0x7f46000ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 0x7f46000ff8f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f45f800cce0 tx=0x7f45f800a620 comp rx=0 tx=0).stop 2024-09-21T02:59:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 shutdown_connections 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f45ec0b6680 0x7f45ec0b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4600107b70 0x7f46000ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46001084c0 0x7f46000ff8f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 --2- 172.21.15.31:0/1884845627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4600102230 0x7f46000fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 >> 172.21.15.31:0/1884845627 conn(0x7f4600079e40 msgr2=0x7f4600108ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 shutdown_connections 2024-09-21T02:59:51.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:51.094+0000 7f4607bd6640 1 -- 172.21.15.31:0/1884845627 wait complete. 2024-09-21T02:59:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:50 smithi165 ceph-mon[34818]: pgmap v533: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:50 smithi178 ceph-mon[34759]: pgmap v533: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:51.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T02:59:52.166 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 -- 172.21.15.31:0/1743959684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74010de10 msgr2=0x7fb7401102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 --2- 172.21.15.31:0/1743959684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74010de10 0x7fb7401102f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb738009ca0 tx=0x7fb73802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 -- 172.21.15.31:0/1743959684 shutdown_connections 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 --2- 172.21.15.31:0/1743959684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb74010de10 0x7fb7401102f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 --2- 172.21.15.31:0/1743959684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb740105290 0x7fb74010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 --2- 172.21.15.31:0/1743959684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb740104860 0x7fb740104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 -- 172.21.15.31:0/1743959684 >> 172.21.15.31:0/1743959684 conn(0x7fb740079730 msgr2=0x7fb740079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 -- 172.21.15.31:0/1743959684 shutdown_connections 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.680+0000 7fb748558640 1 -- 172.21.15.31:0/1743959684 wait complete. 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.681+0000 7fb748558640 1 Processor -- start 2024-09-21T02:59:52.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.681+0000 7fb748558640 1 -- start start 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.681+0000 7fb748558640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 0x7fb74019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb748558640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb740105290 0x7fb74019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb748558640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74010de10 0x7fb74019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 0x7fb74019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb748558640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb740113a80 con 0x7fb740104860 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 0x7fb74019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50190/0 (socket says 172.21.15.31:50190) 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 -- 172.21.15.31:0/3414259563 learned_addr learned my addr 172.21.15.31:0/3414259563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb748558640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb740113900 con 0x7fb74010de10 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb740113c00 con 0x7fb740105290 2024-09-21T02:59:52.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb745af3640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb740105290 0x7fb74019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb746af5640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74010de10 0x7fb74019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 -- 172.21.15.31:0/3414259563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb740105290 msgr2=0x7fb74019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb740105290 0x7fb74019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 -- 172.21.15.31:0/3414259563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74010de10 msgr2=0x7fb74019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74010de10 0x7fb74019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 -- 172.21.15.31:0/3414259563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7401a03a0 con 0x7fb740104860 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.682+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 0x7fb74019a430 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00e9f0 tx=0x7fb73c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.683+0000 7fb7377fe640 1 -- 172.21.15.31:0/3414259563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00cd80 con 0x7fb740104860 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.683+0000 7fb7377fe640 1 -- 172.21.15.31:0/3414259563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb73c0054d0 con 0x7fb740104860 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.683+0000 7fb7377fe640 1 -- 172.21.15.31:0/3414259563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c010690 con 0x7fb740104860 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.683+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7401a0690 con 0x7fb740104860 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.683+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7401a6df0 con 0x7fb740104860 2024-09-21T02:59:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.685+0000 7fb74eacc740 1 Processor -- start 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.685+0000 7fb74eacc740 1 -- start start 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2d8070 0x55806e67f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e67fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e680120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb74eacc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55806e5f5ea0 con 0x55806e2b7390 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb74eacc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55806e5e52d0 con 0x55806e2af1b0 2024-09-21T02:59:52.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb74eacc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55806e5e90e0 con 0x55806e2d8070 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb7377fe640 1 -- 172.21.15.31:0/3414259563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb73c004e90 con 0x7fb740104860 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e680120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e680120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50194/0 (socket says 172.21.15.31:50194) 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 -- 172.21.15.31:0/2073634939 learned_addr learned my addr 172.21.15.31:0/2073634939 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb746af5640 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2d8070 0x55806e67f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb7462f4640 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e67fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 -- 172.21.15.31:0/2073634939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2d8070 msgr2=0x55806e67f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2d8070 0x55806e67f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 -- 172.21.15.31:0/2073634939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 msgr2=0x55806e67fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e67fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb745af3640 1 -- 172.21.15.31:0/2073634939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806e343650 con 0x55806e2b7390 2024-09-21T02:59:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.686+0000 7fb7462f4640 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e67fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb745af3640 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e680120 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb73000bc60 tx=0x7fb7300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10de06adfd994b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb734ff9640 1 -- 172.21.15.31:0/2073634939 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73001c480 con 0x55806e2b7390 2024-09-21T02:59:52.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb734ff9640 1 -- 172.21.15.31:0/2073634939 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb73001cad0 con 0x55806e2b7390 2024-09-21T02:59:52.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb734ff9640 1 -- 172.21.15.31:0/2073634939 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730011b20 con 0x55806e2b7390 2024-09-21T02:59:52.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb7377fe640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb72c0b6700 0x7fb72c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb746af5640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb72c0b6700 0x7fb72c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.687+0000 7fb7377fe640 1 -- 172.21.15.31:0/3414259563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb73c0e7a70 con 0x7fb740104860 2024-09-21T02:59:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 -- 172.21.15.31:0/2073634939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 msgr2=0x55806e680120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb746af5640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb72c0b6700 0x7fb72c0b8c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb73802f500 tx=0x7fb73802f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e680120 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb73000bc60 tx=0x7fb7300107a0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 -- 172.21.15.31:0/2073634939 shutdown_connections 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e680120 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e67fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2073634939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2d8070 0x55806e67f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 -- 172.21.15.31:0/2073634939 >> 172.21.15.31:0/2073634939 conn(0x55806e2cc460 msgr2=0x55806e67d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 -- 172.21.15.31:0/2073634939 shutdown_connections 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 -- 172.21.15.31:0/2073634939 wait complete. 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.688+0000 7fb74eacc740 1 -- start start 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 Processor -- start 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2af1b0 0x55806e68f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e68f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e69b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55806e446700 con 0x55806e2b7390 2024-09-21T02:59:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55806e580160 con 0x55806e2d8070 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55806e6921e0 con 0x55806e2af1b0 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb74eacc740 10 client.0 _ll_register_callbacks cb 0x55806e5999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e68f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e68f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50198/0 (socket says 172.21.15.31:50198) 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 -- 172.21.15.31:0/3574145385 learned_addr learned my addr 172.21.15.31:0/3574145385 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2af1b0 0x55806e68f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb746af5640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e69b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 -- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2af1b0 msgr2=0x55806e68f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2af1b0 0x55806e68f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 -- 172.21.15.31:0/3574145385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 msgr2=0x55806e69b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e69b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb745af3640 1 -- 172.21.15.31:0/3574145385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806e5adc90 con 0x55806e2b7390 2024-09-21T02:59:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2af1b0 0x55806e68f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.689+0000 7fb746af5640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e69b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb745af3640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e68f680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb73000bf80 tx=0x7fb730005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ee4a00b4c2fc08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb729ffb640 1 -- 172.21.15.31:0/3574145385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73002b060 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb729ffb640 1 -- 172.21.15.31:0/3574145385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb73001c690 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb73001cad0 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb729ffb640 1 -- 172.21.15.31:0/3574145385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb73001c890 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb74eacc740 10 client.17124 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55806e69de40 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb729ffb640 1 -- 172.21.15.31:0/3574145385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb73001cad0 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.690+0000 7fb729ffb640 1 -- 172.21.15.31:0/3574145385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb73001a880 con 0x55806e2b7390 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb74eacc740 10 client.17124 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:52.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb74eacc740 10 client.17124 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb74eacc740 20 client.17124 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e2969f0 0x55806e6a1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb74eacc740 4 client.17124 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55806e5f7df0 con 0x55806e2969f0 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e2969f0 0x55806e6a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.691+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e2969f0 0x55806e6a1250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bb3459b9e730c4a server_cookie=5795352f2b5a0eab in_seq=0 out_seq=0 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb729ffb640 10 client.17124 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb729ffb640 1 -- 172.21.15.31:0/3574145385 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55806e5f7df0 con 0x55806e2969f0 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb729ffb640 10 client.17124 handle_command_reply: tid=0 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 client.17124 shutdown 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 msgr2=0x55806e68f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e68f680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb73000bf80 tx=0x7fb730005760 comp rx=0 tx=0).stop 2024-09-21T02:59:52.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 shutdown_connections 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e2969f0 0x55806e6a1250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e69b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2b7390 0x55806e68f680 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3574145385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2af1b0 0x55806e68f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.692+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 >> 172.21.15.31:0/3574145385 conn(0x55806e2cc460 msgr2=0x55806e67bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.693+0000 7fb729ffb640 0 client.17124 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.693+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 shutdown_connections 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.693+0000 7fb74eacc740 1 -- 172.21.15.31:0/3574145385 wait complete. 2024-09-21T02:59:52.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.693+0000 7fb74eacc740 20 client.17124 trim_cache size 0 max 16384 2024-09-21T02:59:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 Processor -- start 2024-09-21T02:59:52.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 -- start start 2024-09-21T02:59:52.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e684e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e69b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 0x55806e68c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ad3b0 con 0x55806e2d8070 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401aeef0 con 0x55806e2af1b0 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb74eacc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ada80 con 0x55806e2b7390 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb746af5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 0x55806e68c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb746af5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 0x55806e68c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50200/0 (socket says 172.21.15.31:50200) 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.706+0000 7fb746af5640 1 -- 172.21.15.31:0/3588840635 learned_addr learned my addr 172.21.15.31:0/3588840635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb7462f4640 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e684e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb745af3640 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e69b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb746af5640 1 -- 172.21.15.31:0/3588840635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 msgr2=0x55806e69b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb746af5640 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e69b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb746af5640 1 -- 172.21.15.31:0/3588840635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 msgr2=0x55806e684e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb746af5640 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e684e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb746af5640 1 -- 172.21.15.31:0/3588840635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806e296c20 con 0x55806e2d8070 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb746af5640 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 0x55806e68c330 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb738035000 tx=0x7fb738041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e671d2d9dbcffde2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb72bfff640 1 -- 172.21.15.31:0/3588840635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738003970 con 0x55806e2d8070 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb72bfff640 1 -- 172.21.15.31:0/3588840635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb738036050 con 0x55806e2d8070 2024-09-21T02:59:52.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.707+0000 7fb72bfff640 1 -- 172.21.15.31:0/3588840635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73802f660 con 0x55806e2d8070 2024-09-21T02:59:52.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 -- 172.21.15.31:0/3588840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 msgr2=0x55806e68c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 0x55806e68c330 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb738035000 tx=0x7fb738041000 comp rx=0 tx=0).stop 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 -- 172.21.15.31:0/3588840635 shutdown_connections 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e69b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2d8070 0x55806e68c330 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 --2- 172.21.15.31:0/3588840635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2af1b0 0x55806e684e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 -- 172.21.15.31:0/3588840635 >> 172.21.15.31:0/3588840635 conn(0x55806e2cc460 msgr2=0x55806e684780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 -- 172.21.15.31:0/3588840635 shutdown_connections 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 -- 172.21.15.31:0/3588840635 wait complete. 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.708+0000 7fb74eacc740 1 -- start start 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 Processor -- start 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 0x55806e6a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e6a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e703b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ae820 con 0x55806e2af1b0 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ae150 con 0x55806e2d8070 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ab390 con 0x55806e2b7390 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb74eacc740 10 client.0 _ll_register_callbacks cb 0x7fb738009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb7462f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 0x55806e6a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb7462f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 0x55806e6a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50214/0 (socket says 172.21.15.31:50214) 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb7462f4640 1 -- 172.21.15.31:0/2156383030 learned_addr learned my addr 172.21.15.31:0/2156383030 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb746af5640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e703b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb745af3640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e6a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.709+0000 7fb7462f4640 1 -- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 msgr2=0x55806e6a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb7462f4640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e6a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb7462f4640 1 -- 172.21.15.31:0/2156383030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 msgr2=0x55806e703b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb7462f4640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e703b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb7462f4640 1 -- 172.21.15.31:0/2156383030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806e7041f0 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb746af5640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e703b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb7462f4640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 0x55806e6a1e10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0ad5e0 tx=0x7fb73c0e9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44050867bcc2b0c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb729ffb640 1 -- 172.21.15.31:0/2156383030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c005180 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb729ffb640 1 -- 172.21.15.31:0/2156383030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fb73c0a9b50 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb738036050 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb729ffb640 1 -- 172.21.15.31:0/2156383030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb73c0ad3f0 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb74eacc740 10 client.17130 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55806e70b640 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.710+0000 7fb729ffb640 1 -- 172.21.15.31:0/2156383030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb738036050 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb729ffb640 1 -- 172.21.15.31:0/2156383030 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb73c0a9350 con 0x55806e2af1b0 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb74eacc740 10 client.17130 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:52.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb74eacc740 10 client.17130 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb74eacc740 20 client.17130 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e26bda0 0x55806e70de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb74eacc740 4 client.17130 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55806e5f38a0 con 0x55806e26bda0 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb745af3640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e26bda0 0x55806e70de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb729ffb640 10 client.17130 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb745af3640 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e26bda0 0x55806e70de00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6591f8690e5139c8 server_cookie=95c76e6d592a5955 in_seq=0 out_seq=0 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.711+0000 7fb729ffb640 1 -- 172.21.15.31:0/2156383030 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55806e5f38a0 con 0x55806e26bda0 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb729ffb640 10 client.17130 handle_command_reply: tid=0 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 client.17130 shutdown 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 msgr2=0x55806e6a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 0x55806e6a1e10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0ad5e0 tx=0x7fb73c0e9f80 comp rx=0 tx=0).stop 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 shutdown_connections 2024-09-21T02:59:52.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55806e26bda0 0x55806e70de00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55806e2b7390 0x55806e6a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55806e2d8070 0x55806e703b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb729ffb640 0 client.17130 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 --2- 172.21.15.31:0/2156383030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55806e2af1b0 0x55806e6a1e10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 >> 172.21.15.31:0/2156383030 conn(0x55806e2cc460 msgr2=0x55806e6806d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 shutdown_connections 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 1 -- 172.21.15.31:0/2156383030 wait complete. 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb74eacc740 20 client.17130 trim_cache size 0 max 16384 2024-09-21T02:59:52.718 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb72c0b6700 msgr2=0x7fb72c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb748558640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb72c0b6700 0x7fb72c0b8c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb73802f500 tx=0x7fb73802f310 comp rx=0 tx=0).stop 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 msgr2=0x7fb74019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.712+0000 7fb748558640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 0x7fb74019a430 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00e9f0 tx=0x7fb73c00edf0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 shutdown_connections 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb72c0b6700 0x7fb72c0b8c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb740105290 0x7fb74019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74010de10 0x7fb74019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 --2- 172.21.15.31:0/3414259563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb740104860 0x7fb74019a430 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 >> 172.21.15.31:0/3414259563 conn(0x7fb740079730 msgr2=0x7fb740100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 shutdown_connections 2024-09-21T02:59:52.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:52.713+0000 7fb748558640 1 -- 172.21.15.31:0/3414259563 wait complete. 2024-09-21T02:59:53.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:52 smithi031 ceph-mon[31209]: pgmap v534: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:52 smithi165 ceph-mon[34818]: pgmap v534: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:52 smithi178 ceph-mon[34759]: pgmap v534: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T02:59:53.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T02:59:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:54.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 -- 172.21.15.31:0/1039256471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c107620 msgr2=0x7f561c107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1039256471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c107620 0x7f561c107a30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f560c009900 tx=0x7f560c02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 -- 172.21.15.31:0/1039256471 shutdown_connections 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1039256471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 0x7f561c10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1039256471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561c102f90 0x7f561c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1039256471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c107620 0x7f561c107a30 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.384+0000 7f5621e7b640 1 -- 172.21.15.31:0/1039256471 >> 172.21.15.31:0/1039256471 conn(0x7f561c079e30 msgr2=0x7f561c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.385+0000 7f5621e7b640 1 -- 172.21.15.31:0/1039256471 shutdown_connections 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.385+0000 7f5621e7b640 1 -- 172.21.15.31:0/1039256471 wait complete. 2024-09-21T02:59:54.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.385+0000 7f5621e7b640 1 Processor -- start 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.385+0000 7f5621e7b640 1 -- start start 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f5621e7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 0x7f561c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f5621e7b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 0x7f561c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f5621e7b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561c107620 0x7f561c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f5621e7b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1129d0 con 0x7f561c102f90 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f5621e7b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f561c112850 con 0x7f561c107620 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f5621e7b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f561c112b50 con 0x7f561c1038e0 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 0x7f561c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 0x7f561c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50234/0 (socket says 172.21.15.31:50234) 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 -- 172.21.15.31:0/1841731123 learned_addr learned my addr 172.21.15.31:0/1841731123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:54.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561affd640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 0x7f561c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561bfff640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561c107620 0x7f561c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 -- 172.21.15.31:0/1841731123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 msgr2=0x7f561c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 0x7f561c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 -- 172.21.15.31:0/1841731123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561c107620 msgr2=0x7f561c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561c107620 0x7f561c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561b7fe640 1 -- 172.21.15.31:0/1841731123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c1a0420 con 0x7f561c102f90 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.386+0000 7f561affd640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 0x7f561c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.387+0000 7f561b7fe640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 0x7f561c19a4b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f560c005940 tx=0x7f560c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.387+0000 7f5618ff9640 1 -- 172.21.15.31:0/1841731123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c03f030 con 0x7f561c102f90 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.387+0000 7f5618ff9640 1 -- 172.21.15.31:0/1841731123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c0044c0 con 0x7f561c102f90 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.387+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c1a06b0 con 0x7f561c102f90 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.387+0000 7f5618ff9640 1 -- 172.21.15.31:0/1841731123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c035070 con 0x7f561c102f90 2024-09-21T02:59:54.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.387+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c1a6e70 con 0x7f561c102f90 2024-09-21T02:59:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.389+0000 7f56283ef740 1 Processor -- start 2024-09-21T02:59:54.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 -- start start 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f5618ff9640 1 -- 172.21.15.31:0/1841731123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f560c042030 con 0x7f561c102f90 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f26800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f26d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac32e9cea0 con 0x55ac32b5e390 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac32e8c2d0 con 0x55ac32b561b0 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f56283ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac32e900e0 con 0x55ac32b7f070 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f26d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f26d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50236/0 (socket says 172.21.15.31:50236) 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561affd640 1 -- 172.21.15.31:0/3288928789 learned_addr learned my addr 172.21.15.31:0/3288928789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561b7fe640 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f26800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561bfff640 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561affd640 1 -- 172.21.15.31:0/3288928789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 msgr2=0x55ac32f262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.390+0000 7f561affd640 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f561affd640 1 -- 172.21.15.31:0/3288928789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 msgr2=0x55ac32f26800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f561affd640 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f26800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f561affd640 1 -- 172.21.15.31:0/3288928789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac32bea650 con 0x55ac32b5e390 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f5618ff9640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f56040b6680 0x7f56040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f561affd640 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f26d40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5610011470 tx=0x7f5610011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b5e7cd34354943 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f561bfff640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f56040b6680 0x7f56040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f5618ff9640 1 -- 172.21.15.31:0/1841731123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f560c107a30 con 0x7f561c102f90 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f5601ffb640 1 -- 172.21.15.31:0/3288928789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610004310 con 0x55ac32b5e390 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f5601ffb640 1 -- 172.21.15.31:0/3288928789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56100044b0 con 0x55ac32b5e390 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.391+0000 7f5601ffb640 1 -- 172.21.15.31:0/3288928789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56100035f0 con 0x55ac32b5e390 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 -- 172.21.15.31:0/3288928789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 msgr2=0x55ac32f26d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f561bfff640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f56040b6680 0x7f56040b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f5614015040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f26d40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5610011470 tx=0x7f5610011930 comp rx=0 tx=0).stop 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 -- 172.21.15.31:0/3288928789 shutdown_connections 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f26d40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f26800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 --2- 172.21.15.31:0/3288928789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 -- 172.21.15.31:0/3288928789 >> 172.21.15.31:0/3288928789 conn(0x55ac32b73460 msgr2=0x55ac32f241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 -- 172.21.15.31:0/3288928789 shutdown_connections 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.392+0000 7f56283ef740 1 -- 172.21.15.31:0/3288928789 wait complete. 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 -- start start 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 Processor -- start 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 0x55ac32f31870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b7f070 0x55ac32f322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac32ced700 con 0x55ac32b5e390 2024-09-21T02:59:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac32e27160 con 0x55ac32b7f070 2024-09-21T02:59:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac32f34aa0 con 0x55ac32b561b0 2024-09-21T02:59:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.393+0000 7f56283ef740 10 client.0 _ll_register_callbacks cb 0x55ac32e409e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 0x55ac32f31870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 0x55ac32f31870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54534/0 (socket says 172.21.15.31:54534) 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 -- 172.21.15.31:0/2671518794 learned_addr learned my addr 172.21.15.31:0/2671518794 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561bfff640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b7f070 0x55ac32f322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561affd640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 -- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b7f070 msgr2=0x55ac32f322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b7f070 0x55ac32f322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 -- 172.21.15.31:0/2671518794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 msgr2=0x55ac32f31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 -- 172.21.15.31:0/2671518794 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac32e54c90 con 0x55ac32b561b0 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561affd640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 0x55ac32f31870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f560c006010 tx=0x7f560c105650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d506fd8627de6de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f55faffd640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T02:59:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f55faffd640 1 -- 172.21.15.31:0/2671518794 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c049070 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56100044b0 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.394+0000 7f55faffd640 1 -- 172.21.15.31:0/2671518794 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f560c0d7030 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f55faffd640 1 -- 172.21.15.31:0/2671518794 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f560c0c97f0 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 10 client.26686 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac32f406a0 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f55faffd640 1 -- 172.21.15.31:0/2671518794 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c10e020 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f55faffd640 1 -- 172.21.15.31:0/2671518794 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f560c0cebd0 con 0x55ac32b561b0 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 10 client.26686 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 10 client.26686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 20 client.26686 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b3d9f0 0x55ac32f43b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 4 client.26686 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.395+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac32e9edf0 con 0x55ac32b3d9f0 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.396+0000 7f561affd640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b3d9f0 0x55ac32f43b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.396+0000 7f55faffd640 10 client.26686 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.396+0000 7f561affd640 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b3d9f0 0x55ac32f43b60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f169a6ded8f3eed server_cookie=5fe8b846cf930280 in_seq=0 out_seq=0 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f55faffd640 1 -- 172.21.15.31:0/2671518794 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac32e9edf0 con 0x55ac32b3d9f0 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f55faffd640 10 client.26686 handle_command_reply: tid=0 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 client.26686 shutdown 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 msgr2=0x55ac32f31870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 0x55ac32f31870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f560c006010 tx=0x7f560c105650 comp rx=0 tx=0).stop 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 shutdown_connections 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b3d9f0 0x55ac32f43b60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b7f070 0x55ac32f322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b5e390 0x55ac32f31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 --2- 172.21.15.31:0/2671518794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b561b0 0x55ac32f31870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f55faffd640 0 client.26686 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 >> 172.21.15.31:0/2671518794 conn(0x55ac32b73460 msgr2=0x55ac32f29450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 shutdown_connections 2024-09-21T02:59:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 1 -- 172.21.15.31:0/2671518794 wait complete. 2024-09-21T02:59:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.397+0000 7f56283ef740 20 client.26686 trim_cache size 0 max 16384 2024-09-21T02:59:54.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.410+0000 7f56283ef740 1 Processor -- start 2024-09-21T02:59:54.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.410+0000 7f56283ef740 1 -- start start 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f56283ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 0x55ac32f27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f56283ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b5e390 0x55ac32f3d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f56283ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f56283ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ad4e0 con 0x55ac32b561b0 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f56283ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1af020 con 0x55ac32b5e390 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f56283ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1adbb0 con 0x55ac32b7f070 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 0x55ac32f27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 0x55ac32f27cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50252/0 (socket says 172.21.15.31:50252) 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 -- 172.21.15.31:0/3819939747 learned_addr learned my addr 172.21.15.31:0/3819939747 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561bfff640 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561affd640 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b5e390 0x55ac32f3d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 -- 172.21.15.31:0/3819939747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 msgr2=0x55ac32f23760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f23760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 -- 172.21.15.31:0/3819939747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b5e390 msgr2=0x55ac32f3d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b5e390 0x55ac32f3d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 -- 172.21.15.31:0/3819939747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac32f1f5d0 con 0x55ac32b561b0 2024-09-21T02:59:54.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561affd640 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b5e390 0x55ac32f3d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:54.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.411+0000 7f561b7fe640 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 0x55ac32f27cb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f560c0cabc0 tx=0x7f560c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb6f51b4a53626a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56017fa640 1 -- 172.21.15.31:0/3819939747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c049070 con 0x55ac32b561b0 2024-09-21T02:59:54.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56017fa640 1 -- 172.21.15.31:0/3819939747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c10e070 con 0x55ac32b561b0 2024-09-21T02:59:54.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56017fa640 1 -- 172.21.15.31:0/3819939747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c0cd0c0 con 0x55ac32b561b0 2024-09-21T02:59:54.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 -- 172.21.15.31:0/3819939747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 msgr2=0x55ac32f27cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 0x55ac32f27cb0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f560c0cabc0 tx=0x7f560c005a20 comp rx=0 tx=0).stop 2024-09-21T02:59:54.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 -- 172.21.15.31:0/3819939747 shutdown_connections 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b7f070 0x55ac32f23760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b5e390 0x55ac32f3d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 --2- 172.21.15.31:0/3819939747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b561b0 0x55ac32f27cb0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 -- 172.21.15.31:0/3819939747 >> 172.21.15.31:0/3819939747 conn(0x55ac32b73460 msgr2=0x55ac32f3cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.412+0000 7f56283ef740 1 -- 172.21.15.31:0/3819939747 shutdown_connections 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 -- 172.21.15.31:0/3819939747 wait complete. 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 -- start start 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 Processor -- start 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f35dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b5e390 0x55ac32f36310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 0x55ac32f36850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ae950 con 0x55ac32b7f070 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ae280 con 0x55ac32b561b0 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.413+0000 7f56283ef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f561c1ab4c0 con 0x55ac32b5e390 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f56283ef740 10 client.0 _ll_register_callbacks cb 0x7f560c10b270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 0x55ac32f36850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 0x55ac32f36850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50256/0 (socket says 172.21.15.31:50256) 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561bfff640 1 -- 172.21.15.31:0/2528140531 learned_addr learned my addr 172.21.15.31:0/2528140531 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f35dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561affd640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b5e390 0x55ac32f36310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 -- 172.21.15.31:0/2528140531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b5e390 msgr2=0x55ac32f36310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b5e390 0x55ac32f36310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 -- 172.21.15.31:0/2528140531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 msgr2=0x55ac32f36850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 0x55ac32f36850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 -- 172.21.15.31:0/2528140531 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac32fad5f0 con 0x55ac32b561b0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561bfff640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 0x55ac32f36850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f561b7fe640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f35dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f560c005a90 tx=0x7f560c0d7040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16c2ef1654e8d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.414+0000 7f55faffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f55faffd640 1 -- 172.21.15.31:0/2528140531 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560c049040 con 0x55ac32b561b0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f560c10e070 con 0x55ac32b561b0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f55faffd640 1 -- 172.21.15.31:0/2528140531 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f560c0c9570 con 0x55ac32b561b0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f55faffd640 1 -- 172.21.15.31:0/2528140531 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f560c0cd2a0 con 0x55ac32b561b0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 10 client.26540 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac32fad5f0 con 0x55ac32b561b0 2024-09-21T02:59:54.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f55faffd640 1 -- 172.21.15.31:0/2528140531 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f560c10e070 con 0x55ac32b561b0 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f55faffd640 1 -- 172.21.15.31:0/2528140531 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f560c0cc620 con 0x55ac32b561b0 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 10 client.26540 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 10 client.26540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 20 client.26540 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b12da0 0x55ac32fb0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 4 client.26540 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.415+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac32e9a8a0 con 0x55ac32b12da0 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.416+0000 7f561affd640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b12da0 0x55ac32fb0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.416+0000 7f561affd640 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b12da0 0x55ac32fb0b30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10f7ff53cf59e8f server_cookie=ccbc6c4c99e5d61c in_seq=0 out_seq=0 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.416+0000 7f55faffd640 10 client.26540 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.416+0000 7f55faffd640 1 -- 172.21.15.31:0/2528140531 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac32e9a8a0 con 0x55ac32b12da0 2024-09-21T02:59:54.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.416+0000 7f55faffd640 10 client.26540 handle_command_reply: tid=0 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.416+0000 7f56283ef740 1 client.26540 shutdown 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 msgr2=0x55ac32f35dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f35dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f560c005a90 tx=0x7f560c0d7040 comp rx=0 tx=0).stop 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 shutdown_connections 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55ac32b12da0 0x55ac32fb0b30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac32b5e390 0x55ac32f36310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac32b7f070 0x55ac32f36850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f55faffd640 0 client.26540 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 --2- 172.21.15.31:0/2528140531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac32b561b0 0x55ac32f35dd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 >> 172.21.15.31:0/2528140531 conn(0x55ac32b73460 msgr2=0x55ac32f27a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 shutdown_connections 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 1 -- 172.21.15.31:0/2528140531 wait complete. 2024-09-21T02:59:54.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f56283ef740 20 client.26540 trim_cache size 0 max 16384 2024-09-21T02:59:54.423 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:54.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f56040b6680 msgr2=0x7f56040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f56040b6680 0x7f56040b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f5614015040 comp rx=0 tx=0).stop 2024-09-21T02:59:54.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 msgr2=0x7f561c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 0x7f561c19a4b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f560c005940 tx=0x7f560c0040c0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 shutdown_connections 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f56040b6680 0x7f56040b8b90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561c1038e0 0x7f561c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561c107620 0x7f561c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 --2- 172.21.15.31:0/1841731123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f561c102f90 0x7f561c19a4b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.417+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 >> 172.21.15.31:0/1841731123 conn(0x7f561c079e30 msgr2=0x7f561c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.418+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 shutdown_connections 2024-09-21T02:59:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:54.418+0000 7f5621e7b640 1 -- 172.21.15.31:0/1841731123 wait complete. 2024-09-21T02:59:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:54 smithi031 ceph-mon[31209]: pgmap v535: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:54 smithi165 ceph-mon[34818]: pgmap v535: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:54 smithi178 ceph-mon[34759]: pgmap v535: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T02:59:55.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T02:59:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 -- 172.21.15.31:0/279043187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 msgr2=0x7fa7041125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 --2- 172.21.15.31:0/279043187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041125d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009ca0 tx=0x7fa6fc02a950 comp rx=0 tx=0).stop 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 -- 172.21.15.31:0/279043187 shutdown_connections 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 --2- 172.21.15.31:0/279043187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704102f50 0x7fa70410fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 --2- 172.21.15.31:0/279043187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041125d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 --2- 172.21.15.31:0/279043187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa704102600 0x7fa704102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 -- 172.21.15.31:0/279043187 >> 172.21.15.31:0/279043187 conn(0x7fa704079750 msgr2=0x7fa704079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 -- 172.21.15.31:0/279043187 shutdown_connections 2024-09-21T02:59:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.125+0000 7fa70b869640 1 -- 172.21.15.31:0/279043187 wait complete. 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 Processor -- start 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 -- start start 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa704102600 0x7fa70419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704102f50 0x7fa70419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa704115ca0 con 0x7fa7041101b0 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa704115b20 con 0x7fa704102600 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa70b869640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa704115e20 con 0x7fa704102f50 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa709e06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa709e06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50270/0 (socket says 172.21.15.31:50270) 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.126+0000 7fa709e06640 1 -- 172.21.15.31:0/1520962315 learned_addr learned my addr 172.21.15.31:0/1520962315 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa708e04640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704102f50 0x7fa70419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709605640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa704102600 0x7fa70419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709e06640 1 -- 172.21.15.31:0/1520962315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704102f50 msgr2=0x7fa70419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709e06640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704102f50 0x7fa70419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709e06640 1 -- 172.21.15.31:0/1520962315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa704102600 msgr2=0x7fa70419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709e06640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa704102600 0x7fa70419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709e06640 1 -- 172.21.15.31:0/1520962315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7041a24a0 con 0x7fa7041101b0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa709e06640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041a1e30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc02ae70 tx=0x7fa6fc02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa6fa7fc640 1 -- 172.21.15.31:0/1520962315 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc034040 con 0x7fa7041101b0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa6fa7fc640 1 -- 172.21.15.31:0/1520962315 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6fc0308f0 con 0x7fa7041101b0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa6fa7fc640 1 -- 172.21.15.31:0/1520962315 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc004720 con 0x7fa7041101b0 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7041a2730 con 0x7fa7041101b0 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.127+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7041a2bc0 con 0x7fa7041101b0 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.130+0000 7fa711ddd740 1 Processor -- start 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.130+0000 7fa6fa7fc640 1 -- 172.21.15.31:0/1520962315 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa6fc037020 con 0x7fa7041101b0 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.130+0000 7fa711ddd740 1 -- start start 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.130+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x5655129192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.130+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 0x565512919800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512551390 0x565512919d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa711ddd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56551288fea0 con 0x5655125491b0 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa711ddd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56551287f2d0 con 0x565512572070 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa711ddd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5655128830e0 con 0x565512551390 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 0x565512919800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 0x565512919800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50274/0 (socket says 172.21.15.31:50274) 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 -- 172.21.15.31:0/1999008417 learned_addr learned my addr 172.21.15.31:0/1999008417 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa708e04640 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x5655129192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa708e04640 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512551390 0x565512919d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa6fa7fc640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa6f00b66b0 0x7fa6f00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 -- 172.21.15.31:0/1999008417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512551390 msgr2=0x565512919d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512551390 0x565512919d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 -- 172.21.15.31:0/1999008417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 msgr2=0x5655129192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x5655129192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 -- 172.21.15.31:0/1999008417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655125dd650 con 0x5655125491b0 2024-09-21T02:59:56.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa708e04640 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x5655129192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709e06640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa6f00b66b0 0x7fa6f00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa6fa7fc640 1 -- 172.21.15.31:0/1520962315 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa6fc108500 con 0x7fa7041101b0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.131+0000 7fa709605640 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 0x565512919800 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa70000ad00 tx=0x7fa70000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90a03555c9d2655 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.132+0000 7fa709e06640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa6f00b66b0 0x7fa6f00b8bc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc105d20 tx=0x7fa6fc092850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.132+0000 7fa6ef7fe640 1 -- 172.21.15.31:0/1999008417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d070 con 0x5655125491b0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.132+0000 7fa6ef7fe640 1 -- 172.21.15.31:0/1999008417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa700018af0 con 0x5655125491b0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.132+0000 7fa6ef7fe640 1 -- 172.21.15.31:0/1999008417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700010b30 con 0x5655125491b0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 -- 172.21.15.31:0/1999008417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 msgr2=0x565512919800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 0x565512919800 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa70000ad00 tx=0x7fa70000f790 comp rx=0 tx=0).stop 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 -- 172.21.15.31:0/1999008417 shutdown_connections 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512551390 0x565512919d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5655125491b0 0x565512919800 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1999008417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x5655129192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 -- 172.21.15.31:0/1999008417 >> 172.21.15.31:0/1999008417 conn(0x565512566460 msgr2=0x5655129171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 -- 172.21.15.31:0/1999008417 shutdown_connections 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 -- 172.21.15.31:0/1999008417 wait complete. 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 -- start start 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.133+0000 7fa711ddd740 1 Processor -- start 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x5655129249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512924ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5655126e0700 con 0x565512551390 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56551281a160 con 0x5655125491b0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x565512927aa0 con 0x565512572070 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa711ddd740 10 client.0 _ll_register_callbacks cb 0x5655128339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512924ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512924ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50280/0 (socket says 172.21.15.31:50280) 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 -- 172.21.15.31:0/3717406660 learned_addr learned my addr 172.21.15.31:0/3717406660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa708e04640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa708e04640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x5655129249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 -- 172.21.15.31:0/3717406660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 msgr2=0x565512930d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 -- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 msgr2=0x5655129249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x5655129249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 -- 172.21.15.31:0/3717406660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565512847c90 con 0x565512551390 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa708e04640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x5655129249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa709605640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512924ee0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa700010470 tx=0x7fa7000186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527fe04f48d8bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.134+0000 7fa6ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa6ecff9640 1 -- 172.21.15.31:0/3717406660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d040 con 0x565512551390 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa6ecff9640 1 -- 172.21.15.31:0/3717406660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa700010870 con 0x565512551390 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa700018af0 con 0x565512551390 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa6ecff9640 1 -- 172.21.15.31:0/3717406660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa700031050 con 0x565512551390 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 10 client.17157 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565512933770 con 0x565512551390 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa6ecff9640 1 -- 172.21.15.31:0/3717406660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa700018af0 con 0x565512551390 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa6ecff9640 1 -- 172.21.15.31:0/3717406660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa700020030 con 0x565512551390 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 10 client.17157 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 20 client.17157 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5655125309f0 0x565512936b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 4 client.17157 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.135+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565512891df0 con 0x5655125309f0 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.136+0000 7fa708e04640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5655125309f0 0x565512936b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.136+0000 7fa708e04640 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5655125309f0 0x565512936b80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b161c2ca4cf2b12d server_cookie=d7e3bd00f18f933a in_seq=0 out_seq=0 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.136+0000 7fa6ecff9640 10 client.17157 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:56.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa6ecff9640 1 -- 172.21.15.31:0/3717406660 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565512891df0 con 0x5655125309f0 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa6ecff9640 10 client.17157 handle_command_reply: tid=0 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 client.17157 shutdown 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 msgr2=0x565512924ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512924ee0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa700010470 tx=0x7fa7000186f0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 shutdown_connections 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5655125309f0 0x565512936b80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512924ee0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa6ecff9640 0 client.17157 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3717406660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x5655129249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.137+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 >> 172.21.15.31:0/3717406660 conn(0x565512566460 msgr2=0x56551291c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.138+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 shutdown_connections 2024-09-21T02:59:56.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.138+0000 7fa711ddd740 1 -- 172.21.15.31:0/3717406660 wait complete. 2024-09-21T02:59:56.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.138+0000 7fa711ddd740 20 client.17157 trim_cache size 0 max 16384 2024-09-21T02:59:56.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 Processor -- start 2024-09-21T02:59:56.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 -- start start 2024-09-21T02:59:56.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x565512923890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512917200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b1830 con 0x565512551390 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b3370 con 0x5655125491b0 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa711ddd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b1f00 con 0x565512572070 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512917200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512917200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50282/0 (socket says 172.21.15.31:50282) 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.151+0000 7fa709605640 1 -- 172.21.15.31:0/3849249278 learned_addr learned my addr 172.21.15.31:0/3849249278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa708e04640 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa709605640 1 -- 172.21.15.31:0/3849249278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 msgr2=0x565512930d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa709605640 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa709605640 1 -- 172.21.15.31:0/3849249278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 msgr2=0x565512923890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa709605640 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x565512923890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa709605640 1 -- 172.21.15.31:0/3849249278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565512925c60 con 0x565512551390 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa708e04640 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa709605640 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512917200 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa7000186c0 tx=0x7fa700005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76446c8733558b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa6eeffd640 1 -- 172.21.15.31:0/3849249278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d040 con 0x565512551390 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa6eeffd640 1 -- 172.21.15.31:0/3849249278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa700028070 con 0x565512551390 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.152+0000 7fa6eeffd640 1 -- 172.21.15.31:0/3849249278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000189c0 con 0x565512551390 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 -- 172.21.15.31:0/3849249278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 msgr2=0x565512917200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512917200 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa7000186c0 tx=0x7fa700005e40 comp rx=0 tx=0).stop 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 -- 172.21.15.31:0/3849249278 shutdown_connections 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x565512572070 0x565512930d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x565512917200 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 --2- 172.21.15.31:0/3849249278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655125491b0 0x565512923890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 -- 172.21.15.31:0/3849249278 >> 172.21.15.31:0/3849249278 conn(0x565512566460 msgr2=0x565512916af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 -- 172.21.15.31:0/3849249278 shutdown_connections 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 -- 172.21.15.31:0/3849249278 wait complete. 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 -- start start 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.153+0000 7fa711ddd740 1 Processor -- start 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655125491b0 0x56551293a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x5655129377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x565512937d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b2ca0 con 0x565512551390 2024-09-21T02:59:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041b25d0 con 0x565512572070 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041af810 con 0x5655125491b0 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa711ddd740 10 client.0 _ll_register_callbacks cb 0x7fa70001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x5655129377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x5655129377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50286/0 (socket says 172.21.15.31:50286) 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 -- 172.21.15.31:0/1478991457 learned_addr learned my addr 172.21.15.31:0/1478991457 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa708e04640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x565512937d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa708e04640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655125491b0 0x56551293a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 -- 172.21.15.31:0/1478991457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655125491b0 msgr2=0x56551293a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655125491b0 0x56551293a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 -- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 msgr2=0x565512937d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x565512937d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.154+0000 7fa709605640 1 -- 172.21.15.31:0/1478991457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565512938460 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa709605640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x5655129377e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa70000ad00 tx=0x7fa700018900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f2cbd5247f4acb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa6ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa6ecff9640 1 -- 172.21.15.31:0/1478991457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70001d040 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa6ecff9640 1 -- 172.21.15.31:0/1478991457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fa70002d450 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa700028070 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa6ecff9640 1 -- 172.21.15.31:0/1478991457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa70002d650 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa711ddd740 10 client.17163 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5655129a1120 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.155+0000 7fa6ecff9640 1 -- 172.21.15.31:0/1478991457 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa700028070 con 0x565512551390 2024-09-21T02:59:56.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa6ecff9640 1 -- 172.21.15.31:0/1478991457 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa700045030 con 0x565512551390 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa711ddd740 10 client.17163 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa711ddd740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa711ddd740 20 client.17163 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x565512505da0 0x5655129a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa711ddd740 4 client.17163 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56551288d8a0 con 0x565512505da0 2024-09-21T02:59:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa708e04640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x565512505da0 0x5655129a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa6ecff9640 10 client.17163 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:56.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.156+0000 7fa708e04640 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x565512505da0 0x5655129a3870 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48fc21039dc7a6f9 server_cookie=4575daf9e3204677 in_seq=0 out_seq=0 2024-09-21T02:59:56.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa6ecff9640 1 -- 172.21.15.31:0/1478991457 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56551288d8a0 con 0x565512505da0 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa6ecff9640 10 client.17163 handle_command_reply: tid=0 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 client.17163 shutdown 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 msgr2=0x5655129377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x5655129377e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa70000ad00 tx=0x7fa700018900 comp rx=0 tx=0).stop 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 shutdown_connections 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x565512505da0 0x5655129a3870 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565512572070 0x565512937d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565512551390 0x5655129377e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 --2- 172.21.15.31:0/1478991457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655125491b0 0x56551293a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 >> 172.21.15.31:0/1478991457 conn(0x565512566460 msgr2=0x56551291c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa6ecff9640 0 client.17163 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:56.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 shutdown_connections 2024-09-21T02:59:56.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 1 -- 172.21.15.31:0/1478991457 wait complete. 2024-09-21T02:59:56.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.157+0000 7fa711ddd740 20 client.17163 trim_cache size 0 max 16384 2024-09-21T02:59:56.166 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa6f00b66b0 msgr2=0x7fa6f00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa6f00b66b0 0x7fa6f00b8bc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc105d20 tx=0x7fa6fc092850 comp rx=0 tx=0).stop 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 msgr2=0x7fa7041a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041a1e30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc02ae70 tx=0x7fa6fc02f5d0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 shutdown_connections 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa6f00b66b0 0x7fa6f00b8bc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704102f50 0x7fa70419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7041101b0 0x7fa7041a1e30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 --2- 172.21.15.31:0/1520962315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa704102600 0x7fa70419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 >> 172.21.15.31:0/1520962315 conn(0x7fa704079750 msgr2=0x7fa704109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 shutdown_connections 2024-09-21T02:59:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:56.158+0000 7fa70b869640 1 -- 172.21.15.31:0/1520962315 wait complete. 2024-09-21T02:59:57.007 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:56 smithi031 ceph-mon[31209]: pgmap v536: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:56 smithi165 ceph-mon[34818]: pgmap v536: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:56 smithi178 ceph-mon[34759]: pgmap v536: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T02:59:58.009 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T02:59:58.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T02:59:58.281 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 -- 172.21.15.31:0/1793644968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4102a80 msgr2=0x7f61e4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 --2- 172.21.15.31:0/1793644968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4102a80 0x7f61e4102e90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 -- 172.21.15.31:0/1793644968 shutdown_connections 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 --2- 172.21.15.31:0/1793644968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e4108d10 0x7f61e410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 --2- 172.21.15.31:0/1793644968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61e41083c0 0x7f61e41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 --2- 172.21.15.31:0/1793644968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4102a80 0x7f61e4102e90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 -- 172.21.15.31:0/1793644968 >> 172.21.15.31:0/1793644968 conn(0x7f61e4079740 msgr2=0x7f61e4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 -- 172.21.15.31:0/1793644968 shutdown_connections 2024-09-21T02:59:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.794+0000 7f61ead77640 1 -- 172.21.15.31:0/1793644968 wait complete. 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 Processor -- start 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 -- start start 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e4102a80 0x7f61e419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 0x7f61e419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61e4108d10 0x7f61e419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4113aa0 con 0x7f61e41083c0 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4113920 con 0x7f61e4108d10 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61ead77640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4113c20 con 0x7f61e4102a80 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 0x7f61e419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 0x7f61e419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50300/0 (socket says 172.21.15.31:50300) 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61e3fff640 1 -- 172.21.15.31:0/3564614146 learned_addr learned my addr 172.21.15.31:0/3564614146 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.795+0000 7f61e8b13640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e4102a80 0x7f61e419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e9314640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61e4108d10 0x7f61e419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e3fff640 1 -- 172.21.15.31:0/3564614146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e4102a80 msgr2=0x7f61e419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e3fff640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e4102a80 0x7f61e419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e3fff640 1 -- 172.21.15.31:0/3564614146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61e4108d10 msgr2=0x7f61e419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e3fff640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61e4108d10 0x7f61e419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e3fff640 1 -- 172.21.15.31:0/3564614146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e41a03c0 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e3fff640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 0x7f61e419a9f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f61d400ea80 tx=0x7f61d400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e1ffb640 1 -- 172.21.15.31:0/3564614146 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d400ce40 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e1ffb640 1 -- 172.21.15.31:0/3564614146 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4011920 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61e1ffb640 1 -- 172.21.15.31:0/3564614146 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d401aa20 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.796+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e41a06b0 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.797+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a6e10 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.799+0000 7f61e1ffb640 1 -- 172.21.15.31:0/3564614146 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f61d401abc0 con 0x7f61e41083c0 2024-09-21T02:59:58.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.799+0000 7f61f12eb740 1 Processor -- start 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.799+0000 7f61f12eb740 1 -- start start 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896ef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896ef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89327670 0x559c896efeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61f12eb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559c89665e60 con 0x559c89348350 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61f12eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c89655290 con 0x559c8931f490 2024-09-21T02:59:58.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61f12eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c896590a0 con 0x559c89327670 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896ef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50308/0 (socket says 172.21.15.31:50308) 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 -- 172.21.15.31:0/3323777479 learned_addr learned my addr 172.21.15.31:0/3323777479 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e8b13640 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e3fff640 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89327670 0x559c896efeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 -- 172.21.15.31:0/3323777479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89327670 msgr2=0x559c896efeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89327670 0x559c896efeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 -- 172.21.15.31:0/3323777479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 msgr2=0x559c896ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 -- 172.21.15.31:0/3323777479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c893b3930 con 0x559c89348350 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e1ffb640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61d00b6680 0x7f61d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e8b13640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61d00b6680 0x7f61d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.800+0000 7f61e9314640 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896ef430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b700 tx=0x7f61dc00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a2c33accf4053c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.801+0000 7f61e1ffb640 1 -- 172.21.15.31:0/3564614146 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f61d40e7be0 con 0x7f61e41083c0 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.801+0000 7f61daffd640 1 -- 172.21.15.31:0/3323777479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc003dd0 con 0x559c89348350 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.801+0000 7f61daffd640 1 -- 172.21.15.31:0/3323777479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61dc015070 con 0x559c89348350 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.801+0000 7f61daffd640 1 -- 172.21.15.31:0/3323777479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0106a0 con 0x559c89348350 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 -- 172.21.15.31:0/3323777479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 msgr2=0x559c896ef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61e8b13640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61d00b6680 0x7f61d00b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f61cc031440 tx=0x7f61cc009610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896ef430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b700 tx=0x7f61dc00bbc0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 -- 172.21.15.31:0/3323777479 shutdown_connections 2024-09-21T02:59:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89327670 0x559c896efeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 --2- 172.21.15.31:0/3323777479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896ef430 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 -- 172.21.15.31:0/3323777479 >> 172.21.15.31:0/3323777479 conn(0x559c8933c740 msgr2=0x559c896ed320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 -- 172.21.15.31:0/3323777479 shutdown_connections 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 -- 172.21.15.31:0/3323777479 wait complete. 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 -- start start 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.802+0000 7f61f12eb740 1 Processor -- start 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 0x559c896fb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89706f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559c894b6890 con 0x559c89327670 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c895efd40 con 0x559c8931f490 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c896fdc10 con 0x559c89348350 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61f12eb740 10 client.0 _ll_register_callbacks cb 0x559c89653c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:58.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 0x559c896fb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 0x559c896fb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50318/0 (socket says 172.21.15.31:50318) 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 -- 172.21.15.31:0/1267682058 learned_addr learned my addr 172.21.15.31:0/1267682058 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e9314640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89706f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e8b13640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 -- 172.21.15.31:0/1267682058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 msgr2=0x559c89706f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89706f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 -- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 msgr2=0x559c896fb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896fb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e3fff640 1 -- 172.21.15.31:0/1267682058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c897074c0 con 0x559c89327670 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e8b13640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.803+0000 7f61e9314640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89706f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61e3fff640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 0x559c896fb550 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c700 tx=0x7f61d401af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439798481cc028be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:58.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61c3fff640 1 -- 172.21.15.31:0/1267682058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40ad1c0 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61c3fff640 1 -- 172.21.15.31:0/1267682058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f61d40b21b0 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61dc015070 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61c3fff640 1 -- 172.21.15.31:0/1267682058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61d40ad560 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 10 client.17175 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c89709df0 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61c3fff640 1 -- 172.21.15.31:0/1267682058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61dc015070 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61c3fff640 1 -- 172.21.15.31:0/1267682058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61d4022030 con 0x559c89327670 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 10 client.17175 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 20 client.17175 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c89306cd0 0x559c8970d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 4 client.17175 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:58.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.804+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c89667db0 con 0x559c89306cd0 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.805+0000 7f61e8b13640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c89306cd0 0x559c8970d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.805+0000 7f61e8b13640 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c89306cd0 0x559c8970d350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be9dd07bdb55a99 server_cookie=de4c29b0bb5d14ad in_seq=0 out_seq=0 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.805+0000 7f61c3fff640 10 client.17175 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61c3fff640 1 -- 172.21.15.31:0/1267682058 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c89667db0 con 0x559c89306cd0 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61c3fff640 10 client.17175 handle_command_reply: tid=0 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 client.17175 shutdown 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 msgr2=0x559c896fb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 0x559c896fb550 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c700 tx=0x7f61d401af60 comp rx=0 tx=0).stop 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 shutdown_connections 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c89306cd0 0x559c8970d350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89706f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89327670 0x559c896fb550 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 --2- 172.21.15.31:0/1267682058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c8931f490 0x559c896fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 >> 172.21.15.31:0/1267682058 conn(0x559c8933c740 msgr2=0x559c896f2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.806+0000 7f61c3fff640 0 client.17175 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.807+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 shutdown_connections 2024-09-21T02:59:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.807+0000 7f61f12eb740 1 -- 172.21.15.31:0/1267682058 wait complete. 2024-09-21T02:59:58.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.807+0000 7f61f12eb740 20 client.17175 trim_cache size 0 max 16384 2024-09-21T02:59:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 Processor -- start 2024-09-21T02:59:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 -- start start 2024-09-21T02:59:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c8931f490 0x559c896f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c896fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41af630 con 0x559c89348350 2024-09-21T02:59:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b1170 con 0x559c89327670 2024-09-21T02:59:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61f12eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41afd00 con 0x559c8931f490 2024-09-21T02:59:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50320/0 (socket says 172.21.15.31:50320) 2024-09-21T02:59:58.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 -- 172.21.15.31:0/2261479839 learned_addr learned my addr 172.21.15.31:0/2261479839 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e8b13640 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c8931f490 0x559c896f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e3fff640 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c896fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 -- 172.21.15.31:0/2261479839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c8931f490 msgr2=0x559c896f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c8931f490 0x559c896f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 -- 172.21.15.31:0/2261479839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 msgr2=0x559c896fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c896fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.825+0000 7f61e9314640 1 -- 172.21.15.31:0/2261479839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c896f1950 con 0x559c89348350 2024-09-21T02:59:58.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.826+0000 7f61e9314640 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896fccf0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f61dc019000 tx=0x7f61dc00b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c663a6ff57461eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.828+0000 7f61da7fc640 1 -- 172.21.15.31:0/2261479839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc028070 con 0x559c89348350 2024-09-21T02:59:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.828+0000 7f61da7fc640 1 -- 172.21.15.31:0/2261479839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61dc015040 con 0x559c89348350 2024-09-21T02:59:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.828+0000 7f61da7fc640 1 -- 172.21.15.31:0/2261479839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc004b40 con 0x559c89348350 2024-09-21T02:59:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 -- 172.21.15.31:0/2261479839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 msgr2=0x559c896fccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896fccf0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f61dc019000 tx=0x7f61dc00b620 comp rx=0 tx=0).stop 2024-09-21T02:59:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 -- 172.21.15.31:0/2261479839 shutdown_connections 2024-09-21T02:59:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c89348350 0x559c896fccf0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c896fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2261479839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c8931f490 0x559c896f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 -- 172.21.15.31:0/2261479839 >> 172.21.15.31:0/2261479839 conn(0x559c8933c740 msgr2=0x559c896f0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 -- 172.21.15.31:0/2261479839 shutdown_connections 2024-09-21T02:59:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 -- 172.21.15.31:0/2261479839 wait complete. 2024-09-21T02:59:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 -- start start 2024-09-21T02:59:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.829+0000 7f61f12eb740 1 Processor -- start 2024-09-21T02:59:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 0x559c89703230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c89703770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89771510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b0aa0 con 0x559c8931f490 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b03d0 con 0x559c89327670 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ad610 con 0x559c89348350 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61f12eb740 10 client.0 _ll_register_callbacks cb 0x7f61d4009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 0x559c89703230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 0x559c89703230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50332/0 (socket says 172.21.15.31:50332) 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 -- 172.21.15.31:0/2353490935 learned_addr learned my addr 172.21.15.31:0/2353490935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e3fff640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c89703770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e9314640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89771510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 -- 172.21.15.31:0/2353490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 msgr2=0x559c89771510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89771510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 -- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 msgr2=0x559c89703770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c89703770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.830+0000 7f61e8b13640 1 -- 172.21.15.31:0/2353490935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c89771ca0 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61e3fff640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c89703770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61e8b13640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 0x559c89703230 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc03f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db7f0a2375123cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61c3fff640 1 -- 172.21.15.31:0/2353490935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc056070 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61c3fff640 1 -- 172.21.15.31:0/2353490935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f61cc005240 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61dc015040 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61c3fff640 1 -- 172.21.15.31:0/2353490935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f61cc035040 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61f12eb740 10 client.17181 fetch_fsmap learned FSMap version 27 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c8977cc20 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.831+0000 7f61c3fff640 1 -- 172.21.15.31:0/2353490935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61dc015040 con 0x559c8931f490 2024-09-21T02:59:58.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61c3fff640 1 -- 172.21.15.31:0/2353490935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61cc0035f0 con 0x559c8931f490 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61f12eb740 10 client.17181 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61f12eb740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61f12eb740 20 client.17181 populate_metadata read hostname 'smithi031' 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c892dc080 0x559c8977f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61f12eb740 4 client.17181 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c89663860 con 0x559c892dc080 2024-09-21T02:59:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.832+0000 7f61e3fff640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c892dc080 0x559c8977f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.833+0000 7f61e3fff640 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c892dc080 0x559c8977f3a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddcab66957f42afd server_cookie=93a7bf9e058da0ac in_seq=0 out_seq=0 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.833+0000 7f61c3fff640 10 client.17181 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.834+0000 7f61c3fff640 1 -- 172.21.15.31:0/2353490935 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c89663860 con 0x559c892dc080 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.834+0000 7f61c3fff640 10 client.17181 handle_command_reply: tid=0 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.834+0000 7f61f12eb740 1 client.17181 shutdown 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.834+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 msgr2=0x559c89703230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.834+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 0x559c89703230 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc03f070 comp rx=0 tx=0).stop 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.834+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 shutdown_connections 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x559c892dc080 0x559c8977f3a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c89348350 0x559c89771510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c89327670 0x559c89703770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 --2- 172.21.15.31:0/2353490935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c8931f490 0x559c89703230 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61c3fff640 0 client.17181 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 >> 172.21.15.31:0/2353490935 conn(0x559c8933c740 msgr2=0x559c896f2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 shutdown_connections 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 1 -- 172.21.15.31:0/2353490935 wait complete. 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61f12eb740 20 client.17181 trim_cache size 0 max 16384 2024-09-21T02:59:58.838 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T02:59:58.839 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T02:59:58.839 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "73ff63e2-bd89-498f-99f1-9c76816e1b1e", 2024-09-21T02:59:58.839 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T02:59:58.839 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T02:59:58.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61d00b6680 msgr2=0x7f61d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61ead77640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61d00b6680 0x7f61d00b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f61cc031440 tx=0x7f61cc009610 comp rx=0 tx=0).stop 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 msgr2=0x7f61e419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.835+0000 7f61ead77640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 0x7f61e419a9f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f61d400ea80 tx=0x7f61d400ef40 comp rx=0 tx=0).stop 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 shutdown_connections 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61d00b6680 0x7f61d00b8b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61e4108d10 0x7f61e419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e41083c0 0x7f61e419a9f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 --2- 172.21.15.31:0/3564614146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e4102a80 0x7f61e419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 >> 172.21.15.31:0/3564614146 conn(0x7f61e4079740 msgr2=0x7f61e4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 shutdown_connections 2024-09-21T02:59:58.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T02:59:58.836+0000 7f61ead77640 1 -- 172.21.15.31:0/3564614146 wait complete. 2024-09-21T02:59:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:58 smithi031 ceph-mon[31209]: pgmap v537: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:58 smithi165 ceph-mon[34818]: pgmap v537: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:58 smithi178 ceph-mon[34759]: pgmap v537: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T02:59:59.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T02:59:59.913 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:00.034 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:59 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T03:00:00.034 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:59 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:00.034 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:59 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T03:00:00.034 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 02:59:59 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:59 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T03:00:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:59 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:59 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T03:00:00.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 02:59:59 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:59 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T03:00:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:59 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:59 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T03:00:00.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 02:59:59 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 -- 172.21.15.31:0/2215557940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e41094c0 msgr2=0x7f14e41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 --2- 172.21.15.31:0/2215557940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e41094c0 0x7f14e41098d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009900 tx=0x7f14d802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 -- 172.21.15.31:0/2215557940 shutdown_connections 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 --2- 172.21.15.31:0/2215557940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e4109e10 0x7f14e4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 --2- 172.21.15.31:0/2215557940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e41094c0 0x7f14e41098d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 --2- 172.21.15.31:0/2215557940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14e4103b80 0x7f14e4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 -- 172.21.15.31:0/2215557940 >> 172.21.15.31:0/2215557940 conn(0x7f14e40ff5d0 msgr2=0x7f14e4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 -- 172.21.15.31:0/2215557940 shutdown_connections 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.450+0000 7f14ea913640 1 -- 172.21.15.31:0/2215557940 wait complete. 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.451+0000 7f14ea913640 1 Processor -- start 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.451+0000 7f14ea913640 1 -- start start 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14ea913640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 0x7f14e419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14ea913640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e41094c0 0x7f14e419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14ea913640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14e4109e10 0x7f14e419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14ea913640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4113ad0 con 0x7f14e4103b80 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14ea913640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4113950 con 0x7f14e4109e10 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14ea913640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4113c50 con 0x7f14e41094c0 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 0x7f14e419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 0x7f14e419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47188/0 (socket says 172.21.15.31:47188) 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 -- 172.21.15.31:0/3926991964 learned_addr learned my addr 172.21.15.31:0/3926991964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e37fe640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e41094c0 0x7f14e419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14e4109e10 0x7f14e419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 -- 172.21.15.31:0/3926991964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e41094c0 msgr2=0x7f14e419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e41094c0 0x7f14e419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 -- 172.21.15.31:0/3926991964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14e4109e10 msgr2=0x7f14e419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14e4109e10 0x7f14e419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 -- 172.21.15.31:0/3926991964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e41a0370 con 0x7f14e4103b80 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.452+0000 7f14e3fff640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 0x7f14e419a4c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f14d400c920 tx=0x7f14d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.453+0000 7f14e17fa640 1 -- 172.21.15.31:0/3926991964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4007c70 con 0x7f14e4103b80 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.453+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e41a0660 con 0x7f14e4103b80 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.453+0000 7f14e17fa640 1 -- 172.21.15.31:0/3926991964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d4007e10 con 0x7f14e4103b80 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.453+0000 7f14e17fa640 1 -- 172.21.15.31:0/3926991964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4018820 con 0x7f14e4103b80 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.453+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e41a6e80 con 0x7f14e4103b80 2024-09-21T03:00:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 Processor -- start 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 -- start start 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560da7056e50 con 0x560da6d101b0 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560da7046280 con 0x560da6d39070 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14f0e87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560da704a090 con 0x560da6d18390 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e17fa640 1 -- 172.21.15.31:0/3926991964 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f14d400f450 con 0x7f14e4103b80 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:50632/0 (socket says 172.21.15.31:50632) 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e8eb0640 1 -- 172.21.15.31:0/860191844 learned_addr learned my addr 172.21.15.31:0/860191844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e3fff640 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70e0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47194/0 (socket says 172.21.15.31:47194) 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.456+0000 7f14e37fe640 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e8eb0640 1 -- 172.21.15.31:0/860191844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 msgr2=0x560da70e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e8eb0640 1 -- 172.21.15.31:0/860191844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 msgr2=0x560da70e0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70e0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e8eb0640 1 -- 172.21.15.31:0/860191844 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da6da4650 con 0x560da6d39070 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e3fff640 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70e0650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009ee0 tx=0x7f14dc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e466b6ee9558dda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e17fa640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14cc0b66b0 0x7f14cc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14ca7fc640 1 -- 172.21.15.31:0/860191844 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc016070 con 0x560da6d39070 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14ca7fc640 1 -- 172.21.15.31:0/860191844 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc004d20 con 0x560da6d39070 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14ca7fc640 1 -- 172.21.15.31:0/860191844 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc012740 con 0x560da6d39070 2024-09-21T03:00:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e3fff640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14cc0b66b0 0x7f14cc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.457+0000 7f14e17fa640 1 -- 172.21.15.31:0/3926991964 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f14d40e7880 con 0x7f14e4103b80 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 -- 172.21.15.31:0/860191844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 msgr2=0x560da70e0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70e0650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009ee0 tx=0x7f14dc010740 comp rx=0 tx=0).stop 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14e3fff640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14cc0b66b0 0x7f14cc0b8bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14d40e50a0 tx=0x7f14d40f1960 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 -- 172.21.15.31:0/860191844 shutdown_connections 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 --2- 172.21.15.31:0/860191844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70e0650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 -- 172.21.15.31:0/860191844 >> 172.21.15.31:0/860191844 conn(0x560da6d2d460 msgr2=0x560da70de540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 -- 172.21.15.31:0/860191844 shutdown_connections 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.458+0000 7f14f0e87740 1 -- 172.21.15.31:0/860191844 wait complete. 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 -- start start 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 Processor -- start 2024-09-21T03:00:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70ec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560da6ea7700 con 0x560da6d101b0 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560da6fe1110 con 0x560da6d39070 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560da70eee30 con 0x560da6d18390 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14f0e87740 10 client.0 _ll_register_callbacks cb 0x560da6ffa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70ebd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47200/0 (socket says 172.21.15.31:47200) 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14e8eb0640 1 -- 172.21.15.31:0/1314883991 learned_addr learned my addr 172.21.15.31:0/1314883991 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.459+0000 7f14e37fe640 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14e8eb0640 1 -- 172.21.15.31:0/1314883991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 msgr2=0x560da70ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70ec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14e8eb0640 1 -- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 msgr2=0x560da70f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14e8eb0640 1 -- 172.21.15.31:0/1314883991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da700ec40 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70ebd10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f14dc005a20 tx=0x7f14dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3fa6e490cf36f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14c37fe640 1 -- 172.21.15.31:0/1314883991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc016070 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc004d20 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14c37fe640 1 -- 172.21.15.31:0/1314883991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f14dc0052e0 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14c37fe640 1 -- 172.21.15.31:0/1314883991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14dc0054e0 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14f0e87740 10 client.17193 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560da70faae0 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.460+0000 7f14c37fe640 1 -- 172.21.15.31:0/1314883991 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc004d20 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14c37fe640 1 -- 172.21.15.31:0/1314883991 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f14dc00f040 con 0x560da6d101b0 2024-09-21T03:00:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14f0e87740 10 client.17193 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14f0e87740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14f0e87740 20 client.17193 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14f0e87740 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6cf79f0 0x560da70fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14f0e87740 4 client.17193 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da7058da0 con 0x560da6cf79f0 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14e37fe640 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6cf79f0 0x560da70fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14c37fe640 10 client.17193 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.461+0000 7f14e37fe640 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6cf79f0 0x560da70fde40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276e5ac05e57273e server_cookie=99f622d274fa31d1 in_seq=0 out_seq=0 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14c37fe640 1 -- 172.21.15.31:0/1314883991 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560da7058da0 con 0x560da6cf79f0 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14c37fe640 10 client.17193 handle_command_reply: tid=0 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 client.17193 shutdown 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 msgr2=0x560da70ebd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70ebd10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f14dc005a20 tx=0x7f14dc004a40 comp rx=0 tx=0).stop 2024-09-21T03:00:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 shutdown_connections 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6cf79f0 0x560da70fde40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6d18390 0x560da70ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.462+0000 7f14f0e87740 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d39070 0x560da70f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.463+0000 7f14f0e87740 1 --2- 172.21.15.31:0/1314883991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d101b0 0x560da70ebd10 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.463+0000 7f14c37fe640 0 client.17193 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.463+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 >> 172.21.15.31:0/1314883991 conn(0x560da6d2d460 msgr2=0x560da70e3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.463+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 shutdown_connections 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.463+0000 7f14f0e87740 1 -- 172.21.15.31:0/1314883991 wait complete. 2024-09-21T03:00:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.463+0000 7f14f0e87740 20 client.17193 trim_cache size 0 max 16384 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 Processor -- start 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 -- start start 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da70e2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41ad480 con 0x560da6d18390 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41aefc0 con 0x560da6d101b0 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.481+0000 7f14f0e87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41adb50 con 0x560da6cf79f0 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70ddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47202/0 (socket says 172.21.15.31:47202) 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 -- 172.21.15.31:0/862923801 learned_addr learned my addr 172.21.15.31:0/862923801 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e37fe640 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da70e2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 -- 172.21.15.31:0/862923801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 msgr2=0x560da70e2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da70e2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 -- 172.21.15.31:0/862923801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 msgr2=0x560da70ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 -- 172.21.15.31:0/862923801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da70e34b0 con 0x560da6d18390 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70ddb70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009700 tx=0x7f14dc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ced3720dfc98a84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14c9ffb640 1 -- 172.21.15.31:0/862923801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc016070 con 0x560da6d18390 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14c9ffb640 1 -- 172.21.15.31:0/862923801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc021070 con 0x560da6d18390 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.482+0000 7f14c9ffb640 1 -- 172.21.15.31:0/862923801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc034040 con 0x560da6d18390 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 -- 172.21.15.31:0/862923801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 msgr2=0x560da70ddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70ddb70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009700 tx=0x7f14dc010740 comp rx=0 tx=0).stop 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 -- 172.21.15.31:0/862923801 shutdown_connections 2024-09-21T03:00:00.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70ddb70 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 --2- 172.21.15.31:0/862923801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da70e2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 -- 172.21.15.31:0/862923801 >> 172.21.15.31:0/862923801 conn(0x560da6d2d460 msgr2=0x560da70ed230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 -- 172.21.15.31:0/862923801 shutdown_connections 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.483+0000 7f14f0e87740 1 -- 172.21.15.31:0/862923801 wait complete. 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 -- start start 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 Processor -- start 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da7101550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41ae8f0 con 0x560da6d18390 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41ae220 con 0x560da6d101b0 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f14e41ab460 con 0x560da6cf79f0 2024-09-21T03:00:00.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14f0e87740 10 client.0 _ll_register_callbacks cb 0x7f14d40f1830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14e8eb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70fef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47208/0 (socket says 172.21.15.31:47208) 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.484+0000 7f14e8eb0640 1 -- 172.21.15.31:0/2423036973 learned_addr learned my addr 172.21.15.31:0/2423036973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da7101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e37fe640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 -- 172.21.15.31:0/2423036973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 msgr2=0x560da7101550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da7101550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 -- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 msgr2=0x560da70fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 -- 172.21.15.31:0/2423036973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da70ff600 con 0x560da6d18390 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e37fe640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14e8eb0640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70fef60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009e00 tx=0x7f14dc026570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e8dac8d147e988 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:00.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14c37fe640 1 -- 172.21.15.31:0/2423036973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc016040 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14c37fe640 1 -- 172.21.15.31:0/2423036973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f14dc026990 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc021070 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14c37fe640 1 -- 172.21.15.31:0/2423036973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14dc004a70 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14f0e87740 10 client.17199 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.485+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560da7168400 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14c37fe640 1 -- 172.21.15.31:0/2423036973 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc021070 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14c37fe640 1 -- 172.21.15.31:0/2423036973 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f14dc03b540 con 0x560da6d18390 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14f0e87740 10 client.17199 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14f0e87740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14f0e87740 20 client.17199 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14f0e87740 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6d39070 0x560da716aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:00.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14f0e87740 4 client.17199 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560da7054850 con 0x560da6d39070 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.486+0000 7f14e37fe640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6d39070 0x560da716aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14e37fe640 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6d39070 0x560da716aa50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca7812f5f89020d server_cookie=3c5ad7536e72b9d8 in_seq=0 out_seq=0 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14c37fe640 10 client.17199 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14c37fe640 1 -- 172.21.15.31:0/2423036973 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x560da7054850 con 0x560da6d39070 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14c37fe640 10 client.17199 handle_command_reply: tid=0 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 client.17199 shutdown 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 msgr2=0x560da70fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70fef60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009e00 tx=0x7f14dc026570 comp rx=0 tx=0).stop 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 shutdown_connections 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560da6d39070 0x560da716aa50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560da6d18390 0x560da70fef60 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560da6d101b0 0x560da70fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14c37fe640 0 client.17199 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 --2- 172.21.15.31:0/2423036973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560da6cf79f0 0x560da7101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.487+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 >> 172.21.15.31:0/2423036973 conn(0x560da6d2d460 msgr2=0x560da70e1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 shutdown_connections 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14f0e87740 1 -- 172.21.15.31:0/2423036973 wait complete. 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14f0e87740 20 client.17199 trim_cache size 0 max 16384 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T03:00:00.500 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:00:00.501 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.502 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.503 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:00:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:00.507 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14cc0b66b0 msgr2=0x7f14cc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14cc0b66b0 0x7f14cc0b8bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14d40e50a0 tx=0x7f14d40f1960 comp rx=0 tx=0).stop 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 msgr2=0x7f14e419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 0x7f14e419a4c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f14d400c920 tx=0x7f14d400cde0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 shutdown_connections 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f14cc0b66b0 0x7f14cc0b8bc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14e41094c0 0x7f14e419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14e4109e10 0x7f14e419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 --2- 172.21.15.31:0/3926991964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14e4103b80 0x7f14e419a4c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 >> 172.21.15.31:0/3926991964 conn(0x7f14e40ff5d0 msgr2=0x7f14e41019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 shutdown_connections 2024-09-21T03:00:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:00.488+0000 7f14ea913640 1 -- 172.21.15.31:0/3926991964 wait complete. 2024-09-21T03:00:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:00 smithi031 ceph-mon[31209]: pgmap v538: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:00 smithi031 ceph-mon[31209]: overall HEALTH_OK 2024-09-21T03:00:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:00 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T03:00:01.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:00 smithi178 ceph-mon[34759]: pgmap v538: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:01.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:00 smithi178 ceph-mon[34759]: overall HEALTH_OK 2024-09-21T03:00:01.129 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:00 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T03:00:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:00 smithi165 ceph-mon[34818]: pgmap v538: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:00 smithi165 ceph-mon[34818]: overall HEALTH_OK 2024-09-21T03:00:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:00 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T03:00:01.221 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:00:01.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:00:01.483 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 -- 172.21.15.31:0/2391786055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 msgr2=0x7f77c810fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 --2- 172.21.15.31:0/2391786055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c810fb80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009900 tx=0x7f77b802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 -- 172.21.15.31:0/2391786055 shutdown_connections 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 --2- 172.21.15.31:0/2391786055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c81100c0 0x7f77c8112500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 --2- 172.21.15.31:0/2391786055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c810fb80 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 --2- 172.21.15.31:0/2391786055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c80757b0 0x7f77c8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 -- 172.21.15.31:0/2391786055 >> 172.21.15.31:0/2391786055 conn(0x7f77c807b850 msgr2=0x7f77c807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 -- 172.21.15.31:0/2391786055 shutdown_connections 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.012+0000 7f77d032d640 1 -- 172.21.15.31:0/2391786055 wait complete. 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 Processor -- start 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 -- start start 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80757b0 0x7f77c819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:02.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c81100c0 0x7f77c81a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8115c70 con 0x7f77c8076100 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8115af0 con 0x7f77c81100c0 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77d032d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8115df0 con 0x7f77c80757b0 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77cd8c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77cd8c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47230/0 (socket says 172.21.15.31:47230) 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77cd8c8640 1 -- 172.21.15.31:0/855136410 learned_addr learned my addr 172.21.15.31:0/855136410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.013+0000 7f77ce8ca640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c81100c0 0x7f77c81a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77ce0c9640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80757b0 0x7f77c819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77cd8c8640 1 -- 172.21.15.31:0/855136410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80757b0 msgr2=0x7f77c819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77cd8c8640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80757b0 0x7f77c819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77cd8c8640 1 -- 172.21.15.31:0/855136410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c81100c0 msgr2=0x7f77c81a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77cd8c8640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c81100c0 0x7f77c81a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77cd8c8640 1 -- 172.21.15.31:0/855136410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c81a25d0 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77ce8ca640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c81100c0 0x7f77c81a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77ce0c9640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80757b0 0x7f77c819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77cd8c8640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c819cc00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f77b8032fd0 tx=0x7f77b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b803f030 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c81a2860 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b8004490 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8035070 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.014+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c81a9020 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.015+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c8075bc0 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.017+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f77b8042030 con 0x7f77c8076100 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.017+0000 7f77bf7fe640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77b40b6680 0x7f77b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.018+0000 7f77ce0c9640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77b40b6680 0x7f77b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.018+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f77b81080a0 con 0x7f77c8076100 2024-09-21T03:00:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.018+0000 7f77ce0c9640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77b40b6680 0x7f77b40b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77c4005d00 tx=0x7f77c4005ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.021+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77b80ce610 con 0x7f77c8076100 2024-09-21T03:00:02.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.250+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c810b270 con 0x7f77c8076100 2024-09-21T03:00:02.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.251+0000 7f77bf7fe640 1 -- 172.21.15.31:0/855136410 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f77b80d29f0 con 0x7f77c8076100 2024-09-21T03:00:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:00:02.255 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77b40b6680 msgr2=0x7f77b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77b40b6680 0x7f77b40b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77c4005d00 tx=0x7f77c4005ed0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 msgr2=0x7f77c819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c819cc00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f77b8032fd0 tx=0x7f77b8004270 comp rx=0 tx=0).stop 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 shutdown_connections 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f77b40b6680 0x7f77b40b8b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77c81100c0 0x7f77c81a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77c8076100 0x7f77c819cc00 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 --2- 172.21.15.31:0/855136410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80757b0 0x7f77c819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.253+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 >> 172.21.15.31:0/855136410 conn(0x7f77c807b850 msgr2=0x7f77c8101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.254+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 shutdown_connections 2024-09-21T03:00:02.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:02.254+0000 7f77d032d640 1 -- 172.21.15.31:0/855136410 wait complete. 2024-09-21T03:00:02.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:00:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:02 smithi165 ceph-mon[34818]: pgmap v539: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:00:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:02 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/855136410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:02 smithi178 ceph-mon[34759]: pgmap v539: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:00:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:02 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/855136410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:03.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:02 smithi031 ceph-mon[31209]: pgmap v539: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:00:03.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:02 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/855136410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 -- 172.21.15.31:0/306259116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b981049b0 msgr2=0x7f3b98104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/306259116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b981049b0 0x7f3b98104dc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009ca0 tx=0x7f3b8802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 -- 172.21.15.31:0/306259116 shutdown_connections 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/306259116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 0x7f3b9810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/306259116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b9810de10 0x7f3b98110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/306259116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b981049b0 0x7f3b98104dc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 -- 172.21.15.31:0/306259116 >> 172.21.15.31:0/306259116 conn(0x7f3b98079730 msgr2=0x7f3b98079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 -- 172.21.15.31:0/306259116 shutdown_connections 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.766+0000 7f3b9f62e640 1 -- 172.21.15.31:0/306259116 wait complete. 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 Processor -- start 2024-09-21T03:00:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 -- start start 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b981049b0 0x7f3b981026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 0x7f3b98102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 0x7f3b98103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b98113a70 con 0x7f3b9810de10 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981138f0 con 0x7f3b981049b0 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9f62e640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b98113bf0 con 0x7f3b98105300 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9dbcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 0x7f3b98103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9dbcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 0x7f3b98103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47248/0 (socket says 172.21.15.31:47248) 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.767+0000 7f3b9dbcb640 1 -- 172.21.15.31:0/2619711352 learned_addr learned my addr 172.21.15.31:0/2619711352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 0x7f3b98102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b981049b0 0x7f3b981026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9dbcb640 1 -- 172.21.15.31:0/2619711352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 msgr2=0x7f3b98102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9dbcb640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 0x7f3b98102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9dbcb640 1 -- 172.21.15.31:0/2619711352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b981049b0 msgr2=0x7f3b981026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9dbcb640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b981049b0 0x7f3b981026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9dbcb640 1 -- 172.21.15.31:0/2619711352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b981a87c0 con 0x7f3b9810de10 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 0x7f3b98102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b981049b0 0x7f3b981026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9dbcb640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 0x7f3b98103170 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000d750 tx=0x7f3b9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b967fc640 1 -- 172.21.15.31:0/2619711352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90017070 con 0x7f3b9810de10 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b981a8ab0 con 0x7f3b9810de10 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b967fc640 1 -- 172.21.15.31:0/2619711352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b900054d0 con 0x7f3b9810de10 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b967fc640 1 -- 172.21.15.31:0/2619711352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90004c60 con 0x7f3b9810de10 2024-09-21T03:00:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.768+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b981a8fa0 con 0x7f3b9810de10 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.771+0000 7f3ba5ba2740 1 Processor -- start 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3b967fc640 1 -- 172.21.15.31:0/2619711352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3b90009bd0 con 0x7f3b9810de10 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 -- start start 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbd6070 0x55819ff7d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819ff7dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 0x55819ff7e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55819fef3ea0 con 0x55819fbb5390 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55819fee32d0 con 0x55819fbd6070 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55819fee70e0 con 0x55819fbad1b0 2024-09-21T03:00:03.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 0x55819ff7e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 0x55819ff7e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47260/0 (socket says 172.21.15.31:47260) 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/3359985014 learned_addr learned my addr 172.21.15.31:0/3359985014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819ff7dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.772+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbd6070 0x55819ff7d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b967fc640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b800b6680 0x7f3b800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/3359985014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 msgr2=0x55819ff7dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819ff7dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/3359985014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbd6070 msgr2=0x55819ff7d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbd6070 0x55819ff7d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/3359985014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819fc41650 con 0x55819fbb5390 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9dbcb640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b800b6680 0x7f3b800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 0x55819ff7e120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0149f0 tx=0x7f3b8c00c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82003bb432a0aeec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b967fc640 1 -- 172.21.15.31:0/2619711352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3b900e7100 con 0x7f3b9810de10 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b7f7fe640 1 -- 172.21.15.31:0/3359985014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0142d0 con 0x55819fbb5390 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b7f7fe640 1 -- 172.21.15.31:0/3359985014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c00f760 con 0x55819fbb5390 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.773+0000 7f3b7f7fe640 1 -- 172.21.15.31:0/3359985014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c003850 con 0x55819fbb5390 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/3359985014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 msgr2=0x55819ff7e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 0x55819ff7e120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0149f0 tx=0x7f3b8c00c510 comp rx=0 tx=0).stop 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3b9dbcb640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b800b6680 0x7f3b800b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b900526d0 tx=0x7f3b90052040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/3359985014 shutdown_connections 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819ff7dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbb5390 0x55819ff7e120 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/3359985014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbd6070 0x55819ff7d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/3359985014 >> 172.21.15.31:0/3359985014 conn(0x55819fbca460 msgr2=0x55819ff7b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:03.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/3359985014 shutdown_connections 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.774+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/3359985014 wait complete. 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 -- start start 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 Processor -- start 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbad1b0 0x55819ff88de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbb5390 0x55819ff89320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55819fd44700 con 0x55819fbd6070 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55819fe7e160 con 0x55819fbad1b0 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55819ff8be20 con 0x55819fbb5390 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.775+0000 7f3ba5ba2740 10 client.0 _ll_register_callbacks cb 0x55819fe979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff95110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47276/0 (socket says 172.21.15.31:47276) 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2209998300 learned_addr learned my addr 172.21.15.31:0/2209998300 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbad1b0 0x55819ff88de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbb5390 0x55819ff89320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2209998300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbb5390 msgr2=0x55819ff89320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbb5390 0x55819ff89320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbad1b0 msgr2=0x55819ff88de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbad1b0 0x55819ff88de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2209998300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55819feabc90 con 0x55819fbd6070 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff95110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ffc0 tx=0x7f3b8c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd83f6104f136873 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2209998300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00fb20 con 0x55819fbd6070 2024-09-21T03:00:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c00f760 con 0x55819fbd6070 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.776+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2209998300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3b8c004030 con 0x55819fbd6070 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2209998300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c004230 con 0x55819fbd6070 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 10 client.17217 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55819ff97bb0 con 0x55819fbd6070 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2209998300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c00f760 con 0x55819fbd6070 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2209998300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b8c012450 con 0x55819fbd6070 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 10 client.17217 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 10 client.17217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 20 client.17217 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb949f0 0x55819ff9af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 4 client.17217 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55819fef5df0 con 0x55819fb949f0 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.777+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb949f0 0x55819ff9af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.778+0000 7f3b7cff9640 10 client.17217 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.778+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb949f0 0x55819ff9af10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10fbd9594905f372 server_cookie=6f3002f9b56842ea in_seq=0 out_seq=0 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2209998300 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55819fef5df0 con 0x55819fb949f0 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3b7cff9640 10 client.17217 handle_command_reply: tid=0 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 client.17217 shutdown 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 msgr2=0x55819ff95110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff95110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ffc0 tx=0x7f3b8c00cc10 comp rx=0 tx=0).stop 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 shutdown_connections 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb949f0 0x55819ff9af10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbb5390 0x55819ff89320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff95110 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3b7cff9640 0 client.17217 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2209998300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbad1b0 0x55819ff88de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 >> 172.21.15.31:0/2209998300 conn(0x55819fbca460 msgr2=0x55819ff80880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 shutdown_connections 2024-09-21T03:00:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2209998300 wait complete. 2024-09-21T03:00:03.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.779+0000 7f3ba5ba2740 20 client.17217 trim_cache size 0 max 16384 2024-09-21T03:00:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.792+0000 7f3ba5ba2740 1 Processor -- start 2024-09-21T03:00:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.792+0000 7f3ba5ba2740 1 -- start start 2024-09-21T03:00:03.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 0x55819ff8acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819ff7ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbd6070 0x55819ff95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981b1a30 con 0x55819fbad1b0 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981b3570 con 0x55819fbb5390 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981b2100 con 0x55819fbd6070 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 0x55819ff8acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 0x55819ff8acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47288/0 (socket says 172.21.15.31:47288) 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/4269935741 learned_addr learned my addr 172.21.15.31:0/4269935741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819ff7ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/4269935741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbd6070 msgr2=0x55819ff95080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbd6070 0x55819ff95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/4269935741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 msgr2=0x55819ff7ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819ff7ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/4269935741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55819ff8a0a0 con 0x55819fbad1b0 2024-09-21T03:00:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819ff7ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 0x55819ff8acc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0059d0 tx=0x7f3b8c013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ad6e8525b05641 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b7effd640 1 -- 172.21.15.31:0/4269935741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c03c070 con 0x55819fbad1b0 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b7effd640 1 -- 172.21.15.31:0/4269935741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c00c990 con 0x55819fbad1b0 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.793+0000 7f3b7effd640 1 -- 172.21.15.31:0/4269935741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c012450 con 0x55819fbad1b0 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/4269935741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 msgr2=0x55819ff8acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 0x55819ff8acc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0059d0 tx=0x7f3b8c013f60 comp rx=0 tx=0).stop 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/4269935741 shutdown_connections 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbd6070 0x55819ff95080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819ff7ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/4269935741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbad1b0 0x55819ff8acc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/4269935741 >> 172.21.15.31:0/4269935741 conn(0x55819fbca460 msgr2=0x55819ff7e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/4269935741 shutdown_connections 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.794+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/4269935741 wait complete. 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 -- start start 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 Processor -- start 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819fffdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819fffe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff9dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981b2ea0 con 0x55819fbd6070 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981b27d0 con 0x55819fbb5390 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b981afa10 con 0x55819fbad1b0 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3ba5ba2740 10 client.0 _ll_register_callbacks cb 0x7f3b8c00cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff9dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3b9cbc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff9dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47298/0 (socket says 172.21.15.31:47298) 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2439064483 learned_addr learned my addr 172.21.15.31:0/2439064483 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.795+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819fffdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819fffe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2439064483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 msgr2=0x55819fffdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819fffdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 msgr2=0x55819fffe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819fffe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9cbc9640 1 -- 172.21.15.31:0/2439064483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55819ff9e450 con 0x55819fbd6070 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819fffe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b9cbc9640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff9dd10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55819ff83b80 tx=0x7f3b8c0144f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8916f1d2de633828 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2439064483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c03c020 con 0x55819fbd6070 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2439064483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f3b8c00fd20 con 0x55819fbd6070 2024-09-21T03:00:03.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c00c990 con 0x55819fbd6070 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2439064483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c037440 con 0x55819fbd6070 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3ba5ba2740 10 client.17223 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.796+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5581a0005660 con 0x55819fbd6070 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2439064483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c00c990 con 0x55819fbd6070 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2439064483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3b8c01f420 con 0x55819fbd6070 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3ba5ba2740 10 client.17223 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3ba5ba2740 10 client.17223 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3ba5ba2740 20 client.17223 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb69da0 0x5581a0007bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3ba5ba2740 4 client.17223 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55819fef18a0 con 0x55819fb69da0 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.797+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb69da0 0x5581a0007bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3b9d3ca640 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb69da0 0x5581a0007bd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=87aef74941202cd2 server_cookie=b7db669d6af10bcf in_seq=0 out_seq=0 2024-09-21T03:00:03.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3b7cff9640 10 client.17223 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3b7cff9640 1 -- 172.21.15.31:0/2439064483 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55819fef18a0 con 0x55819fb69da0 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3b7cff9640 10 client.17223 handle_command_reply: tid=0 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 client.17223 shutdown 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 msgr2=0x55819ff9dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff9dd10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55819ff83b80 tx=0x7f3b8c0144f0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 shutdown_connections 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55819fbd6070 0x55819ff9dd10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55819fbb5390 0x55819fffe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55819fb69da0 0x5581a0007bd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 --2- 172.21.15.31:0/2439064483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55819fbad1b0 0x55819fffdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 >> 172.21.15.31:0/2439064483 conn(0x55819fbca460 msgr2=0x55819ff8ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:03.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3b7cff9640 0 client.17223 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:03.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 shutdown_connections 2024-09-21T03:00:03.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 1 -- 172.21.15.31:0/2439064483 wait complete. 2024-09-21T03:00:03.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.798+0000 7f3ba5ba2740 20 client.17223 trim_cache size 0 max 16384 2024-09-21T03:00:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b800b6680 msgr2=0x7f3b800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b800b6680 0x7f3b800b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b900526d0 tx=0x7f3b90052040 comp rx=0 tx=0).stop 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 msgr2=0x7f3b98103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 0x7f3b98103170 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000d750 tx=0x7f3b9000dc10 comp rx=0 tx=0).stop 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 shutdown_connections 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3b800b6680 0x7f3b800b8b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b98105300 0x7f3b98102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9810de10 0x7f3b98103170 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 --2- 172.21.15.31:0/2619711352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b981049b0 0x7f3b981026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 >> 172.21.15.31:0/2619711352 conn(0x7f3b98079730 msgr2=0x7f3b980ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 shutdown_connections 2024-09-21T03:00:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:03.799+0000 7f3b9f62e640 1 -- 172.21.15.31:0/2619711352 wait complete. 2024-09-21T03:00:04.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:00:04.894 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:05.015 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:04 smithi031 ceph-mon[31209]: pgmap v540: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:00:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:04 smithi165 ceph-mon[34818]: pgmap v540: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:00:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:04 smithi178 ceph-mon[34759]: pgmap v540: 97 pgs: 97 active+clean; 259 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 -- 172.21.15.31:0/705582449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c1083a0 msgr2=0x7f443c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 --2- 172.21.15.31:0/705582449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c1083a0 0x7f443c1087b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4430009900 tx=0x7f443002a410 comp rx=0 tx=0).stop 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 -- 172.21.15.31:0/705582449 shutdown_connections 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 --2- 172.21.15.31:0/705582449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c108cf0 0x7f443c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 --2- 172.21.15.31:0/705582449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c1083a0 0x7f443c1087b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 --2- 172.21.15.31:0/705582449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c102a60 0x7f443c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 -- 172.21.15.31:0/705582449 >> 172.21.15.31:0/705582449 conn(0x7f443c079730 msgr2=0x7f443c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 -- 172.21.15.31:0/705582449 shutdown_connections 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.400+0000 7f4442635640 1 -- 172.21.15.31:0/705582449 wait complete. 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.401+0000 7f4442635640 1 Processor -- start 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.401+0000 7f4442635640 1 -- start start 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.401+0000 7f4442635640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c102a60 0x7f443c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4442635640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c1083a0 0x7f443c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4442635640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 0x7f443c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4442635640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f443c112960 con 0x7f443c108cf0 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4442635640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1127e0 con 0x7f443c102a60 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4442635640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f443c112ae0 con 0x7f443c1083a0 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 0x7f443c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 0x7f443c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47320/0 (socket says 172.21.15.31:47320) 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 -- 172.21.15.31:0/2530108697 learned_addr learned my addr 172.21.15.31:0/2530108697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f443bfff640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c102a60 0x7f443c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c1083a0 0x7f443c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 -- 172.21.15.31:0/2530108697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c1083a0 msgr2=0x7f443c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c1083a0 0x7f443c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 -- 172.21.15.31:0/2530108697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c102a60 msgr2=0x7f443c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c102a60 0x7f443c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 -- 172.21.15.31:0/2530108697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c19f2e0 con 0x7f443c108cf0 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 0x7f443c19ebb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f443400ebb0 tx=0x7f4434007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.402+0000 7f44397fa640 1 -- 172.21.15.31:0/2530108697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434007ca0 con 0x7f443c108cf0 2024-09-21T03:00:05.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.403+0000 7f44397fa640 1 -- 172.21.15.31:0/2530108697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4434007e40 con 0x7f443c108cf0 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.403+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c19f570 con 0x7f443c108cf0 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.403+0000 7f44397fa640 1 -- 172.21.15.31:0/2530108697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44340057d0 con 0x7f443c108cf0 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.403+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c06ae60 con 0x7f443c108cf0 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.405+0000 7f4448ba9740 1 Processor -- start 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f44397fa640 1 -- 172.21.15.31:0/2530108697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4434004b20 con 0x7f443c108cf0 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 -- start start 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348ea070 0x55ed34c912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 0x55ed34c91800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34c91d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed34c07ea0 con 0x55ed348c11b0 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed34bf72d0 con 0x55ed348c9390 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f4448ba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed34bfb0e0 con 0x55ed348ea070 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f443bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 0x55ed34c91800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.406+0000 7f443bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 0x55ed34c91800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47336/0 (socket says 172.21.15.31:47336) 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 -- 172.21.15.31:0/2936067526 learned_addr learned my addr 172.21.15.31:0/2936067526 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348ea070 0x55ed34c912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34c91d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 -- 172.21.15.31:0/2936067526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348ea070 msgr2=0x55ed34c912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348ea070 0x55ed34c912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 -- 172.21.15.31:0/2936067526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 msgr2=0x55ed34c91d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34c91d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 -- 172.21.15.31:0/2936067526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed34955650 con 0x55ed348c11b0 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34c91d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f44397fa640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44240b6680 0x7f44240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44240b6680 0x7f44240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f443bfff640 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 0x55ed34c91800 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f442c00bd40 tx=0x7f442c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259f08b3416ed3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f44397fa640 1 -- 172.21.15.31:0/2530108697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f44340e8c10 con 0x7f443c108cf0 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f44227fc640 1 -- 172.21.15.31:0/2936067526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c004230 con 0x55ed348c11b0 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f44227fc640 1 -- 172.21.15.31:0/2936067526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c00be30 con 0x55ed348c11b0 2024-09-21T03:00:05.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.407+0000 7f44227fc640 1 -- 172.21.15.31:0/2936067526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c01a850 con 0x55ed348c11b0 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 -- 172.21.15.31:0/2936067526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 msgr2=0x55ed34c91800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44240b6680 0x7f44240b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4430000c00 tx=0x7f4430031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 0x55ed34c91800 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f442c00bd40 tx=0x7f442c0107a0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 -- 172.21.15.31:0/2936067526 shutdown_connections 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34c91d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c11b0 0x55ed34c91800 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2936067526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348ea070 0x55ed34c912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 -- 172.21.15.31:0/2936067526 >> 172.21.15.31:0/2936067526 conn(0x55ed348de460 msgr2=0x55ed34c8f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 -- 172.21.15.31:0/2936067526 shutdown_connections 2024-09-21T03:00:05.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.408+0000 7f4448ba9740 1 -- 172.21.15.31:0/2936067526 wait complete. 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.409+0000 7f4448ba9740 1 -- start start 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.409+0000 7f4448ba9740 1 Processor -- start 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c9390 0x55ed34ca1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348ea070 0x55ed34ca17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed34a58700 con 0x55ed348c9390 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed34b92160 con 0x55ed348ea070 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed34ca3dd0 con 0x55ed348c11b0 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4448ba9740 10 client.0 _ll_register_callbacks cb 0x55ed34bab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f443b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c9390 0x55ed34ca1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f443bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348ea070 0x55ed34ca17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f443bfff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:43730/0 (socket says 172.21.15.31:43730) 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f443bfff640 1 -- 172.21.15.31:0/2987428052 learned_addr learned my addr 172.21.15.31:0/2987428052 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4440bd2640 1 -- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 msgr2=0x55ed34ca0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4440bd2640 1 -- 172.21.15.31:0/2987428052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c9390 msgr2=0x55ed34ca1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c9390 0x55ed34ca1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.410+0000 7f4440bd2640 1 -- 172.21.15.31:0/2987428052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed34bbfc90 con 0x55ed348ea070 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348ea070 0x55ed34ca17c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44340e6380 tx=0x7f44340ae390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2e4048ae4403c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f441b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f441b7fe640 1 -- 172.21.15.31:0/2987428052 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44340ae640 con 0x55ed348ea070 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c00be30 con 0x55ed348ea070 2024-09-21T03:00:05.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f441b7fe640 1 -- 172.21.15.31:0/2987428052 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f44340aa480 con 0x55ed348ea070 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f441b7fe640 1 -- 172.21.15.31:0/2987428052 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f443400e0d0 con 0x55ed348ea070 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f4448ba9740 10 client.26609 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed34cafb00 con 0x55ed348ea070 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.411+0000 7f441b7fe640 1 -- 172.21.15.31:0/2987428052 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c00be30 con 0x55ed348ea070 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f441b7fe640 1 -- 172.21.15.31:0/2987428052 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f44340b3080 con 0x55ed348ea070 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f4448ba9740 10 client.26609 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f4448ba9740 10 client.26609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f4448ba9740 20 client.26609 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed348a89f0 0x55ed34cb2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f4448ba9740 4 client.26609 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:05.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed34c09df0 con 0x55ed348a89f0 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f443bfff640 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed348a89f0 0x55ed34cb2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f441b7fe640 10 client.26609 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.412+0000 7f443bfff640 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed348a89f0 0x55ed34cb2f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52e384d3e2be3ce7 server_cookie=ce7f3670c9a4f3f8 in_seq=0 out_seq=0 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f441b7fe640 1 -- 172.21.15.31:0/2987428052 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed34c09df0 con 0x55ed348a89f0 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f441b7fe640 10 client.26609 handle_command_reply: tid=0 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 client.26609 shutdown 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348ea070 msgr2=0x55ed34ca17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348ea070 0x55ed34ca17c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44340e6380 tx=0x7f44340ae390 comp rx=0 tx=0).stop 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 shutdown_connections 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed348a89f0 0x55ed34cb2f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348ea070 0x55ed34ca17c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f441b7fe640 0 client.26609 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348c9390 0x55ed34ca1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2987428052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.413+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 >> 172.21.15.31:0/2987428052 conn(0x55ed348de460 msgr2=0x55ed34c94410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.414+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 shutdown_connections 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.414+0000 7f4448ba9740 1 -- 172.21.15.31:0/2987428052 wait complete. 2024-09-21T03:00:05.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.414+0000 7f4448ba9740 20 client.26609 trim_cache size 0 max 16384 2024-09-21T03:00:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 Processor -- start 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 -- start start 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c11b0 0x55ed34ca2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c9390 0x55ed34c92a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34c8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1ad500 con 0x55ed348ea070 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1af040 con 0x55ed348c11b0 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4448ba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1adbd0 con 0x55ed348c9390 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34c8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34c8e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47350/0 (socket says 172.21.15.31:47350) 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.427+0000 7f4440bd2640 1 -- 172.21.15.31:0/927413687 learned_addr learned my addr 172.21.15.31:0/927413687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f443bfff640 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c11b0 0x55ed34ca2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4440bd2640 1 -- 172.21.15.31:0/927413687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c9390 msgr2=0x55ed34c92a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4440bd2640 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c9390 0x55ed34c92a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4440bd2640 1 -- 172.21.15.31:0/927413687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c11b0 msgr2=0x55ed34ca2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4440bd2640 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c11b0 0x55ed34ca2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4440bd2640 1 -- 172.21.15.31:0/927413687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed34c9f200 con 0x55ed348ea070 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4440bd2640 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34c8e760 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4434009270 tx=0x7f4434005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9880b7adf5774d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4421ffb640 1 -- 172.21.15.31:0/927413687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44340ae640 con 0x55ed348ea070 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4421ffb640 1 -- 172.21.15.31:0/927413687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44340b3560 con 0x55ed348ea070 2024-09-21T03:00:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.428+0000 7f4421ffb640 1 -- 172.21.15.31:0/927413687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44340aa480 con 0x55ed348ea070 2024-09-21T03:00:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 -- 172.21.15.31:0/927413687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 msgr2=0x55ed34c8e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34c8e760 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4434009270 tx=0x7f4434005270 comp rx=0 tx=0).stop 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 -- 172.21.15.31:0/927413687 shutdown_connections 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c9390 0x55ed34c92a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34c8e760 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 --2- 172.21.15.31:0/927413687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c11b0 0x55ed34ca2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 -- 172.21.15.31:0/927413687 >> 172.21.15.31:0/927413687 conn(0x55ed348de460 msgr2=0x55ed34c922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 -- 172.21.15.31:0/927413687 shutdown_connections 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 -- 172.21.15.31:0/927413687 wait complete. 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.429+0000 7f4448ba9740 1 -- start start 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 Processor -- start 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34ca5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34d1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1ae970 con 0x55ed348ea070 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1ae2a0 con 0x55ed348c9390 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f443c1ab4e0 con 0x55ed348c11b0 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4448ba9740 10 client.0 _ll_register_callbacks cb 0x7f4434004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34d1c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34d1c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47356/0 (socket says 172.21.15.31:47356) 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 -- 172.21.15.31:0/2566767355 learned_addr learned my addr 172.21.15.31:0/2566767355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f443bfff640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f443b7fe640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34ca5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 -- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 msgr2=0x55ed34ca5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 -- 172.21.15.31:0/2566767355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 msgr2=0x55ed34ca5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34ca5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.430+0000 7f4440bd2640 1 -- 172.21.15.31:0/2566767355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed34d1cef0 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f4440bd2640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34d1c8e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f44340b3460 tx=0x7f44340b3320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beeb3b618f66280 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f441b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f441b7fe640 1 -- 172.21.15.31:0/2566767355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44340ae640 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f441b7fe640 1 -- 172.21.15.31:0/2566767355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f44340aa680 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44340b3560 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f441b7fe640 1 -- 172.21.15.31:0/2566767355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44340d6020 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f4448ba9740 10 client.17241 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed34d1e170 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f441b7fe640 1 -- 172.21.15.31:0/2566767355 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44340b3560 con 0x55ed348ea070 2024-09-21T03:00:05.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f441b7fe640 1 -- 172.21.15.31:0/2566767355 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f44340d0be0 con 0x55ed348ea070 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f4448ba9740 10 client.17241 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.431+0000 7f4448ba9740 10 client.17241 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f4448ba9740 20 client.17241 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed3487dda0 0x55ed34d20650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f4448ba9740 4 client.17241 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed34c058a0 con 0x55ed3487dda0 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f443bfff640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed3487dda0 0x55ed34d20650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f443bfff640 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed3487dda0 0x55ed34d20650 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d808388f27553611 server_cookie=29905585df7cf8ab in_seq=0 out_seq=0 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f441b7fe640 10 client.17241 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f441b7fe640 1 -- 172.21.15.31:0/2566767355 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed34c058a0 con 0x55ed3487dda0 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f441b7fe640 10 client.17241 handle_command_reply: tid=0 2024-09-21T03:00:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.432+0000 7f4448ba9740 1 client.17241 shutdown 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 msgr2=0x55ed34d1c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34d1c8e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f44340b3460 tx=0x7f44340b3320 comp rx=0 tx=0).stop 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 shutdown_connections 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55ed3487dda0 0x55ed34d20650 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed348ea070 0x55ed34d1c8e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed348c9390 0x55ed34ca5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 --2- 172.21.15.31:0/2566767355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed348c11b0 0x55ed34ca5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 >> 172.21.15.31:0/2566767355 conn(0x55ed348de460 msgr2=0x55ed34c92cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f441b7fe640 0 client.17241 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 shutdown_connections 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 1 -- 172.21.15.31:0/2566767355 wait complete. 2024-09-21T03:00:05.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4448ba9740 20 client.17241 trim_cache size 0 max 16384 2024-09-21T03:00:05.441 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:05.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44240b6680 msgr2=0x7f44240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4442635640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44240b6680 0x7f44240b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4430000c00 tx=0x7f4430031400 comp rx=0 tx=0).stop 2024-09-21T03:00:05.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 msgr2=0x7f443c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.433+0000 7f4442635640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 0x7f443c19ebb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f443400ebb0 tx=0x7f4434007590 comp rx=0 tx=0).stop 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 shutdown_connections 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f44240b6680 0x7f44240b8b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f443c1083a0 0x7f443c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f443c108cf0 0x7f443c19ebb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 --2- 172.21.15.31:0/2530108697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c102a60 0x7f443c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 >> 172.21.15.31:0/2530108697 conn(0x7f443c079730 msgr2=0x7f443c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 shutdown_connections 2024-09-21T03:00:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:05.434+0000 7f4442635640 1 -- 172.21.15.31:0/2530108697 wait complete. 2024-09-21T03:00:06.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:00:06.362 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:06.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.867+0000 7f704fdca640 1 -- 172.21.15.31:0/202282849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70481083c0 msgr2=0x7f70481087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.867+0000 7f704fdca640 1 --2- 172.21.15.31:0/202282849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70481083c0 0x7f70481087d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7038009900 tx=0x7f703802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:06.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 -- 172.21.15.31:0/202282849 shutdown_connections 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 --2- 172.21.15.31:0/202282849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 0x7f704810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 --2- 172.21.15.31:0/202282849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70481083c0 0x7f70481087d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 --2- 172.21.15.31:0/202282849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7048102a80 0x7f7048102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 -- 172.21.15.31:0/202282849 >> 172.21.15.31:0/202282849 conn(0x7f7048079740 msgr2=0x7f7048079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 -- 172.21.15.31:0/202282849 shutdown_connections 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 -- 172.21.15.31:0/202282849 wait complete. 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 Processor -- start 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.868+0000 7f704fdca640 1 -- start start 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704fdca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 0x7f704810fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704fdca640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70481083c0 0x7f7048110310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704fdca640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 0x7f7048110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704fdca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7048113af0 con 0x7f7048102a80 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704fdca640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7048113970 con 0x7f7048108d10 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704fdca640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7048113c70 con 0x7f70481083c0 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 0x7f704810fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 0x7f704810fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47380/0 (socket says 172.21.15.31:47380) 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 -- 172.21.15.31:0/3778588369 learned_addr learned my addr 172.21.15.31:0/3778588369 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704d365640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70481083c0 0x7f7048110310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704e367640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 0x7f7048110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 -- 172.21.15.31:0/3778588369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70481083c0 msgr2=0x7f7048110310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70481083c0 0x7f7048110310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 -- 172.21.15.31:0/3778588369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 msgr2=0x7f7048110850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 0x7f7048110850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 -- 172.21.15.31:0/3778588369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70481a87c0 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704e367640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 0x7f7048110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.869+0000 7f704db66640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 0x7f704810fdd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f704400b4a0 tx=0x7f704400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.870+0000 7f703effd640 1 -- 172.21.15.31:0/3778588369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044014070 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.870+0000 7f703effd640 1 -- 172.21.15.31:0/3778588369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7044010ac0 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.870+0000 7f703effd640 1 -- 172.21.15.31:0/3778588369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044017c30 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.870+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70481a8ab0 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.870+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70481a8ff0 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.872+0000 7f705633e740 1 Processor -- start 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.872+0000 7f703effd640 1 -- 172.21.15.31:0/3778588369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7044017070 con 0x7f7048102a80 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.872+0000 7f705633e740 1 -- start start 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f705633e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cf22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f705633e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cf2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f705633e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cf2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f705633e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564180c68ea0 con 0x5641809221b0 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f705633e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564180c582d0 con 0x56418092a390 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f705633e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564180c5c0e0 con 0x56418094b070 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cf2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cf2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47396/0 (socket says 172.21.15.31:47396) 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 -- 172.21.15.31:0/2509022872 learned_addr learned my addr 172.21.15.31:0/2509022872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704e367640 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cf22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704d365640 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cf2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 -- 172.21.15.31:0/2509022872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 msgr2=0x564180cf22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cf22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 -- 172.21.15.31:0/2509022872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 msgr2=0x564180cf2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cf2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704db66640 1 -- 172.21.15.31:0/2509022872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641809b6650 con 0x5641809221b0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.873+0000 7f704e367640 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cf22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f703effd640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70340b6680 0x7f70340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f704db66640 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cf2800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f70440e43f0 tx=0x7f70440eaf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f22808ae7e4f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f704e367640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70340b6680 0x7f70340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f7033fff640 1 -- 172.21.15.31:0/2509022872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440a91d0 con 0x5641809221b0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f703effd640 1 -- 172.21.15.31:0/3778588369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f70440e6d10 con 0x7f7048102a80 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f7033fff640 1 -- 172.21.15.31:0/2509022872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7044017360 con 0x5641809221b0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.874+0000 7f7033fff640 1 -- 172.21.15.31:0/2509022872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440ac2f0 con 0x5641809221b0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 -- 172.21.15.31:0/2509022872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 msgr2=0x564180cf2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f704e367640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70340b6680 0x7f70340b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f70400095f0 tx=0x7f7040009580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:06.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cf2800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f70440e43f0 tx=0x7f70440eaf60 comp rx=0 tx=0).stop 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 -- 172.21.15.31:0/2509022872 shutdown_connections 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cf2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cf2800 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 --2- 172.21.15.31:0/2509022872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cf22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 -- 172.21.15.31:0/2509022872 >> 172.21.15.31:0/2509022872 conn(0x56418093f460 msgr2=0x564180cf01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 -- 172.21.15.31:0/2509022872 shutdown_connections 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 -- 172.21.15.31:0/2509022872 wait complete. 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.875+0000 7f705633e740 1 -- start start 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 Processor -- start 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cfd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564180ab9700 con 0x5641809221b0 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564180bf3160 con 0x56418092a390 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564180d00aa0 con 0x56418094b070 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f705633e740 10 client.0 _ll_register_callbacks cb 0x564180c0c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cfd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cfd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47400/0 (socket says 172.21.15.31:47400) 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 -- 172.21.15.31:0/2846385749 learned_addr learned my addr 172.21.15.31:0/2846385749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 -- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 msgr2=0x564180cfe3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704e367640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704d365640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfe3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 -- 172.21.15.31:0/2846385749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 msgr2=0x564180cfde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.876+0000 7f704db66640 1 -- 172.21.15.31:0/2846385749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564180c20c90 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f704e367640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f704db66640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cfd920 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7044007fd0 tx=0x7f7044017550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf666574b9ad7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f70317fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f70317fa640 1 -- 172.21.15.31:0/2846385749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704401e030 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f70317fa640 1 -- 172.21.15.31:0/2846385749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7044017dd0 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564180c6adf0 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f70317fa640 1 -- 172.21.15.31:0/2846385749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f70440ac4d0 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f705633e740 10 client.17253 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564180d0c720 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f70317fa640 1 -- 172.21.15.31:0/2846385749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564180c6adf0 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.877+0000 7f70317fa640 1 -- 172.21.15.31:0/2846385749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70440e4700 con 0x5641809221b0 2024-09-21T03:00:06.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f705633e740 10 client.17253 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f705633e740 10 client.17253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f705633e740 20 client.17253 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f705633e740 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641809099f0 0x564180d0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f705633e740 4 client.17253 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564180c668a0 con 0x5641809099f0 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f704d365640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641809099f0 0x564180d0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f704d365640 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641809099f0 0x564180d0fa80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10b39c9f3d10f5be server_cookie=b83409fae0291db1 in_seq=0 out_seq=0 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.878+0000 7f70317fa640 10 client.17253 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f70317fa640 1 -- 172.21.15.31:0/2846385749 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564180c668a0 con 0x5641809099f0 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f70317fa640 10 client.17253 handle_command_reply: tid=0 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 client.17253 shutdown 2024-09-21T03:00:06.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 msgr2=0x564180cfd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cfd920 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7044007fd0 tx=0x7f7044017550 comp rx=0 tx=0).stop 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 shutdown_connections 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641809099f0 0x564180d0fa80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f70317fa640 0 client.17253 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 --2- 172.21.15.31:0/2846385749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cfd920 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.879+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 >> 172.21.15.31:0/2846385749 conn(0x56418093f460 msgr2=0x564180cf5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.880+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 shutdown_connections 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.880+0000 7f705633e740 1 -- 172.21.15.31:0/2846385749 wait complete. 2024-09-21T03:00:06.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.880+0000 7f705633e740 20 client.17253 trim_cache size 0 max 16384 2024-09-21T03:00:06.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 Processor -- start 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 -- start start 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b1a70 con 0x5641809221b0 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b35b0 con 0x56418092a390 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f705633e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b2140 con 0x56418094b070 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f704db66640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47406/0 (socket says 172.21.15.31:47406) 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.893+0000 7f704db66640 1 -- 172.21.15.31:0/4026125587 learned_addr learned my addr 172.21.15.31:0/4026125587 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704d365640 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704e367640 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704db66640 1 -- 172.21.15.31:0/4026125587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 msgr2=0x564180cfb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704db66640 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704db66640 1 -- 172.21.15.31:0/4026125587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 msgr2=0x564180cfab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704db66640 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704db66640 1 -- 172.21.15.31:0/4026125587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564180ceb5d0 con 0x5641809221b0 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704d365640 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f704db66640 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cff910 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f70440a9150 tx=0x7f7044017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7290f22f4b7269 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f70337fe640 1 -- 172.21.15.31:0/4026125587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704401e030 con 0x5641809221b0 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f70337fe640 1 -- 172.21.15.31:0/4026125587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70440ea860 con 0x5641809221b0 2024-09-21T03:00:06.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.894+0000 7f70337fe640 1 -- 172.21.15.31:0/4026125587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440ac950 con 0x5641809221b0 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 -- 172.21.15.31:0/4026125587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 msgr2=0x564180cff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cff910 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f70440a9150 tx=0x7f7044017650 comp rx=0 tx=0).stop 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 -- 172.21.15.31:0/4026125587 shutdown_connections 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180cfb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56418092a390 0x564180cfab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 --2- 172.21.15.31:0/4026125587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5641809221b0 0x564180cff910 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 -- 172.21.15.31:0/4026125587 >> 172.21.15.31:0/4026125587 conn(0x56418093f460 msgr2=0x564180cf3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 -- 172.21.15.31:0/4026125587 shutdown_connections 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 -- 172.21.15.31:0/4026125587 wait complete. 2024-09-21T03:00:06.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 -- start start 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.895+0000 7f705633e740 1 Processor -- start 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641809221b0 0x564180d10460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 0x564180d10a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180d722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b2ee0 con 0x56418092a390 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b2810 con 0x5641809221b0 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70481afa30 con 0x56418094b070 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f705633e740 10 client.0 _ll_register_callbacks cb 0x7f70440eada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 0x564180d10a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 0x564180d10a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47420/0 (socket says 172.21.15.31:47420) 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 -- 172.21.15.31:0/1569955508 learned_addr learned my addr 172.21.15.31:0/1569955508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:06.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704db66640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641809221b0 0x564180d10460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704e367640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180d722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 -- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 msgr2=0x564180d722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180d722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 -- 172.21.15.31:0/1569955508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641809221b0 msgr2=0x564180d10460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641809221b0 0x564180d10460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704d365640 1 -- 172.21.15.31:0/1569955508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564180d72910 con 0x56418092a390 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.896+0000 7f704db66640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641809221b0 0x564180d10460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f704d365640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 0x564180d10a40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f704806b6c0 tx=0x7f7038032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21175ed506490dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f70317fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f70317fa640 1 -- 172.21.15.31:0/1569955508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038035040 con 0x56418092a390 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f70317fa640 1 -- 172.21.15.31:0/1569955508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7f7038005040 con 0x56418092a390 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70440ea860 con 0x56418092a390 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f70317fa640 1 -- 172.21.15.31:0/1569955508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7038052050 con 0x56418092a390 2024-09-21T03:00:06.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f705633e740 10 client.17259 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564180d79cc0 con 0x56418092a390 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f70317fa640 1 -- 172.21.15.31:0/1569955508 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70440ea860 con 0x56418092a390 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.897+0000 7f70317fa640 1 -- 172.21.15.31:0/1569955508 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f70380035f0 con 0x56418092a390 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 10 client.17259 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 10 client.17259 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 20 client.17259 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641808deda0 0x564180d7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 4 client.17259 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564180c63cd0 con 0x5641808deda0 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f704db66640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641808deda0 0x564180d7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f70317fa640 10 client.17259 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f704db66640 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641808deda0 0x564180d7c500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad78c0d66f9840a2 server_cookie=6b4c27c87c60cfc3 in_seq=0 out_seq=0 2024-09-21T03:00:06.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f70317fa640 1 -- 172.21.15.31:0/1569955508 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564180c63cd0 con 0x5641808deda0 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f70317fa640 10 client.17259 handle_command_reply: tid=0 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.898+0000 7f705633e740 1 client.17259 shutdown 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 msgr2=0x564180d10a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 0x564180d10a40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f704806b6c0 tx=0x7f7038032c00 comp rx=0 tx=0).stop 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 shutdown_connections 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5641808deda0 0x564180d7c500 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56418094b070 0x564180d722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56418092a390 0x564180d10a40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f70317fa640 0 client.17259 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 --2- 172.21.15.31:0/1569955508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641809221b0 0x564180d10460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 >> 172.21.15.31:0/1569955508 conn(0x56418093f460 msgr2=0x564180cf3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 shutdown_connections 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 1 -- 172.21.15.31:0/1569955508 wait complete. 2024-09-21T03:00:06.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f705633e740 20 client.17259 trim_cache size 0 max 16384 2024-09-21T03:00:06.906 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70340b6680 msgr2=0x7f70340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.899+0000 7f704fdca640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70340b6680 0x7f70340b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f70400095f0 tx=0x7f7040009580 comp rx=0 tx=0).stop 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 msgr2=0x7f704810fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 0x7f704810fdd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f704400b4a0 tx=0x7f704400b960 comp rx=0 tx=0).stop 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 shutdown_connections 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f70340b6680 0x7f70340b8b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70481083c0 0x7f7048110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7048108d10 0x7f7048110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 --2- 172.21.15.31:0/3778588369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7048102a80 0x7f704810fdd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 >> 172.21.15.31:0/3778588369 conn(0x7f7048079740 msgr2=0x7f70481052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 shutdown_connections 2024-09-21T03:00:06.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:06.900+0000 7f704fdca640 1 -- 172.21.15.31:0/3778588369 wait complete. 2024-09-21T03:00:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:06 smithi031 ceph-mon[31209]: pgmap v541: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:00:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:06 smithi165 ceph-mon[34818]: pgmap v541: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:00:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:06 smithi178 ceph-mon[34759]: pgmap v541: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:00:07.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:00:07.939 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 -- 172.21.15.31:0/3408626806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10103b60 msgr2=0x7fcb10103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 --2- 172.21.15.31:0/3408626806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10103b60 0x7fcb10103f70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 -- 172.21.15.31:0/3408626806 shutdown_connections 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 --2- 172.21.15.31:0/3408626806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb10109df0 0x7fcb10110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 --2- 172.21.15.31:0/3408626806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb101094a0 0x7fcb101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 --2- 172.21.15.31:0/3408626806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10103b60 0x7fcb10103f70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 -- 172.21.15.31:0/3408626806 >> 172.21.15.31:0/3408626806 conn(0x7fcb1007b850 msgr2=0x7fcb1007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 -- 172.21.15.31:0/3408626806 shutdown_connections 2024-09-21T03:00:08.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.444+0000 7fcb185e1640 1 -- 172.21.15.31:0/3408626806 wait complete. 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 Processor -- start 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 -- start start 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb10103b60 0x7fcb1019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb101094a0 0x7fcb1019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 0x7fcb101a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10113ab0 con 0x7fcb10109df0 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10113930 con 0x7fcb10103b60 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb185e1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10113c30 con 0x7fcb101094a0 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.445+0000 7fcb16b7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 0x7fcb101a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 0x7fcb101a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47442/0 (socket says 172.21.15.31:47442) 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 -- 172.21.15.31:0/980594906 learned_addr learned my addr 172.21.15.31:0/980594906 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb1637d640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb10103b60 0x7fcb1019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb101094a0 0x7fcb1019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 -- 172.21.15.31:0/980594906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb101094a0 msgr2=0x7fcb1019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb101094a0 0x7fcb1019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 -- 172.21.15.31:0/980594906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb10103b60 msgr2=0x7fcb1019c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb10103b60 0x7fcb1019c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 -- 172.21.15.31:0/980594906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101a24b0 con 0x7fcb10109df0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb16b7e640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 0x7fcb101a1e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800cce0 tx=0x7fcb0800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb077fe640 1 -- 172.21.15.31:0/980594906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800f450 con 0x7fcb10109df0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb077fe640 1 -- 172.21.15.31:0/980594906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb0800faa0 con 0x7fcb10109df0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101a27a0 con 0x7fcb10109df0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb077fe640 1 -- 172.21.15.31:0/980594906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08018b70 con 0x7fcb10109df0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.446+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a8fc0 con 0x7fcb10109df0 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.449+0000 7fcb1eb55740 1 Processor -- start 2024-09-21T03:00:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.449+0000 7fcb077fe640 1 -- 172.21.15.31:0/980594906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcb08018d10 con 0x7fcb10109df0 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.449+0000 7fcb1eb55740 1 -- start start 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f71ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a4f7164ea0 con 0x55a4f6e26390 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a4f71542d0 con 0x55a4f6e47070 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a4f71580e0 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71ef120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47448/0 (socket says 172.21.15.31:47448) 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1923895376 learned_addr learned my addr 172.21.15.31:0/1923895376 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f71ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb1637d640 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb077fe640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcafc0b66b0 0x7fcafc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1923895376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e1e1b0 msgr2=0x55a4f71eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1923895376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 msgr2=0x55a4f71ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f71ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.450+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1923895376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f6eb2650 con 0x55a4f6e26390 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.451+0000 7fcb16b7e640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcafc0b66b0 0x7fcafc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.451+0000 7fcb077fe640 1 -- 172.21.15.31:0/980594906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fcb080e7b70 con 0x7fcb10109df0 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.451+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71ef120 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005fd0 tx=0x7fcb0000b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed49c0af55293b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.451+0000 7fcb04ff9640 1 -- 172.21.15.31:0/1923895376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000e920 con 0x55a4f6e26390 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.451+0000 7fcb04ff9640 1 -- 172.21.15.31:0/1923895376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00010430 con 0x55a4f6e26390 2024-09-21T03:00:08.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.451+0000 7fcb04ff9640 1 -- 172.21.15.31:0/1923895376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000f620 con 0x55a4f6e26390 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1923895376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 msgr2=0x55a4f71ef120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71ef120 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005fd0 tx=0x7fcb0000b440 comp rx=0 tx=0).stop 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb16b7e640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcafc0b66b0 0x7fcafc0b8bc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcb080e5390 tx=0x7fcb08018610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1923895376 shutdown_connections 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71ef120 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1923895376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f71ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1923895376 >> 172.21.15.31:0/1923895376 conn(0x55a4f6e3b460 msgr2=0x55a4f71ec590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1923895376 shutdown_connections 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1923895376 wait complete. 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 -- start start 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.452+0000 7fcb1eb55740 1 Processor -- start 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e47070 0x55a4f71fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a4f6fb5700 con 0x55a4f6e26390 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a4f70ef160 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a4f71fce80 con 0x55a4f6e47070 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1eb55740 10 client.0 _ll_register_callbacks cb 0x55a4f71089e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47452/0 (socket says 172.21.15.31:47452) 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 -- 172.21.15.31:0/2020361103 learned_addr learned my addr 172.21.15.31:0/2020361103 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e47070 0x55a4f71fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 -- 172.21.15.31:0/2020361103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e47070 msgr2=0x55a4f71fa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e47070 0x55a4f71fa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 -- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 msgr2=0x55a4f71f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 -- 172.21.15.31:0/2020361103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f711cc90 con 0x55a4f6e26390 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcb1637d640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71fa1e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d207705b04a8733b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.453+0000 7fcaf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/2020361103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c035040 con 0x55a4f6e26390 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb00010430 con 0x55a4f6e26390 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/2020361103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fcb0c004040 con 0x55a4f6e26390 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/2020361103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c004240 con 0x55a4f6e26390 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 10 client.17271 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4f7208b50 con 0x55a4f6e26390 2024-09-21T03:00:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/2020361103 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00010430 con 0x55a4f6e26390 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/2020361103 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcb0c0465a0 con 0x55a4f6e26390 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 10 client.17271 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 20 client.17271 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6e059f0 0x55a4f720bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 4 client.17271 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.454+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f7166df0 con 0x55a4f6e059f0 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.455+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6e059f0 0x55a4f720bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.455+0000 7fcaf9ffb640 10 client.17271 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.455+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6e059f0 0x55a4f720bfd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db6c011831cb4c25 server_cookie=fd742e2737376409 in_seq=0 out_seq=0 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.456+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/2020361103 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a4f7166df0 con 0x55a4f6e059f0 2024-09-21T03:00:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.456+0000 7fcaf9ffb640 10 client.17271 handle_command_reply: tid=0 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.456+0000 7fcb1eb55740 1 client.17271 shutdown 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.456+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 msgr2=0x55a4f71fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.456+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71fa1e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c0316d0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 shutdown_connections 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6e059f0 0x55a4f720bfd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcaf9ffb640 0 client.17271 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e47070 0x55a4f71fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e26390 0x55a4f71fa1e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/2020361103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 >> 172.21.15.31:0/2020361103 conn(0x55a4f6e3b460 msgr2=0x55a4f71f1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 shutdown_connections 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 1 -- 172.21.15.31:0/2020361103 wait complete. 2024-09-21T03:00:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.457+0000 7fcb1eb55740 20 client.17271 trim_cache size 0 max 16384 2024-09-21T03:00:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 Processor -- start 2024-09-21T03:00:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 -- start start 2024-09-21T03:00:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 0x55a4f71f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b16f0 con 0x55a4f6e47070 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b3230 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b1dc0 con 0x55a4f6e26390 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb15b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 0x55a4f71f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb15b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 0x55a4f71f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47460/0 (socket says 172.21.15.31:47460) 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.470+0000 7fcb15b7c640 1 -- 172.21.15.31:0/3589340823 learned_addr learned my addr 172.21.15.31:0/3589340823 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb1637d640 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 -- 172.21.15.31:0/3589340823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 msgr2=0x55a4f71fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 -- 172.21.15.31:0/3589340823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 msgr2=0x55a4f71f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 -- 172.21.15.31:0/3589340823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f71faf60 con 0x55a4f6e47070 2024-09-21T03:00:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 0x55a4f71f6e90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcb0001a000 tx=0x7fcb00010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96105e60c1095083 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcafbfff640 1 -- 172.21.15.31:0/3589340823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0002c040 con 0x55a4f6e47070 2024-09-21T03:00:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcafbfff640 1 -- 172.21.15.31:0/3589340823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00005520 con 0x55a4f6e47070 2024-09-21T03:00:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.471+0000 7fcafbfff640 1 -- 172.21.15.31:0/3589340823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000e720 con 0x55a4f6e47070 2024-09-21T03:00:08.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 -- 172.21.15.31:0/3589340823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 msgr2=0x55a4f71f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 0x55a4f71f6e90 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcb0001a000 tx=0x7fcb00010d30 comp rx=0 tx=0).stop 2024-09-21T03:00:08.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 -- 172.21.15.31:0/3589340823 shutdown_connections 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e47070 0x55a4f71f6e90 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/3589340823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 -- 172.21.15.31:0/3589340823 >> 172.21.15.31:0/3589340823 conn(0x55a4f6e3b460 msgr2=0x55a4f71fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 -- 172.21.15.31:0/3589340823 shutdown_connections 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 -- 172.21.15.31:0/3589340823 wait complete. 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.472+0000 7fcb1eb55740 1 -- start start 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 Processor -- start 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71feb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f7275a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b2b60 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b2490 con 0x55a4f6e47070 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101af6d0 con 0x55a4f6e26390 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1eb55740 10 client.0 _ll_register_callbacks cb 0x7fcb0c031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71fe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47470/0 (socket says 172.21.15.31:47470) 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1640430053 learned_addr learned my addr 172.21.15.31:0/1640430053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb1637d640 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1640430053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 msgr2=0x55a4f71feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 msgr2=0x55a4f7275a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f7275a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.473+0000 7fcb15b7c640 1 -- 172.21.15.31:0/1640430053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f7275f60 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcb15b7c640 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71fe550 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcb00010d00 tx=0x7fcb000136f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d055349c5290b0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcaf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/1640430053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00035020 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/1640430053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fcb0000bbf0 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb00005520 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/1640430053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb00010800 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcb1eb55740 10 client.17277 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a4f7277110 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/1640430053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00005520 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.474+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/1640430053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcb000035f0 con 0x55a4f6e1e1b0 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 10 client.17277 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 10 client.17277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 20 client.17277 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6ddada0 0x55a4f7279670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 4 client.17277 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4f71628a0 con 0x55a4f6ddada0 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1637d640 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6ddada0 0x55a4f7279670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1637d640 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6ddada0 0x55a4f7279670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ac9b07233995ecb server_cookie=53a6c83786aba2f2 in_seq=0 out_seq=0 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcaf9ffb640 10 client.17277 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcaf9ffb640 1 -- 172.21.15.31:0/1640430053 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4f71628a0 con 0x55a4f6ddada0 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcaf9ffb640 10 client.17277 handle_command_reply: tid=0 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.475+0000 7fcb1eb55740 1 client.17277 shutdown 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 msgr2=0x55a4f71fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71fe550 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcb00010d00 tx=0x7fcb000136f0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 shutdown_connections 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a4f6ddada0 0x55a4f7279670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4f6e26390 0x55a4f71feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a4f6e47070 0x55a4f7275a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 --2- 172.21.15.31:0/1640430053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4f6e1e1b0 0x55a4f71fe550 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 >> 172.21.15.31:0/1640430053 conn(0x55a4f6e3b460 msgr2=0x55a4f71efe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcaf9ffb640 0 client.17277 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 shutdown_connections 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 1 -- 172.21.15.31:0/1640430053 wait complete. 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb1eb55740 20 client.17277 trim_cache size 0 max 16384 2024-09-21T03:00:08.482 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcafc0b66b0 msgr2=0x7fcafc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcafc0b66b0 0x7fcafc0b8bc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcb080e5390 tx=0x7fcb08018610 comp rx=0 tx=0).stop 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 msgr2=0x7fcb101a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 0x7fcb101a1e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800cce0 tx=0x7fcb0800a620 comp rx=0 tx=0).stop 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 shutdown_connections 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcafc0b66b0 0x7fcafc0b8bc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb101094a0 0x7fcb1019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb10109df0 0x7fcb101a1e40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 --2- 172.21.15.31:0/980594906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb10103b60 0x7fcb1019c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.476+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 >> 172.21.15.31:0/980594906 conn(0x7fcb1007b850 msgr2=0x7fcb101072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.477+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 shutdown_connections 2024-09-21T03:00:08.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:08.477+0000 7fcb185e1640 1 -- 172.21.15.31:0/980594906 wait complete. 2024-09-21T03:00:09.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:08 smithi031 ceph-mon[31209]: pgmap v542: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:00:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:08 smithi165 ceph-mon[34818]: pgmap v542: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:00:09.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:00:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:08 smithi178 ceph-mon[34759]: pgmap v542: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:00:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 -- 172.21.15.31:0/2036306997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 msgr2=0x7fcba8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/2036306997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8110620 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ca0 tx=0x7fcba002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 -- 172.21.15.31:0/2036306997 shutdown_connections 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/2036306997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba81094a0 0x7fcba81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/2036306997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8110620 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/2036306997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba8103b60 0x7fcba8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 -- 172.21.15.31:0/2036306997 >> 172.21.15.31:0/2036306997 conn(0x7fcba807b850 msgr2=0x7fcba807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 -- 172.21.15.31:0/2036306997 shutdown_connections 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 -- 172.21.15.31:0/2036306997 wait complete. 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 Processor -- start 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.987+0000 7fcbaebc3640 1 -- start start 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbaebc3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8103b60 0x7fcba8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbaebc3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba81094a0 0x7fcba8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbaebc3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbaebc3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8113ab0 con 0x7fcba8109df0 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbaebc3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8113930 con 0x7fcba81094a0 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbaebc3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8113c30 con 0x7fcba8103b60 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59606/0 (socket says 172.21.15.31:59606) 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 -- 172.21.15.31:0/284205 learned_addr learned my addr 172.21.15.31:0/284205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbac95f640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8103b60 0x7fcba8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcba7fff640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba81094a0 0x7fcba8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 -- 172.21.15.31:0/284205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8103b60 msgr2=0x7fcba8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8103b60 0x7fcba8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 -- 172.21.15.31:0/284205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba81094a0 msgr2=0x7fcba8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba81094a0 0x7fcba8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 -- 172.21.15.31:0/284205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba81a8610 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcba7fff640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba81094a0 0x7fcba8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.988+0000 7fcbad160640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8076190 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005cc0 tx=0x7fcba002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.989+0000 7fcba5ffb640 1 -- 172.21.15.31:0/284205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0003bf0 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.989+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba81a88a0 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.989+0000 7fcba5ffb640 1 -- 172.21.15.31:0/284205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba0003d90 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.989+0000 7fcba5ffb640 1 -- 172.21.15.31:0/284205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba002fb20 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.989+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a8e30 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.992+0000 7fcbb5137740 1 Processor -- start 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.992+0000 7fcba5ffb640 1 -- 172.21.15.31:0/284205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcba002fcc0 con 0x7fcba8109df0 2024-09-21T03:00:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.992+0000 7fcbb5137740 1 -- start start 2024-09-21T03:00:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.992+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff14505070 0x55ff148ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.992+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.992+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcbb5137740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ff14822ea0 con 0x55ff144e4390 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcbb5137740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ff148122d0 con 0x55ff144dc1b0 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcbb5137740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ff148160e0 con 0x55ff14505070 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcbac95f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59616/0 (socket says 172.21.15.31:59616) 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 -- 172.21.15.31:0/2873548305 learned_addr learned my addr 172.21.15.31:0/2873548305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff14505070 0x55ff148ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 -- 172.21.15.31:0/2873548305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff14505070 msgr2=0x55ff148ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff14505070 0x55ff148ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 -- 172.21.15.31:0/2873548305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 msgr2=0x55ff148acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 -- 172.21.15.31:0/2873548305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff14570650 con 0x55ff144e4390 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba5ffb640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb980b6680 0x7fcb980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcbac95f640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb980b6680 0x7fcb980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba7fff640 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148ad120 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcb94013850 tx=0x7fcb9400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b688657c9119b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.993+0000 7fcba5ffb640 1 -- 172.21.15.31:0/284205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fcba0108a10 con 0x7fcba8109df0 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.994+0000 7fcb92ffd640 1 -- 172.21.15.31:0/2873548305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94004220 con 0x55ff144e4390 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.994+0000 7fcb92ffd640 1 -- 172.21.15.31:0/2873548305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94017070 con 0x55ff144e4390 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.994+0000 7fcb92ffd640 1 -- 172.21.15.31:0/2873548305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94003810 con 0x55ff144e4390 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.994+0000 7fcbb5137740 1 -- 172.21.15.31:0/2873548305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 msgr2=0x55ff148ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.994+0000 7fcbac95f640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb980b6680 0x7fcb980b8b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55ff148ab320 tx=0x7fcb9c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.994+0000 7fcbb5137740 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148ad120 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcb94013850 tx=0x7fcb9400b3e0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 -- 172.21.15.31:0/2873548305 shutdown_connections 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148ad120 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 --2- 172.21.15.31:0/2873548305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff14505070 0x55ff148ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 -- 172.21.15.31:0/2873548305 >> 172.21.15.31:0/2873548305 conn(0x55ff144f9460 msgr2=0x55ff148aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 -- 172.21.15.31:0/2873548305 shutdown_connections 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 -- 172.21.15.31:0/2873548305 wait complete. 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 -- start start 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.995+0000 7fcbb5137740 1 Processor -- start 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ff14673700 con 0x55ff144e4390 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ff147ad160 con 0x55ff14505070 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ff148bae80 con 0x55ff144dc1b0 2024-09-21T03:00:10.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbb5137740 10 client.0 _ll_register_callbacks cb 0x55ff147c69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59622/0 (socket says 172.21.15.31:59622) 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 -- 172.21.15.31:0/1140009019 learned_addr learned my addr 172.21.15.31:0/1140009019 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbad160640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcbac95f640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 -- 172.21.15.31:0/1140009019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 msgr2=0x55ff148b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 -- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 msgr2=0x55ff148b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.996+0000 7fcba7fff640 1 -- 172.21.15.31:0/1140009019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff147dac90 con 0x55ff144e4390 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcba7fff640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b8220 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcb940045a0 tx=0x7fcb940045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e118454a8014fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcb87fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcb87fff640 1 -- 172.21.15.31:0/1140009019 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400e6f0 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb94017070 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcb87fff640 1 -- 172.21.15.31:0/1140009019 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fcb9400e890 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcb87fff640 1 -- 172.21.15.31:0/1140009019 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb9400ea90 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcbb5137740 10 client.17289 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff148c6b80 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcb87fff640 1 -- 172.21.15.31:0/1140009019 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94017070 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.997+0000 7fcb87fff640 1 -- 172.21.15.31:0/1140009019 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcb94030070 con 0x55ff144e4390 2024-09-21T03:00:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbb5137740 10 client.17289 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:10.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbb5137740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbb5137740 20 client.17289 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbb5137740 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff144c39f0 0x55ff148c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbb5137740 4 client.17289 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff14824df0 con 0x55ff144c39f0 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbac95f640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff144c39f0 0x55ff148c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcbac95f640 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff144c39f0 0x55ff148c9f90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5160b81a0455edf server_cookie=df62f414b584c2b3 in_seq=0 out_seq=0 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.998+0000 7fcb87fff640 10 client.17289 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcb87fff640 1 -- 172.21.15.31:0/1140009019 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ff14824df0 con 0x55ff144c39f0 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcb87fff640 10 client.17289 handle_command_reply: tid=0 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcbb5137740 1 client.17289 shutdown 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 msgr2=0x55ff148b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcbb5137740 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b8220 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcb940045a0 tx=0x7fcb940045d0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 shutdown_connections 2024-09-21T03:00:10.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:09.999+0000 7fcbb5137740 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff144c39f0 0x55ff148c9f90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b8220 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 1 --2- 172.21.15.31:0/1140009019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcb87fff640 0 client.17289 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 >> 172.21.15.31:0/1140009019 conn(0x55ff144f9460 msgr2=0x55ff148af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 shutdown_connections 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 1 -- 172.21.15.31:0/1140009019 wait complete. 2024-09-21T03:00:10.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.000+0000 7fcbb5137740 20 client.17289 trim_cache size 0 max 16384 2024-09-21T03:00:10.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 Processor -- start 2024-09-21T03:00:10.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 -- start start 2024-09-21T03:00:10.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b17a0 con 0x55ff144e4390 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b32e0 con 0x55ff14505070 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbb5137740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b1e70 con 0x55ff144dc1b0 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcba7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcba7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59634/0 (socket says 172.21.15.31:59634) 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcba7fff640 1 -- 172.21.15.31:0/29529930 learned_addr learned my addr 172.21.15.31:0/29529930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.013+0000 7fcbac95f640 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcbad160640 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcba7fff640 1 -- 172.21.15.31:0/29529930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 msgr2=0x55ff148adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcba7fff640 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcba7fff640 1 -- 172.21.15.31:0/29529930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 msgr2=0x55ff148b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcba7fff640 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcba7fff640 1 -- 172.21.15.31:0/29529930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff144c3c20 con 0x55ff144e4390 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcbad160640 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcba7fff640 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b4ed0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006010 tx=0x7fcb9400b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517f2ed023b95251 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcb927fc640 1 -- 172.21.15.31:0/29529930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94036070 con 0x55ff144e4390 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcb927fc640 1 -- 172.21.15.31:0/29529930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94017040 con 0x55ff144e4390 2024-09-21T03:00:10.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.014+0000 7fcb927fc640 1 -- 172.21.15.31:0/29529930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94003960 con 0x55ff144e4390 2024-09-21T03:00:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 -- 172.21.15.31:0/29529930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 msgr2=0x55ff148b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b4ed0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006010 tx=0x7fcb9400b360 comp rx=0 tx=0).stop 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 -- 172.21.15.31:0/29529930 shutdown_connections 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff14505070 0x55ff148b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff144e4390 0x55ff148b4ed0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 --2- 172.21.15.31:0/29529930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144dc1b0 0x55ff148adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 -- 172.21.15.31:0/29529930 >> 172.21.15.31:0/29529930 conn(0x55ff144f9460 msgr2=0x55ff148ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 -- 172.21.15.31:0/29529930 shutdown_connections 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 -- 172.21.15.31:0/29529930 wait complete. 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 -- start start 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.015+0000 7fcbb5137740 1 Processor -- start 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144e4390 0x55ff148bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 0x55ff148bc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b2c10 con 0x55ff14505070 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b2540 con 0x55ff144dc1b0 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81af780 con 0x55ff144e4390 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbb5137740 10 client.0 _ll_register_callbacks cb 0x7fcb9c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 0x55ff148bc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 0x55ff148bc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59644/0 (socket says 172.21.15.31:59644) 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 -- 172.21.15.31:0/3256533234 learned_addr learned my addr 172.21.15.31:0/3256533234 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcba7fff640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144e4390 0x55ff148bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbac95f640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 -- 172.21.15.31:0/3256533234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144e4390 msgr2=0x55ff148bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144e4390 0x55ff148bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 -- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 msgr2=0x55ff148c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbad160640 1 -- 172.21.15.31:0/3256533234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff148bc9c0 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.016+0000 7fcbac95f640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbad160640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 0x55ff148bc280 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006010 tx=0x7fcba00ce650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9527726c4d33827 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcb87fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcb87fff640 1 -- 172.21.15.31:0/3256533234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00ce390 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcb87fff640 1 -- 172.21.15.31:0/3256533234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8484+0+0 (secure 0 0 0) 0x7fcba00d7e10 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb94017040 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcb87fff640 1 -- 172.21.15.31:0/3256533234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcba00cef80 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbb5137740 10 client.17295 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ff149346a0 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcb87fff640 1 -- 172.21.15.31:0/3256533234 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94017040 con 0x55ff14505070 2024-09-21T03:00:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcb87fff640 1 -- 172.21.15.31:0/3256533234 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcba00cfbb0 con 0x55ff14505070 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbb5137740 10 client.17295 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbb5137740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.017+0000 7fcbb5137740 20 client.17295 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcbb5137740 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff14498da0 0x55ff14936d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcbb5137740 4 client.17295 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff148208a0 con 0x55ff14498da0 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcbac95f640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff14498da0 0x55ff14936d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcbac95f640 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff14498da0 0x55ff14936d80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0345e38fb84c9 server_cookie=3e43263ddbfb8b16 in_seq=0 out_seq=0 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcb87fff640 10 client.17295 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcb87fff640 1 -- 172.21.15.31:0/3256533234 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff148208a0 con 0x55ff14498da0 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcb87fff640 10 client.17295 handle_command_reply: tid=0 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.018+0000 7fcbb5137740 1 client.17295 shutdown 2024-09-21T03:00:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 msgr2=0x55ff148bc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 0x55ff148bc280 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006010 tx=0x7fcba00ce650 comp rx=0 tx=0).stop 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 shutdown_connections 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55ff14498da0 0x55ff14936d80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff144e4390 0x55ff148bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff14505070 0x55ff148bc280 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcb87fff640 0 client.17295 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 --2- 172.21.15.31:0/3256533234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ff144dc1b0 0x55ff148c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 >> 172.21.15.31:0/3256533234 conn(0x55ff144f9460 msgr2=0x55ff148c30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 shutdown_connections 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 1 -- 172.21.15.31:0/3256533234 wait complete. 2024-09-21T03:00:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.019+0000 7fcbb5137740 20 client.17295 trim_cache size 0 max 16384 2024-09-21T03:00:10.026 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb980b6680 msgr2=0x7fcb980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb980b6680 0x7fcb980b8b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55ff148ab320 tx=0x7fcb9c00a2d0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 msgr2=0x7fcba8076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8076190 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005cc0 tx=0x7fcba002f380 comp rx=0 tx=0).stop 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 shutdown_connections 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcb980b6680 0x7fcb980b8b90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba8109df0 0x7fcba8076190 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcba81094a0 0x7fcba8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 --2- 172.21.15.31:0/284205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8103b60 0x7fcba8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 >> 172.21.15.31:0/284205 conn(0x7fcba807b850 msgr2=0x7fcba8107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 shutdown_connections 2024-09-21T03:00:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:10.020+0000 7fcbaebc3640 1 -- 172.21.15.31:0/284205 wait complete. 2024-09-21T03:00:11.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:10 smithi031 ceph-mon[31209]: pgmap v543: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:00:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:10 smithi165 ceph-mon[34818]: pgmap v543: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:00:11.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:10 smithi178 ceph-mon[34759]: pgmap v543: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:00:11.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:00:11.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:00:12.137 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:11 smithi165 ceph-mon[34818]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:00:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:11 smithi178 ceph-mon[34759]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:00:12.259 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:11 smithi031 ceph-mon[31209]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 -- 172.21.15.31:0/2840789253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc108d10 msgr2=0x7ff1fc10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 --2- 172.21.15.31:0/2840789253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc108d10 0x7ff1fc10f540 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009ca0 tx=0x7ff1f402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 -- 172.21.15.31:0/2840789253 shutdown_connections 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 --2- 172.21.15.31:0/2840789253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc108d10 0x7ff1fc10f540 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 --2- 172.21.15.31:0/2840789253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 --2- 172.21.15.31:0/2840789253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1fc102ad0 0x7ff1fc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 -- 172.21.15.31:0/2840789253 >> 172.21.15.31:0/2840789253 conn(0x7ff1fc0796d0 msgr2=0x7ff1fc079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 -- 172.21.15.31:0/2840789253 shutdown_connections 2024-09-21T03:00:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.644+0000 7ff202b0a640 1 -- 172.21.15.31:0/2840789253 wait complete. 2024-09-21T03:00:12.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 Processor -- start 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 -- start start 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1fc102ad0 0x7ff1fc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1fc108d10 0x7ff1fc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1129d0 con 0x7ff1fc1083c0 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc112850 con 0x7ff1fc102ad0 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff202b0a640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc112b50 con 0x7ff1fc108d10 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff1fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff1fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59664/0 (socket says 172.21.15.31:59664) 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff1fbfff640 1 -- 172.21.15.31:0/622860717 learned_addr learned my addr 172.21.15.31:0/622860717 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.645+0000 7ff2008a6640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1fc102ad0 0x7ff1fc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff2010a7640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1fc108d10 0x7ff1fc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1fbfff640 1 -- 172.21.15.31:0/622860717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1fc108d10 msgr2=0x7ff1fc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1fc108d10 0x7ff1fc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1fbfff640 1 -- 172.21.15.31:0/622860717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1fc102ad0 msgr2=0x7ff1fc19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1fc102ad0 0x7ff1fc19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1fbfff640 1 -- 172.21.15.31:0/622860717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc1a0320 con 0x7ff1fc1083c0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc19aa10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1f9ffb640 1 -- 172.21.15.31:0/622860717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800ccc0 con 0x7ff1fc1083c0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc1a0610 con 0x7ff1fc1083c0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1f9ffb640 1 -- 172.21.15.31:0/622860717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e80108e0 con 0x7ff1fc1083c0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff1f9ffb640 1 -- 172.21.15.31:0/622860717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8018880 con 0x7ff1fc1083c0 2024-09-21T03:00:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.646+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1a6e30 con 0x7ff1fc1083c0 2024-09-21T03:00:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 Processor -- start 2024-09-21T03:00:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 -- start start 2024-09-21T03:00:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff1f9ffb640 1 -- 172.21.15.31:0/622860717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff1e8018a20 con 0x7ff1fc1083c0 2024-09-21T03:00:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74935430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a74935970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74935eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563a748abe60 con 0x563a74565490 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563a7489b290 con 0x563a7456d670 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.649+0000 7ff20907e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a7489f0a0 con 0x563a7458e350 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a74935970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a74935970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59668/0 (socket says 172.21.15.31:59668) 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 -- 172.21.15.31:0/781470364 learned_addr learned my addr 172.21.15.31:0/781470364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2010a7640 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74935430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74935eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 -- 172.21.15.31:0/781470364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 msgr2=0x563a74935430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74935430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 -- 172.21.15.31:0/781470364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 msgr2=0x563a74935eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74935eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 -- 172.21.15.31:0/781470364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a745f9930 con 0x563a74565490 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff1f9ffb640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1ec0b66b0 0x7ff1ec0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2008a6640 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a74935970 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000ad00 tx=0x7ff1f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa89ee4900e9654 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.650+0000 7ff2010a7640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1ec0b66b0 0x7ff1ec0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.651+0000 7ff1f9ffb640 1 -- 172.21.15.31:0/622860717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8572+0+0 (secure 0 0 0) 0x7ff1e80e7b60 con 0x7ff1fc1083c0 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.651+0000 7ff1e6ffd640 1 -- 172.21.15.31:0/781470364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f001d070 con 0x563a74565490 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.651+0000 7ff1e6ffd640 1 -- 172.21.15.31:0/781470364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f0018af0 con 0x563a74565490 2024-09-21T03:00:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.651+0000 7ff1e6ffd640 1 -- 172.21.15.31:0/781470364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0010b30 con 0x563a74565490 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff2010a7640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1ec0b66b0 0x7ff1ec0b8bc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4005cc0 tx=0x7ff1f4005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 -- 172.21.15.31:0/781470364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 msgr2=0x563a74935970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a74935970 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000ad00 tx=0x7ff1f000f790 comp rx=0 tx=0).stop 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 -- 172.21.15.31:0/781470364 shutdown_connections 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74935eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a74935970 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 --2- 172.21.15.31:0/781470364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74935430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 -- 172.21.15.31:0/781470364 >> 172.21.15.31:0/781470364 conn(0x563a74582740 msgr2=0x563a74933320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 -- 172.21.15.31:0/781470364 shutdown_connections 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.652+0000 7ff20907e740 1 -- 172.21.15.31:0/781470364 wait complete. 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.653+0000 7ff20907e740 1 -- start start 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.653+0000 7ff20907e740 1 Processor -- start 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74941000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74941540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a7494cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563a746fc890 con 0x563a7458e350 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563a74835d40 con 0x563a7456d670 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a74943c10 con 0x563a74565490 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff2010a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a7494cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff20907e740 10 client.0 _ll_register_callbacks cb 0x563a74899c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff1fbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74941540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff2010a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a7494cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59670/0 (socket says 172.21.15.31:59670) 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff2010a7640 1 -- 172.21.15.31:0/3955686497 learned_addr learned my addr 172.21.15.31:0/3955686497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff2008a6640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74941000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff1fbfff640 1 -- 172.21.15.31:0/3955686497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 msgr2=0x563a74941000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74941000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff1fbfff640 1 -- 172.21.15.31:0/3955686497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 msgr2=0x563a7494cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a7494cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff1fbfff640 1 -- 172.21.15.31:0/3955686497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a7494d4c0 con 0x563a7456d670 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.654+0000 7ff2010a7640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a7494cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74941540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80e5280 tx=0x7ff1e80aa0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394f5855ab2448f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1dbfff640 1 -- 172.21.15.31:0/3955686497 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80ad0f0 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0018af0 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1dbfff640 1 -- 172.21.15.31:0/3955686497 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8572+0+0 (secure 0 0 0) 0x7ff1e80acde0 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1dbfff640 1 -- 172.21.15.31:0/3955686497 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1e80a90f0 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff20907e740 10 client.26660 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a7494fe30 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1dbfff640 1 -- 172.21.15.31:0/3955686497 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f0018af0 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.655+0000 7ff1dbfff640 1 -- 172.21.15.31:0/3955686497 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1e80ac650 con 0x563a7456d670 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff20907e740 10 client.26660 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff20907e740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff20907e740 20 client.26660 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff20907e740 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a7454ccd0 0x563a74953350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff20907e740 4 client.26660 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a748addb0 con 0x563a7454ccd0 2024-09-21T03:00:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff2008a6640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a7454ccd0 0x563a74953350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff2008a6640 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a7454ccd0 0x563a74953350 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=153df594d895dbd server_cookie=ab2742a6907a069 in_seq=0 out_seq=0 2024-09-21T03:00:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.656+0000 7ff1dbfff640 10 client.26660 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff1dbfff640 1 -- 172.21.15.31:0/3955686497 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a748addb0 con 0x563a7454ccd0 2024-09-21T03:00:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff1dbfff640 10 client.26660 handle_command_reply: tid=0 2024-09-21T03:00:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 client.26660 shutdown 2024-09-21T03:00:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 msgr2=0x563a74941540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74941540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80e5280 tx=0x7ff1e80aa0c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 shutdown_connections 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a7454ccd0 0x563a74953350 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a7494cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74941540 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff1dbfff640 0 client.26660 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 --2- 172.21.15.31:0/3955686497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74941000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.657+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 >> 172.21.15.31:0/3955686497 conn(0x563a74582740 msgr2=0x563a749385c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.658+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 shutdown_connections 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.658+0000 7ff20907e740 1 -- 172.21.15.31:0/3955686497 wait complete. 2024-09-21T03:00:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.658+0000 7ff20907e740 20 client.26660 trim_cache size 0 max 16384 2024-09-21T03:00:12.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 Processor -- start 2024-09-21T03:00:12.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 -- start start 2024-09-21T03:00:12.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a7494ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74940690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74942a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1af830 con 0x563a74565490 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b1370 con 0x563a7456d670 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff20907e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1aff00 con 0x563a7458e350 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff2008a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a7494ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff2008a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a7494ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59676/0 (socket says 172.21.15.31:59676) 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff2008a6640 1 -- 172.21.15.31:0/3772202765 learned_addr learned my addr 172.21.15.31:0/3772202765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.691+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74940690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2010a7640 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74942a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2008a6640 1 -- 172.21.15.31:0/3772202765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 msgr2=0x563a74942a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2008a6640 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74942a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2008a6640 1 -- 172.21.15.31:0/3772202765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 msgr2=0x563a74940690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2008a6640 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74940690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2008a6640 1 -- 172.21.15.31:0/3772202765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a7493f300 con 0x563a74565490 2024-09-21T03:00:12.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff2008a6640 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a7494ce30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x563a749340b0 tx=0x7ff1f000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1c1ff3447eaba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff1e67fc640 1 -- 172.21.15.31:0/3772202765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f001d040 con 0x563a74565490 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff1e67fc640 1 -- 172.21.15.31:0/3772202765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f002f070 con 0x563a74565490 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.692+0000 7ff1e67fc640 1 -- 172.21.15.31:0/3772202765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0005650 con 0x563a74565490 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 -- 172.21.15.31:0/3772202765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 msgr2=0x563a7494ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a7494ce30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x563a749340b0 tx=0x7ff1f000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 -- 172.21.15.31:0/3772202765 shutdown_connections 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a7458e350 0x563a74942a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74940690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 --2- 172.21.15.31:0/3772202765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a74565490 0x563a7494ce30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 -- 172.21.15.31:0/3772202765 >> 172.21.15.31:0/3772202765 conn(0x563a74582740 msgr2=0x563a74942320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 -- 172.21.15.31:0/3772202765 shutdown_connections 2024-09-21T03:00:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 -- 172.21.15.31:0/3772202765 wait complete. 2024-09-21T03:00:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 -- start start 2024-09-21T03:00:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.694+0000 7ff20907e740 1 Processor -- start 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74945530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74945a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a749bccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b0ca0 con 0x563a7458e350 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1b05d0 con 0x563a7456d670 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc1ad830 con 0x563a74565490 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff2010a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a749bccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff20907e740 10 client.0 _ll_register_callbacks cb 0x7ff1e800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff2010a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a749bccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59682/0 (socket says 172.21.15.31:59682) 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff2010a7640 1 -- 172.21.15.31:0/726333781 learned_addr learned my addr 172.21.15.31:0/726333781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff2008a6640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74945530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.695+0000 7ff1fbfff640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74945a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2010a7640 1 -- 172.21.15.31:0/726333781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 msgr2=0x563a74945530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2010a7640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74945530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2010a7640 1 -- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 msgr2=0x563a74945a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2010a7640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74945a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2010a7640 1 -- 172.21.15.31:0/726333781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a749bd230 con 0x563a7458e350 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2008a6640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74945530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff2010a7640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a749bccf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4031a30 tx=0x7ff1f40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef277ee2575c5c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff1dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff1dbfff640 1 -- 172.21.15.31:0/726333781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4058060 con 0x563a7458e350 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff1dbfff640 1 -- 172.21.15.31:0/726333781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8572+0+0 (secure 0 0 0) 0x7ff1f40043d0 con 0x563a7458e350 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f002f070 con 0x563a7458e350 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff1dbfff640 1 -- 172.21.15.31:0/726333781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1f40045d0 con 0x563a7458e350 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff20907e740 10 client.17313 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.696+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a749be4b0 con 0x563a7458e350 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff1dbfff640 1 -- 172.21.15.31:0/726333781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f002f070 con 0x563a7458e350 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff1dbfff640 1 -- 172.21.15.31:0/726333781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1f4030750 con 0x563a7458e350 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff20907e740 10 client.17313 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff20907e740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff20907e740 20 client.17313 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff20907e740 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a74522080 0x563a749c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff20907e740 4 client.17313 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a748a9860 con 0x563a74522080 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.697+0000 7ff2008a6640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a74522080 0x563a749c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.698+0000 7ff2008a6640 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a74522080 0x563a749c0cd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d042af8255775e server_cookie=6a238d23df81ed86 in_seq=0 out_seq=0 2024-09-21T03:00:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.698+0000 7ff1dbfff640 10 client.17313 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:12.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.699+0000 7ff1dbfff640 1 -- 172.21.15.31:0/726333781 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a748a9860 con 0x563a74522080 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.699+0000 7ff1dbfff640 10 client.17313 handle_command_reply: tid=0 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.699+0000 7ff20907e740 1 client.17313 shutdown 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.699+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 msgr2=0x563a749bccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.699+0000 7ff20907e740 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a749bccf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4031a30 tx=0x7ff1f40095c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 shutdown_connections 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x563a74522080 0x563a749c0cd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a7458e350 0x563a749bccf0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a7456d670 0x563a74945a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff1dbfff640 0 client.17313 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 --2- 172.21.15.31:0/726333781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a74565490 0x563a74945530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 >> 172.21.15.31:0/726333781 conn(0x563a74582740 msgr2=0x563a7494be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 shutdown_connections 2024-09-21T03:00:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 1 -- 172.21.15.31:0/726333781 wait complete. 2024-09-21T03:00:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.700+0000 7ff20907e740 20 client.17313 trim_cache size 0 max 16384 2024-09-21T03:00:12.704 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:00:12.704 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T03:00:12.704 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea", 2024-09-21T03:00:12.704 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T03:00:12.704 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1ec0b66b0 msgr2=0x7ff1ec0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1ec0b66b0 0x7ff1ec0b8bc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4005cc0 tx=0x7ff1f4005c30 comp rx=0 tx=0).stop 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 msgr2=0x7ff1fc19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc19aa10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e9f0 tx=0x7ff1e800edf0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 shutdown_connections 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1ec0b66b0 0x7ff1ec0b8bc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1fc108d10 0x7ff1fc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1fc1083c0 0x7ff1fc19aa10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 --2- 172.21.15.31:0/622860717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1fc102ad0 0x7ff1fc19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 >> 172.21.15.31:0/622860717 conn(0x7ff1fc0796d0 msgr2=0x7ff1fc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 shutdown_connections 2024-09-21T03:00:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:12.701+0000 7ff202b0a640 1 -- 172.21.15.31:0/622860717 wait complete. 2024-09-21T03:00:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:12 smithi165 ceph-mon[34818]: pgmap v545: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:00:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:12 smithi178 ceph-mon[34759]: pgmap v545: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:00:13.278 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:12 smithi031 ceph-mon[31209]: pgmap v545: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:00:13.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:00:13.784 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:13 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T03:00:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:13 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:13 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T03:00:14.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:13 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:13 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T03:00:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:13 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:13 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T03:00:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:13 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:13 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T03:00:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:13 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:13 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T03:00:14.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:13 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 -- 172.21.15.31:0/3248239315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d81049b0 msgr2=0x7fe3d8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 --2- 172.21.15.31:0/3248239315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d81049b0 0x7fe3d8104dc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009900 tx=0x7fe3d402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 -- 172.21.15.31:0/3248239315 shutdown_connections 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 --2- 172.21.15.31:0/3248239315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 0x7fe3d810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 --2- 172.21.15.31:0/3248239315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3d810de10 0x7fe3d8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 --2- 172.21.15.31:0/3248239315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d81049b0 0x7fe3d8104dc0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 -- 172.21.15.31:0/3248239315 >> 172.21.15.31:0/3248239315 conn(0x7fe3d8079730 msgr2=0x7fe3d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 -- 172.21.15.31:0/3248239315 shutdown_connections 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.309+0000 7fe3defa3640 1 -- 172.21.15.31:0/3248239315 wait complete. 2024-09-21T03:00:14.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 Processor -- start 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 -- start start 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3d81049b0 0x7fe3d81026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 0x7fe3d8102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 0x7fe3d8103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8113a70 con 0x7fe3d810de10 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81138f0 con 0x7fe3d81049b0 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.310+0000 7fe3defa3640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8113bf0 con 0x7fe3d8105300 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 0x7fe3d8103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 0x7fe3d8102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 0x7fe3d8103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59692/0 (socket says 172.21.15.31:59692) 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 -- 172.21.15.31:0/2241992314 learned_addr learned my addr 172.21.15.31:0/2241992314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:14.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 0x7fe3d8102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54214/0 (socket says 172.21.15.31:54214) 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3d81049b0 0x7fe3d81026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 -- 172.21.15.31:0/2241992314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 msgr2=0x7fe3d8102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 0x7fe3d8102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 -- 172.21.15.31:0/2241992314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3d81049b0 msgr2=0x7fe3d81026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3d81049b0 0x7fe3d81026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 -- 172.21.15.31:0/2241992314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d81a8800 con 0x7fe3d810de10 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3dd540640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 0x7fe3d8103170 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000eaa0 tx=0x7fe3d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3cdffb640 1 -- 172.21.15.31:0/2241992314 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000cdb0 con 0x7fe3d810de10 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.311+0000 7fe3cdffb640 1 -- 172.21.15.31:0/2241992314 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d0005020 con 0x7fe3d810de10 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.312+0000 7fe3cdffb640 1 -- 172.21.15.31:0/2241992314 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00116d0 con 0x7fe3d810de10 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.312+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d81a8af0 con 0x7fe3d810de10 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.312+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a8fe0 con 0x7fe3d810de10 2024-09-21T03:00:14.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.315+0000 7fe3cdffb640 1 -- 172.21.15.31:0/2241992314 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe3d00051d0 con 0x7fe3d810de10 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.315+0000 7fe3e5517740 1 Processor -- start 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.315+0000 7fe3e5517740 1 -- start start 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.315+0000 7fe3cdffb640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3c00b6680 0x7fe3c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0e1c070 0x55afe11c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0dfb390 0x55afe11c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3e5517740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55afe1139e50 con 0x55afe0e1c070 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3e5517740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afe1129280 con 0x55afe0dfb390 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3e5517740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55afe112d090 con 0x55afe0df31b0 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cdffb640 1 -- 172.21.15.31:0/2241992314 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8572+0+0 (secure 0 0 0) 0x7fe3d00e7820 con 0x7fe3d810de10 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3c00b6680 0x7fe3c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54226/0 (socket says 172.21.15.31:54226) 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 -- 172.21.15.31:0/1991489726 learned_addr learned my addr 172.21.15.31:0/1991489726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0e1c070 0x55afe11c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3dd540640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0dfb390 0x55afe11c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3c00b6680 0x7fe3c00b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3d806ad30 tx=0x7fe3c80128d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 -- 172.21.15.31:0/1991489726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0dfb390 msgr2=0x55afe11c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0dfb390 0x55afe11c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 -- 172.21.15.31:0/1991489726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0e1c070 msgr2=0x55afe11c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0e1c070 0x55afe11c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3cffff640 1 -- 172.21.15.31:0/1991489726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe0e87650 con 0x55afe0df31b0 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.316+0000 7fe3dd540640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0dfb390 0x55afe11c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.317+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0e1c070 0x55afe11c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:14.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.317+0000 7fe3cffff640 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11c37b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800e730 tx=0x7fe3c801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e09db841a2f56cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.317+0000 7fe3beffd640 1 -- 172.21.15.31:0/1991489726 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801f600 con 0x55afe0df31b0 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.317+0000 7fe3beffd640 1 -- 172.21.15.31:0/1991489726 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8023020 con 0x55afe0df31b0 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.317+0000 7fe3beffd640 1 -- 172.21.15.31:0/1991489726 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801ea80 con 0x55afe0df31b0 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 -- 172.21.15.31:0/1991489726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 msgr2=0x55afe11c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11c37b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800e730 tx=0x7fe3c801e000 comp rx=0 tx=0).stop 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 -- 172.21.15.31:0/1991489726 shutdown_connections 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11c37b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0dfb390 0x55afe11c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1991489726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0e1c070 0x55afe11c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 -- 172.21.15.31:0/1991489726 >> 172.21.15.31:0/1991489726 conn(0x55afe0e10460 msgr2=0x55afe11c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:14.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 -- 172.21.15.31:0/1991489726 shutdown_connections 2024-09-21T03:00:14.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 -- 172.21.15.31:0/1991489726 wait complete. 2024-09-21T03:00:14.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.318+0000 7fe3e5517740 1 -- start start 2024-09-21T03:00:14.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 Processor -- start 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0df31b0 0x55afe11ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 0x55afe11cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55afe0f8a700 con 0x55afe0dfb390 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afe10c4110 con 0x55afe0df31b0 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55afe11d19f0 con 0x55afe0e1c070 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3e5517740 10 client.0 _ll_register_callbacks cb 0x55afe10dd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3dcd3f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0df31b0 0x55afe11ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3dd540640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 0x55afe11cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59714/0 (socket says 172.21.15.31:59714) 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3dd540640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 0x55afe11cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54228/0 (socket says 172.21.15.31:54228) 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.319+0000 7fe3cffff640 1 -- 172.21.15.31:0/3186552063 learned_addr learned my addr 172.21.15.31:0/3186552063 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3cffff640 1 -- 172.21.15.31:0/3186552063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 msgr2=0x55afe11cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3cffff640 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 0x55afe11cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3cffff640 1 -- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0df31b0 msgr2=0x55afe11ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3cffff640 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0df31b0 0x55afe11ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3cffff640 1 -- 172.21.15.31:0/3186552063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe10f1c40 con 0x55afe0dfb390 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3dd540640 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 0x55afe11cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3cffff640 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11cee30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8022040 tx=0x7fe3c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faddf2869747037d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3186552063 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801f800 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3186552063 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8572+0+0 (secure 0 0 0) 0x7fe3c801f9a0 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.320+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c8023020 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3186552063 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3c801e3b0 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 10 client.17325 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55afe11dd6a0 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3186552063 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8023020 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3186552063 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c8031a00 con 0x55afe0dfb390 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 10 client.17325 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 20 client.17325 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dda9f0 0x55afe11e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 4 client.17325 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afe113bda0 con 0x55afe0dda9f0 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.321+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dda9f0 0x55afe11e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.322+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dda9f0 0x55afe11e0b10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff2983770049bd44 server_cookie=c1b0fba10def76b5 in_seq=0 out_seq=0 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.322+0000 7fe3b7fff640 10 client.17325 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3186552063 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55afe113bda0 con 0x55afe0dda9f0 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3b7fff640 10 client.17325 handle_command_reply: tid=0 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 client.17325 shutdown 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 msgr2=0x55afe11cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11cee30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8022040 tx=0x7fe3c800bd70 comp rx=0 tx=0).stop 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 shutdown_connections 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dda9f0 0x55afe11e0b10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0e1c070 0x55afe11cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11cee30 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3186552063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0df31b0 0x55afe11ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 >> 172.21.15.31:0/3186552063 conn(0x55afe0e10460 msgr2=0x55afe11c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:14.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3b7fff640 0 client.17325 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:14.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 shutdown_connections 2024-09-21T03:00:14.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 1 -- 172.21.15.31:0/3186552063 wait complete. 2024-09-21T03:00:14.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.323+0000 7fe3e5517740 20 client.17325 trim_cache size 0 max 16384 2024-09-21T03:00:14.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.341+0000 7fe3e5517740 1 Processor -- start 2024-09-21T03:00:14.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.341+0000 7fe3e5517740 1 -- start start 2024-09-21T03:00:14.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.341+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 0x55afe11c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.341+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0dfb390 0x55afe11d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe11cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3e5517740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b1a60 con 0x55afe0df31b0 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3e5517740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b35a0 con 0x55afe0e1c070 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3e5517740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b2130 con 0x55afe0dfb390 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 0x55afe11c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 0x55afe11c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59730/0 (socket says 172.21.15.31:59730) 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 -- 172.21.15.31:0/1976292664 learned_addr learned my addr 172.21.15.31:0/1976292664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3cffff640 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0dfb390 0x55afe11d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dd540640 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe11cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 -- 172.21.15.31:0/1976292664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0dfb390 msgr2=0x55afe11d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0dfb390 0x55afe11d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 -- 172.21.15.31:0/1976292664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 msgr2=0x55afe11cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe11cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 -- 172.21.15.31:0/1976292664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afe0ddac20 con 0x55afe0df31b0 2024-09-21T03:00:14.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dd540640 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe11cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.342+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 0x55afe11c4c60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005940 tx=0x7fe3d4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d307d1a400918c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.343+0000 7fe3be7fc640 1 -- 172.21.15.31:0/1976292664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4035040 con 0x55afe0df31b0 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.343+0000 7fe3be7fc640 1 -- 172.21.15.31:0/1976292664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d40054f0 con 0x55afe0df31b0 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.343+0000 7fe3be7fc640 1 -- 172.21.15.31:0/1976292664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40035f0 con 0x55afe0df31b0 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 -- 172.21.15.31:0/1976292664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 msgr2=0x55afe11c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 0x55afe11c4c60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005940 tx=0x7fe3d4031400 comp rx=0 tx=0).stop 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 -- 172.21.15.31:0/1976292664 shutdown_connections 2024-09-21T03:00:14.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0dfb390 0x55afe11d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe11cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 --2- 172.21.15.31:0/1976292664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0df31b0 0x55afe11c4c60 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 -- 172.21.15.31:0/1976292664 >> 172.21.15.31:0/1976292664 conn(0x55afe0e10460 msgr2=0x55afe11cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 -- 172.21.15.31:0/1976292664 shutdown_connections 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 -- 172.21.15.31:0/1976292664 wait complete. 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 -- start start 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.344+0000 7fe3e5517740 1 Processor -- start 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11e15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe1249280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b2ed0 con 0x55afe0dfb390 2024-09-21T03:00:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b2800 con 0x55afe0e1c070 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81afa60 con 0x55afe0df31b0 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3e5517740 10 client.0 _ll_register_callbacks cb 0x7fe3d4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11e15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3dcd3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11e15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59742/0 (socket says 172.21.15.31:59742) 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 -- 172.21.15.31:0/3483626285 learned_addr learned my addr 172.21.15.31:0/3483626285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3dcd3f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11e0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:54258/0 (socket says 172.21.15.31:54258) 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3dd540640 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe1249280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 -- 172.21.15.31:0/3483626285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 msgr2=0x55afe11e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 -- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 msgr2=0x55afe1249280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe1249280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.345+0000 7fe3cffff640 1 -- 172.21.15.31:0/3483626285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afe12497c0 con 0x55afe0dfb390 2024-09-21T03:00:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3cffff640 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11e15d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8028480 tx=0x7fe3c8031bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6583380f7dd83f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3483626285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c803d070 con 0x55afe0dfb390 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3483626285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8572+0+0 (secure 0 0 0) 0x7fe3c801e550 con 0x55afe0dfb390 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d40054f0 con 0x55afe0dfb390 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3483626285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3c8046050 con 0x55afe0dfb390 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3e5517740 10 client.17331 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55afe124a980 con 0x55afe0dfb390 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3483626285 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d40054f0 con 0x55afe0dfb390 2024-09-21T03:00:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.346+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3483626285 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c8040020 con 0x55afe0dfb390 2024-09-21T03:00:14.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3e5517740 10 client.17331 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:14.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3e5517740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:14.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3e5517740 20 client.17331 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:14.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dafda0 0x55afe124d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:14.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3e5517740 4 client.17331 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:14.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55afe1137850 con 0x55afe0dafda0 2024-09-21T03:00:14.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dafda0 0x55afe124d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:14.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3dcd3f640 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dafda0 0x55afe124d0e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b979e2d9fc53ec75 server_cookie=e1d85092f9c5fbb in_seq=0 out_seq=0 2024-09-21T03:00:14.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.347+0000 7fe3b7fff640 10 client.17331 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:14.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3b7fff640 1 -- 172.21.15.31:0/3483626285 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55afe1137850 con 0x55afe0dafda0 2024-09-21T03:00:14.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3b7fff640 10 client.17331 handle_command_reply: tid=0 2024-09-21T03:00:14.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 client.17331 shutdown 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 msgr2=0x55afe11e15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11e15d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8028480 tx=0x7fe3c8031bf0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 shutdown_connections 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afe0dafda0 0x55afe124d0e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afe0e1c070 0x55afe1249280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afe0dfb390 0x55afe11e15d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 --2- 172.21.15.31:0/3483626285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55afe0df31b0 0x55afe11e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 >> 172.21.15.31:0/3483626285 conn(0x55afe0e10460 msgr2=0x55afe11c49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3b7fff640 0 client.17331 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 shutdown_connections 2024-09-21T03:00:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 1 -- 172.21.15.31:0/3483626285 wait complete. 2024-09-21T03:00:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.348+0000 7fe3e5517740 20 client.17331 trim_cache size 0 max 16384 2024-09-21T03:00:14.361 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea", 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T03:00:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:00:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:00:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:00:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:00:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:00:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:14.375 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:00:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3c00b6680 msgr2=0x7fe3c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3c00b6680 0x7fe3c00b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3d806ad30 tx=0x7fe3c80128d0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 msgr2=0x7fe3d8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 0x7fe3d8103170 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000eaa0 tx=0x7fe3d000ef60 comp rx=0 tx=0).stop 2024-09-21T03:00:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 shutdown_connections 2024-09-21T03:00:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe3c00b6680 0x7fe3c00b8b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d8105300 0x7fe3d8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3d810de10 0x7fe3d8103170 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 --2- 172.21.15.31:0/2241992314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3d81049b0 0x7fe3d81026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:14.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 >> 172.21.15.31:0/2241992314 conn(0x7fe3d8079730 msgr2=0x7fe3d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:14.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 shutdown_connections 2024-09-21T03:00:14.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:14.349+0000 7fe3defa3640 1 -- 172.21.15.31:0/2241992314 wait complete. 2024-09-21T03:00:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: pgmap v546: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:00:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:00:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:00:15.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:00:15.124 INFO:tasks.cephfs.filesystem:scrub status for tag:e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea - {'path': '/', 'tag': 'e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea', 'options': 'recursive,force'} 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: pgmap v546: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:00:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:00:15.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: pgmap v546: 97 pgs: 97 active+clean; 259 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:00:15.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:00:15.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:00:15.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:00:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:15 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-21T03:00:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:15 smithi165 ceph-mon[34818]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-21T03:00:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:15 smithi165 ceph-mon[34818]: Unable to set osd_memory_target on smithi178 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-21T03:00:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:15 smithi165 ceph-mon[34818]: Unable to set osd_memory_target on smithi165 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-21T03:00:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:15 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T03:00:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:15 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-21T03:00:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:15 smithi178 ceph-mon[34759]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-21T03:00:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:15 smithi178 ceph-mon[34759]: Unable to set osd_memory_target on smithi178 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-21T03:00:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:15 smithi178 ceph-mon[34759]: Unable to set osd_memory_target on smithi165 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-21T03:00:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:15 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T03:00:16.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:15 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-21T03:00:16.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:15 smithi031 ceph-mon[31209]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-21T03:00:16.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:15 smithi031 ceph-mon[31209]: Unable to set osd_memory_target on smithi178 to 184560947: error parsing value: Value '184560947' is below minimum 939524096 2024-09-21T03:00:16.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:15 smithi031 ceph-mon[31209]: Unable to set osd_memory_target on smithi165 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-21T03:00:16.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:15 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T03:00:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:16 smithi165 ceph-mon[34818]: pgmap v547: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:16 smithi178 ceph-mon[34759]: pgmap v547: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:17.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:16 smithi031 ceph-mon[31209]: pgmap v547: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:18 smithi165 ceph-mon[34818]: pgmap v548: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:18 smithi178 ceph-mon[34759]: pgmap v548: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:19.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:18 smithi031 ceph-mon[31209]: pgmap v548: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:20 smithi165 ceph-mon[34818]: pgmap v549: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:21.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:20 smithi178 ceph-mon[34759]: pgmap v549: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:21.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:20 smithi031 ceph-mon[31209]: pgmap v549: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:00:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:22 smithi165 ceph-mon[34818]: pgmap v550: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 195 B/s wr, 11 op/s 2024-09-21T03:00:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:22 smithi165 ceph-mon[34818]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:00:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:22 smithi178 ceph-mon[34759]: pgmap v550: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 195 B/s wr, 11 op/s 2024-09-21T03:00:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:22 smithi178 ceph-mon[34759]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:00:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:22 smithi031 ceph-mon[31209]: pgmap v550: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 195 B/s wr, 11 op/s 2024-09-21T03:00:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:22 smithi031 ceph-mon[31209]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:00:24.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:24 smithi165 ceph-mon[34818]: pgmap v552: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:00:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:24 smithi178 ceph-mon[34759]: pgmap v552: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:00:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:24 smithi031 ceph-mon[31209]: pgmap v552: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:00:26.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:26 smithi165 ceph-mon[34818]: pgmap v553: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:00:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:26 smithi178 ceph-mon[34759]: pgmap v553: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:00:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:26 smithi031 ceph-mon[31209]: pgmap v553: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:00:28.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:28 smithi165 ceph-mon[34818]: pgmap v554: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:28 smithi178 ceph-mon[34759]: pgmap v554: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:28 smithi031 ceph-mon[31209]: pgmap v554: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:30.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:30 smithi165 ceph-mon[34818]: pgmap v555: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:30 smithi178 ceph-mon[34759]: pgmap v555: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:30 smithi031 ceph-mon[31209]: pgmap v555: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:32.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:32 smithi165 ceph-mon[34818]: pgmap v556: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:32 smithi178 ceph-mon[34759]: pgmap v556: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:32 smithi031 ceph-mon[31209]: pgmap v556: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:34.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:34 smithi165 ceph-mon[34818]: pgmap v557: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:00:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:34 smithi178 ceph-mon[34759]: pgmap v557: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:00:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:34 smithi031 ceph-mon[31209]: pgmap v557: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:00:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:36 smithi178 ceph-mon[34759]: pgmap v558: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:36 smithi031 ceph-mon[31209]: pgmap v558: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:36 smithi165 ceph-mon[34818]: pgmap v558: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:38 smithi178 ceph-mon[34759]: pgmap v559: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:38 smithi031 ceph-mon[31209]: pgmap v559: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:38 smithi165 ceph-mon[34818]: pgmap v559: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:40 smithi178 ceph-mon[34759]: pgmap v560: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:40 smithi031 ceph-mon[31209]: pgmap v560: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:40 smithi165 ceph-mon[34818]: pgmap v560: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:42 smithi178 ceph-mon[34759]: pgmap v561: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:42 smithi031 ceph-mon[31209]: pgmap v561: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:42 smithi165 ceph-mon[34818]: pgmap v561: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:44.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:44 smithi178 ceph-mon[34759]: pgmap v562: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:00:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:44 smithi031 ceph-mon[31209]: pgmap v562: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:00:45.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:00:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:44 smithi165 ceph-mon[34818]: pgmap v562: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:00:45.401 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.952+0000 7f3f58372640 1 -- 172.21.15.31:0/1133508687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 msgr2=0x7f3f5010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.952+0000 7f3f58372640 1 --2- 172.21.15.31:0/1133508687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5010f550 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009ca0 tx=0x7f3f4802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.952+0000 7f3f58372640 1 -- 172.21.15.31:0/1133508687 shutdown_connections 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.952+0000 7f3f58372640 1 --2- 172.21.15.31:0/1133508687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f501083d0 0x7f3f501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.952+0000 7f3f58372640 1 --2- 172.21.15.31:0/1133508687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5010f550 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.952+0000 7f3f58372640 1 --2- 172.21.15.31:0/1133508687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f50102ae0 0x7f3f50102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.953+0000 7f3f58372640 1 -- 172.21.15.31:0/1133508687 >> 172.21.15.31:0/1133508687 conn(0x7f3f50079760 msgr2=0x7f3f50079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.953+0000 7f3f58372640 1 -- 172.21.15.31:0/1133508687 shutdown_connections 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.953+0000 7f3f58372640 1 -- 172.21.15.31:0/1133508687 wait complete. 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.953+0000 7f3f58372640 1 Processor -- start 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.953+0000 7f3f58372640 1 -- start start 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f58372640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f50102ae0 0x7f3f50199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f58372640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f501083d0 0x7f3f501998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f58372640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f58372640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50112990 con 0x7f3f50108d20 2024-09-21T03:00:45.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f58372640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50112810 con 0x7f3f50102ae0 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f58372640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50112b10 con 0x7f3f501083d0 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43030/0 (socket says 172.21.15.31:43030) 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 -- 172.21.15.31:0/2279794083 learned_addr learned my addr 172.21.15.31:0/2279794083 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f501083d0 0x7f3f501998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5610e640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f50102ae0 0x7f3f50199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 -- 172.21.15.31:0/2279794083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f501083d0 msgr2=0x7f3f501998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f501083d0 0x7f3f501998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 -- 172.21.15.31:0/2279794083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f50102ae0 msgr2=0x7f3f50199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f50102ae0 0x7f3f50199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 -- 172.21.15.31:0/2279794083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5019f270 con 0x7f3f50108d20 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.954+0000 7f3f5690f640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5019eb40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3f480098d0 tx=0x7f3f480317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.955+0000 7f3f477fe640 1 -- 172.21.15.31:0/2279794083 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48003b70 con 0x7f3f50108d20 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.955+0000 7f3f477fe640 1 -- 172.21.15.31:0/2279794083 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f48003d10 con 0x7f3f50108d20 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.955+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5019f500 con 0x7f3f50108d20 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.955+0000 7f3f477fe640 1 -- 172.21.15.31:0/2279794083 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4802fa20 con 0x7f3f50108d20 2024-09-21T03:00:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.955+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f501a5cc0 con 0x7f3f50108d20 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.957+0000 7f3f5e8e6740 1 Processor -- start 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f477fe640 1 -- 172.21.15.31:0/2279794083 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f3f4802fc40 con 0x7f3f50108d20 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 -- start start 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0002f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0002f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b0002f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b00026de50 con 0x55affff2f390 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b00025d280 con 0x55affff50070 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.958+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b000261090 con 0x55affff271b0 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b0002f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b0002f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43042/0 (socket says 172.21.15.31:43042) 2024-09-21T03:00:45.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 -- 172.21.15.31:0/2194845389 learned_addr learned my addr 172.21.15.31:0/2194845389 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5610e640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0002f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0002f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f477fe640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3f3c0b6680 0x7f3f3c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 -- 172.21.15.31:0/2194845389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 msgr2=0x55b0002f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0002f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 -- 172.21.15.31:0/2194845389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 msgr2=0x55b0002f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0002f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 -- 172.21.15.31:0/2194845389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55affffbb650 con 0x55affff2f390 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5690f640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3f3c0b6680 0x7f3f3c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b0002f80d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3f40012770 tx=0x7f3f40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8500a11ac4a17521 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.959+0000 7f3f477fe640 1 -- 172.21.15.31:0/2279794083 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f3f48108a30 con 0x7f3f50108d20 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.960+0000 7f3f44d9a640 1 -- 172.21.15.31:0/2194845389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40012cf0 con 0x55affff2f390 2024-09-21T03:00:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.960+0000 7f3f44d9a640 1 -- 172.21.15.31:0/2194845389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f40010770 con 0x55affff2f390 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.960+0000 7f3f44d9a640 1 -- 172.21.15.31:0/2194845389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40003850 con 0x55affff2f390 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.960+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2194845389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 msgr2=0x55b0002f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.960+0000 7f3f5690f640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3f3c0b6680 0x7f3f3c0b8b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f48092e40 tx=0x7f3f4809b640 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.960+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b0002f80d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3f40012770 tx=0x7f3f40004730 comp rx=0 tx=0).stop 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2194845389 shutdown_connections 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0002f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b0002f80d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2194845389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0002f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2194845389 >> 172.21.15.31:0/2194845389 conn(0x55affff44460 msgr2=0x55b0002f5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2194845389 shutdown_connections 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2194845389 wait complete. 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 -- start start 2024-09-21T03:00:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 Processor -- start 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 0x55b000304f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff2f390 0x55b000305440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.961+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0003112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b0000be700 con 0x55affff271b0 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b0001f8110 con 0x55affff50070 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b000307f90 con 0x55affff2f390 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5e8e6740 10 client.0 _ll_register_callbacks cb 0x55b000211990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 0x55b000304f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 0x55b000304f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43046/0 (socket says 172.21.15.31:43046) 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 -- 172.21.15.31:0/4272522249 learned_addr learned my addr 172.21.15.31:0/4272522249 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5610e640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff2f390 0x55b000305440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 -- 172.21.15.31:0/4272522249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff2f390 msgr2=0x55b000305440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff2f390 0x55b000305440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 -- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 msgr2=0x55b0003112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0003112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 -- 172.21.15.31:0/4272522249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b000225c40 con 0x55affff271b0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5610e640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff2f390 0x55b000305440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5590d640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 0x55b000304f00 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3f400058c0 tx=0x7f3f400058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6a53806b7a44c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f39ffb640 1 -- 172.21.15.31:0/4272522249 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40025030 con 0x55affff271b0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f40010770 con 0x55affff271b0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.962+0000 7f3f39ffb640 1 -- 172.21.15.31:0/4272522249 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f3f40004040 con 0x55affff271b0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f39ffb640 1 -- 172.21.15.31:0/4272522249 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f40031050 con 0x55affff271b0 2024-09-21T03:00:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 10 client.17343 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b000313cd0 con 0x55affff271b0 2024-09-21T03:00:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f39ffb640 1 -- 172.21.15.31:0/4272522249 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f40010770 con 0x55affff271b0 2024-09-21T03:00:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f39ffb640 1 -- 172.21.15.31:0/4272522249 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f400048c0 con 0x55affff271b0 2024-09-21T03:00:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 10 client.17343 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 20 client.17343 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55affff0e9f0 0x55b000317030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 4 client.17343 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.963+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b00026fda0 con 0x55affff0e9f0 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.964+0000 7f3f5610e640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55affff0e9f0 0x55b000317030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.964+0000 7f3f39ffb640 10 client.17343 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.964+0000 7f3f5610e640 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55affff0e9f0 0x55b000317030 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0b938425d9498d server_cookie=dd0c185d160979 in_seq=0 out_seq=0 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f39ffb640 1 -- 172.21.15.31:0/4272522249 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b00026fda0 con 0x55affff0e9f0 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f39ffb640 10 client.17343 handle_command_reply: tid=0 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 client.17343 shutdown 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 msgr2=0x55b000304f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 0x55b000304f00 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3f400058c0 tx=0x7f3f400058f0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 shutdown_connections 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55affff0e9f0 0x55b000317030 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff2f390 0x55b000305440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff50070 0x55b0003112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/4272522249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff271b0 0x55b000304f00 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 >> 172.21.15.31:0/4272522249 conn(0x55affff44460 msgr2=0x55b0002f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f39ffb640 0 client.17343 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 shutdown_connections 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.965+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/4272522249 wait complete. 2024-09-21T03:00:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.966+0000 7f3f5e8e6740 20 client.17343 trim_cache size 0 max 16384 2024-09-21T03:00:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 Processor -- start 2024-09-21T03:00:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 -- start start 2024-09-21T03:00:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0003053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff2f390 0x55b000311260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 0x55b0002f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501ad4f0 con 0x55affff50070 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501af030 con 0x55affff2f390 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501adbc0 con 0x55affff271b0 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 0x55b0002f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5590d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 0x55b0002f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43060/0 (socket says 172.21.15.31:43060) 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5590d640 1 -- 172.21.15.31:0/2985538329 learned_addr learned my addr 172.21.15.31:0/2985538329 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0003053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.983+0000 7f3f5610e640 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff2f390 0x55b000311260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5590d640 1 -- 172.21.15.31:0/2985538329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 msgr2=0x55b0003053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0003053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5590d640 1 -- 172.21.15.31:0/2985538329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff2f390 msgr2=0x55b000311260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff2f390 0x55b000311260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5590d640 1 -- 172.21.15.31:0/2985538329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0002fd7b0 con 0x55affff50070 2024-09-21T03:00:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5610e640 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff2f390 0x55b000311260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f5590d640 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 0x55b0002f4ea0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000bb20 tx=0x7f3f4000b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a4087f79c77526 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f3bfff640 1 -- 172.21.15.31:0/2985538329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40025040 con 0x55affff50070 2024-09-21T03:00:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f3bfff640 1 -- 172.21.15.31:0/2985538329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f40016040 con 0x55affff50070 2024-09-21T03:00:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.984+0000 7f3f3bfff640 1 -- 172.21.15.31:0/2985538329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40005360 con 0x55affff50070 2024-09-21T03:00:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2985538329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 msgr2=0x55b0002f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 0x55b0002f4ea0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000bb20 tx=0x7f3f4000b590 comp rx=0 tx=0).stop 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2985538329 shutdown_connections 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff50070 0x55b0002f4ea0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff2f390 0x55b000311260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/2985538329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff271b0 0x55b0003053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2985538329 >> 172.21.15.31:0/2985538329 conn(0x55affff44460 msgr2=0x55b000304cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2985538329 shutdown_connections 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/2985538329 wait complete. 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 -- start start 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.985+0000 7f3f5e8e6740 1 Processor -- start 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff271b0 0x55b00031a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b000317c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff50070 0x55b000318160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501ae960 con 0x55affff2f390 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501ae290 con 0x55affff271b0 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501ab4d0 con 0x55affff50070 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5e8e6740 10 client.0 _ll_register_callbacks cb 0x7f3f4000be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b000317c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b000317c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43062/0 (socket says 172.21.15.31:43062) 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 -- 172.21.15.31:0/1498959851 learned_addr learned my addr 172.21.15.31:0/1498959851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5590d640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff271b0 0x55b00031a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5590d640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff50070 0x55b000318160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 -- 172.21.15.31:0/1498959851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff50070 msgr2=0x55b000318160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff50070 0x55b000318160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 -- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff271b0 msgr2=0x55b00031a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff271b0 0x55b00031a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 -- 172.21.15.31:0/1498959851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b000318800 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f5610e640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b000317c20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009ba0 tx=0x7f3f4c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394262ab85af061b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.986+0000 7f3f39ffb640 1 -- 172.21.15.31:0/1498959851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c02a060 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f39ffb640 1 -- 172.21.15.31:0/1498959851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f3f4c0035f0 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f40016040 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f39ffb640 1 -- 172.21.15.31:0/1498959851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c0037f0 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 10 client.17349 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0003815e0 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f39ffb640 1 -- 172.21.15.31:0/1498959851 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f40016040 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f39ffb640 1 -- 172.21.15.31:0/1498959851 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f4c025a30 con 0x55affff2f390 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 10 client.17349 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 20 client.17349 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afffee3da0 0x55b000383d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 4 client.17349 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.987+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b00026b850 con 0x55afffee3da0 2024-09-21T03:00:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.988+0000 7f3f5590d640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afffee3da0 0x55b000383d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.988+0000 7f3f5590d640 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afffee3da0 0x55b000383d80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2673bf35584e0f0f server_cookie=15aa3d07e0711430 in_seq=0 out_seq=0 2024-09-21T03:00:45.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.988+0000 7f3f39ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.988+0000 7f3f39ffb640 1 -- 172.21.15.31:0/1498959851 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55b00026b850 con 0x55afffee3da0 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.988+0000 7f3f39ffb640 10 client.17349 handle_command_reply: tid=0 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 client.17349 shutdown 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 msgr2=0x55b000317c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b000317c20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009ba0 tx=0x7f3f4c012820 comp rx=0 tx=0).stop 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 shutdown_connections 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55afffee3da0 0x55b000383d80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55affff50070 0x55b000318160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55affff2f390 0x55b000317c20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f39ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 --2- 172.21.15.31:0/1498959851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55affff271b0 0x55b00031a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 >> 172.21.15.31:0/1498959851 conn(0x55affff44460 msgr2=0x55b0002f4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 shutdown_connections 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 1 -- 172.21.15.31:0/1498959851 wait complete. 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f5e8e6740 20 client.17349 trim_cache size 0 max 16384 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:00:45.995 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:00:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:45.999 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:46.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:46.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:46.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:46.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:46.003 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:00:46.003 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:00:46.003 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:46.003 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:46.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:46.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:46.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:46.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:46.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:46.006 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:00:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3f3c0b6680 msgr2=0x7f3f3c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f58372640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3f3c0b6680 0x7f3f3c0b8b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f48092e40 tx=0x7f3f4809b640 comp rx=0 tx=0).stop 2024-09-21T03:00:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 msgr2=0x7f3f5019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f58372640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5019eb40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3f480098d0 tx=0x7f3f480317b0 comp rx=0 tx=0).stop 2024-09-21T03:00:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.989+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 shutdown_connections 2024-09-21T03:00:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f3f3c0b6680 0x7f3f3c0b8b90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f501083d0 0x7f3f501998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f50108d20 0x7f3f5019eb40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 --2- 172.21.15.31:0/2279794083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f50102ae0 0x7f3f50199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 >> 172.21.15.31:0/2279794083 conn(0x7f3f50079760 msgr2=0x7f3f501043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 shutdown_connections 2024-09-21T03:00:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:45.990+0000 7f3f58372640 1 -- 172.21.15.31:0/2279794083 wait complete. 2024-09-21T03:00:46.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:46 smithi031 ceph-mon[31209]: pgmap v563: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:46.850 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:00:46.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:00:46.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:46 smithi178 ceph-mon[34759]: pgmap v563: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:47.109 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:46 smithi165 ceph-mon[34818]: pgmap v563: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:47.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 -- 172.21.15.31:0/1190347743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 msgr2=0x7ff5b01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/1190347743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b01098b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009900 tx=0x7ff5a002a410 comp rx=0 tx=0).stop 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 -- 172.21.15.31:0/1190347743 shutdown_connections 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/1190347743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0109df0 0x7ff5b0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/1190347743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b01098b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/1190347743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5b0103b60 0x7ff5b0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 -- 172.21.15.31:0/1190347743 >> 172.21.15.31:0/1190347743 conn(0x7ff5b007b850 msgr2=0x7ff5b007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 -- 172.21.15.31:0/1190347743 shutdown_connections 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 -- 172.21.15.31:0/1190347743 wait complete. 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 Processor -- start 2024-09-21T03:00:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.583+0000 7ff5b6c70640 1 -- start start 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b6c70640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5b0103b60 0x7ff5b019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b6c70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b6c70640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0109df0 0x7ff5b019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b6c70640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113ab0 con 0x7ff5b01094a0 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b6c70640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113930 con 0x7ff5b0109df0 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b6c70640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113c30 con 0x7ff5b0103b60 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43080/0 (socket says 172.21.15.31:43080) 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 -- 172.21.15.31:0/2877657459 learned_addr learned my addr 172.21.15.31:0/2877657459 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b4a0c640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5b0103b60 0x7ff5b019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5b520d640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0109df0 0x7ff5b019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 -- 172.21.15.31:0/2877657459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5b0103b60 msgr2=0x7ff5b019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:47.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5b0103b60 0x7ff5b019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 -- 172.21.15.31:0/2877657459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0109df0 msgr2=0x7ff5b019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0109df0 0x7ff5b019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 -- 172.21.15.31:0/2877657459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b01a04f0 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.584+0000 7ff5a7fff640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b019ab20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff5a002a930 tx=0x7ff5a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.585+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a003f030 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.585+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a0003aa0 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.585+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b01a0780 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.585+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0035070 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.585+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a6f40 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.586+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0104de0 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.587+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff5a0042030 con 0x7ff5b01094a0 2024-09-21T03:00:47.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.588+0000 7ff5a5ffb640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5980b6680 0x7ff5980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:47.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.588+0000 7ff5b4a0c640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5980b6680 0x7ff5980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:47.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.588+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7ff5a0108100 con 0x7ff5b01094a0 2024-09-21T03:00:47.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.589+0000 7ff5b4a0c640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5980b6680 0x7ff5980b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005fd0 tx=0x7ff5ac00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:47.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.592+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5a00d2a30 con 0x7ff5b01094a0 2024-09-21T03:00:47.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.821+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5b010bbc0 con 0x7ff5b01094a0 2024-09-21T03:00:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.822+0000 7ff5a5ffb640 1 -- 172.21.15.31:0/2877657459 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7ff5a00d7970 con 0x7ff5b01094a0 2024-09-21T03:00:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:00:47.823 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:00:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:00:47.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.824+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5980b6680 msgr2=0x7ff5980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:47.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.824+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5980b6680 0x7ff5980b8b90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005fd0 tx=0x7ff5ac00b390 comp rx=0 tx=0).stop 2024-09-21T03:00:47.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.824+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 msgr2=0x7ff5b019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:47.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.824+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b019ab20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff5a002a930 tx=0x7ff5a0004270 comp rx=0 tx=0).stop 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.824+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 shutdown_connections 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff5980b6680 0x7ff5980b8b90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5b0109df0 0x7ff5b019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5b01094a0 0x7ff5b019ab20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 --2- 172.21.15.31:0/2877657459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5b0103b60 0x7ff5b019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 >> 172.21.15.31:0/2877657459 conn(0x7ff5b007b850 msgr2=0x7ff5b0105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 shutdown_connections 2024-09-21T03:00:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:47.825+0000 7ff5b6c70640 1 -- 172.21.15.31:0/2877657459 wait complete. 2024-09-21T03:00:48.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:00:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:49.046 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:48 smithi031 ceph-mon[31209]: pgmap v564: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:49.046 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2877657459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:48 smithi165 ceph-mon[34818]: pgmap v564: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2877657459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:48 smithi178 ceph-mon[34759]: pgmap v564: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2877657459' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 -- 172.21.15.31:0/3148230559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c103b70 msgr2=0x7f620c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 --2- 172.21.15.31:0/3148230559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c103b70 0x7f620c103f80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 -- 172.21.15.31:0/3148230559 shutdown_connections 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 --2- 172.21.15.31:0/3148230559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f620c109e00 0x7f620c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 --2- 172.21.15.31:0/3148230559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c1094b0 0x7f620c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 --2- 172.21.15.31:0/3148230559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c103b70 0x7f620c103f80 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 -- 172.21.15.31:0/3148230559 >> 172.21.15.31:0/3148230559 conn(0x7f620c0ff5e0 msgr2=0x7f620c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.410+0000 7f62134ed640 1 -- 172.21.15.31:0/3148230559 shutdown_connections 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.411+0000 7f62134ed640 1 -- 172.21.15.31:0/3148230559 wait complete. 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.411+0000 7f62134ed640 1 Processor -- start 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.411+0000 7f62134ed640 1 -- start start 2024-09-21T03:00:49.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c103b70 0x7f620c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 0x7f620c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f620c109e00 0x7f620c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f620c113ac0 con 0x7f620c1094b0 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f620c113940 con 0x7f620c103b70 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f620c113c40 con 0x7f620c109e00 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 0x7f620c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 0x7f620c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54038/0 (socket says 172.21.15.31:54038) 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 -- 172.21.15.31:0/2406479675 learned_addr learned my addr 172.21.15.31:0/2406479675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6211a8a640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f620c109e00 0x7f620c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6211289640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c103b70 0x7f620c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 -- 172.21.15.31:0/2406479675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f620c109e00 msgr2=0x7f620c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f620c109e00 0x7f620c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 -- 172.21.15.31:0/2406479675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c103b70 msgr2=0x7f620c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c103b70 0x7f620c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 -- 172.21.15.31:0/2406479675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c1a0420 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6211289640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c103b70 0x7f620c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f6210a88640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 0x7f620c19aa50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00d950 tx=0x7f61fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62027fc640 1 -- 172.21.15.31:0/2406479675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc017070 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62027fc640 1 -- 172.21.15.31:0/2406479675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61fc009c30 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.412+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c1a0710 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.413+0000 7f62027fc640 1 -- 172.21.15.31:0/2406479675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0047a0 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.413+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c1a6e70 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 Processor -- start 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 -- start start 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f62027fc640 1 -- 172.21.15.31:0/2406479675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f61fc004940 con 0x7f620c1094b0 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db5bdacea0 con 0x55db5ba661b0 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55db5bd9c2d0 con 0x55db5ba8f070 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6219a61740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db5bda00e0 con 0x55db5ba6e390 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be36be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54050/0 (socket says 172.21.15.31:54050) 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.416+0000 7f6211289640 1 -- 172.21.15.31:0/3706330937 learned_addr learned my addr 172.21.15.31:0/3706330937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6210a88640 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211a8a640 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211289640 1 -- 172.21.15.31:0/3706330937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 msgr2=0x55db5be37120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211289640 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be37120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211289640 1 -- 172.21.15.31:0/3706330937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 msgr2=0x55db5be366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211289640 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211289640 1 -- 172.21.15.31:0/3706330937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5bafa650 con 0x55db5ba661b0 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f62027fc640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61f80b6700 0x7f61f80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211289640 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be36be0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f620802a930 tx=0x7f6208032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91232076edc6a7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f6211a8a640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61f80b6700 0x7f61f80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f61f77fe640 1 -- 172.21.15.31:0/3706330937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620803f070 con 0x55db5ba661b0 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f62027fc640 1 -- 172.21.15.31:0/2406479675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f61fc0e7410 con 0x7f620c1094b0 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f61f77fe640 1 -- 172.21.15.31:0/3706330937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62080316a0 con 0x55db5ba661b0 2024-09-21T03:00:49.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.417+0000 7f61f77fe640 1 -- 172.21.15.31:0/3706330937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208035070 con 0x55db5ba661b0 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 -- 172.21.15.31:0/3706330937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 msgr2=0x55db5be36be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6211a8a640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61f80b6700 0x7f61f80b8c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6204005fd0 tx=0x7f6204009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be36be0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f620802a930 tx=0x7f6208032c30 comp rx=0 tx=0).stop 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 -- 172.21.15.31:0/3706330937 shutdown_connections 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be37120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be36be0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 --2- 172.21.15.31:0/3706330937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.418+0000 7f6219a61740 1 -- 172.21.15.31:0/3706330937 >> 172.21.15.31:0/3706330937 conn(0x55db5ba83460 msgr2=0x55db5be34590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 -- 172.21.15.31:0/3706330937 shutdown_connections 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 -- 172.21.15.31:0/3706330937 wait complete. 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 -- start start 2024-09-21T03:00:49.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 Processor -- start 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be42280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.419+0000 7f6219a61740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6219a61740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55db5bbfd700 con 0x55db5ba661b0 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6219a61740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55db5bd37160 con 0x55db5ba8f070 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6219a61740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db5be44e20 con 0x55db5ba6e390 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6219a61740 10 client.0 _ll_register_callbacks cb 0x55db5bd509e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be41d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54054/0 (socket says 172.21.15.31:54054) 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 -- 172.21.15.31:0/1125425619 learned_addr learned my addr 172.21.15.31:0/1125425619 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6210a88640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be42280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211a8a640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 -- 172.21.15.31:0/1125425619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 msgr2=0x55db5be42280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be42280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 -- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 msgr2=0x55db5be42800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be42800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 -- 172.21.15.31:0/1125425619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5bd64c90 con 0x55db5ba661b0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6211289640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be41d40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6208005760 tx=0x7f6208005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb550ea0ad2e8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f61f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f61f4ff9640 1 -- 172.21.15.31:0/1125425619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620803f070 con 0x55db5ba661b0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f61f4ff9640 1 -- 172.21.15.31:0/1125425619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f6208004250 con 0x55db5ba661b0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.420+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62080316a0 con 0x55db5ba661b0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f61f4ff9640 1 -- 172.21.15.31:0/1125425619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6208054050 con 0x55db5ba661b0 2024-09-21T03:00:49.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 10 client.17367 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db5be50af0 con 0x55db5ba661b0 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f61f4ff9640 1 -- 172.21.15.31:0/1125425619 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62080316a0 con 0x55db5ba661b0 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f61f4ff9640 1 -- 172.21.15.31:0/1125425619 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6208004750 con 0x55db5ba661b0 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 10 client.17367 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 10 client.17367 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 20 client.17367 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba4d9f0 0x55db5be53e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 4 client.17367 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.421+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db5bdaedf0 con 0x55db5ba4d9f0 2024-09-21T03:00:49.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.422+0000 7f6210a88640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba4d9f0 0x55db5be53e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.422+0000 7f61f4ff9640 10 client.17367 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.422+0000 7f6210a88640 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba4d9f0 0x55db5be53e50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2543ff049127ae53 server_cookie=3a4d61d1736d3a29 in_seq=0 out_seq=0 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f61f4ff9640 1 -- 172.21.15.31:0/1125425619 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db5bdaedf0 con 0x55db5ba4d9f0 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f61f4ff9640 10 client.17367 handle_command_reply: tid=0 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 client.17367 shutdown 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 msgr2=0x55db5be41d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be41d40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6208005760 tx=0x7f6208005d50 comp rx=0 tx=0).stop 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 shutdown_connections 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba4d9f0 0x55db5be53e50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f61f4ff9640 0 client.17367 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba6e390 0x55db5be42280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba8f070 0x55db5be42800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 --2- 172.21.15.31:0/1125425619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba661b0 0x55db5be41d40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 >> 172.21.15.31:0/1125425619 conn(0x55db5ba83460 msgr2=0x55db5be39830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:49.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 shutdown_connections 2024-09-21T03:00:49.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 1 -- 172.21.15.31:0/1125425619 wait complete. 2024-09-21T03:00:49.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.423+0000 7f6219a61740 20 client.17367 trim_cache size 0 max 16384 2024-09-21T03:00:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 Processor -- start 2024-09-21T03:00:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 -- start start 2024-09-21T03:00:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be41af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be42030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be3ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1af7c0 con 0x55db5ba4d9f0 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1b1300 con 0x55db5ba6e390 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6219a61740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1afe90 con 0x55db5ba661b0 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be41af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be41af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54058/0 (socket says 172.21.15.31:54058) 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 -- 172.21.15.31:0/3640917623 learned_addr learned my addr 172.21.15.31:0/3640917623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6210a88640 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be42030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211a8a640 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be3ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 -- 172.21.15.31:0/3640917623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 msgr2=0x55db5be42030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be42030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 -- 172.21.15.31:0/3640917623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 msgr2=0x55db5be3ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be3ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.437+0000 7f6211289640 1 -- 172.21.15.31:0/3640917623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5be44480 con 0x55db5ba4d9f0 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.438+0000 7f6211289640 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be41af0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6208031440 tx=0x7f620802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b31e76885d80d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.438+0000 7f61f6ffd640 1 -- 172.21.15.31:0/3640917623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620805d070 con 0x55db5ba4d9f0 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.438+0000 7f61f6ffd640 1 -- 172.21.15.31:0/3640917623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620804b070 con 0x55db5ba4d9f0 2024-09-21T03:00:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.438+0000 7f61f6ffd640 1 -- 172.21.15.31:0/3640917623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620803f070 con 0x55db5ba4d9f0 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 -- 172.21.15.31:0/3640917623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 msgr2=0x55db5be41af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be41af0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6208031440 tx=0x7f620802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 -- 172.21.15.31:0/3640917623 shutdown_connections 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be42030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be3ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 --2- 172.21.15.31:0/3640917623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be41af0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 -- 172.21.15.31:0/3640917623 >> 172.21.15.31:0/3640917623 conn(0x55db5ba83460 msgr2=0x55db5be405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 -- 172.21.15.31:0/3640917623 shutdown_connections 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 -- 172.21.15.31:0/3640917623 wait complete. 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 -- start start 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.439+0000 7f6219a61740 1 Processor -- start 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be55350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be55890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1b0c30 con 0x55db5ba4d9f0 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1b0560 con 0x55db5ba6e390 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1ad7a0 con 0x55db5ba661b0 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6219a61740 10 client.0 _ll_register_callbacks cb 0x7f6204009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be54e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54060/0 (socket says 172.21.15.31:54060) 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 -- 172.21.15.31:0/3414645527 learned_addr learned my addr 172.21.15.31:0/3414645527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211a8a640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be55890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6210a88640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be55350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 -- 172.21.15.31:0/3414645527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 msgr2=0x55db5be55350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be55350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 -- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 msgr2=0x55db5be55890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be55890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 -- 172.21.15.31:0/3414645527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5bebd5d0 con 0x55db5ba4d9f0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6210a88640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f6211289640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be54e10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f62080059d0 tx=0x7f6208004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfadc34c9338932 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f61f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.440+0000 7f61f4ff9640 1 -- 172.21.15.31:0/3414645527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620805d040 con 0x55db5ba4d9f0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f61f4ff9640 1 -- 172.21.15.31:0/3414645527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f62080046c0 con 0x55db5ba4d9f0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620804b070 con 0x55db5ba4d9f0 2024-09-21T03:00:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f61f4ff9640 1 -- 172.21.15.31:0/3414645527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f620803f040 con 0x55db5ba4d9f0 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 10 client.17373 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db5bebe7e0 con 0x55db5ba4d9f0 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f61f4ff9640 1 -- 172.21.15.31:0/3414645527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620804b070 con 0x55db5ba4d9f0 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f61f4ff9640 1 -- 172.21.15.31:0/3414645527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6208060070 con 0x55db5ba4d9f0 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 10 client.17373 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 10 client.17373 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 20 client.17373 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba8f070 0x55db5bec0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 4 client.17373 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.441+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db5bdaa8a0 con 0x55db5ba8f070 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6210a88640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba8f070 0x55db5bec0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f61f4ff9640 10 client.17373 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6210a88640 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba8f070 0x55db5bec0e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f11d6755be6e275 server_cookie=b2d88c240251b30c in_seq=0 out_seq=0 2024-09-21T03:00:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f61f4ff9640 1 -- 172.21.15.31:0/3414645527 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db5bdaa8a0 con 0x55db5ba8f070 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f61f4ff9640 10 client.17373 handle_command_reply: tid=0 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6219a61740 1 client.17373 shutdown 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 msgr2=0x55db5be54e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6219a61740 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be54e10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f62080059d0 tx=0x7f6208004e20 comp rx=0 tx=0).stop 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 shutdown_connections 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.442+0000 7f6219a61740 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x55db5ba8f070 0x55db5bec0e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db5ba661b0 0x55db5be55350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db5ba6e390 0x55db5be55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 1 --2- 172.21.15.31:0/3414645527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55db5ba4d9f0 0x55db5be54e10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 >> 172.21.15.31:0/3414645527 conn(0x55db5ba83460 msgr2=0x55db5be404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f61f4ff9640 0 client.17373 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 shutdown_connections 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 1 -- 172.21.15.31:0/3414645527 wait complete. 2024-09-21T03:00:49.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f6219a61740 20 client.17373 trim_cache size 0 max 16384 2024-09-21T03:00:49.450 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61f80b6700 msgr2=0x7f61f80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61f80b6700 0x7f61f80b8c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6204005fd0 tx=0x7f6204009500 comp rx=0 tx=0).stop 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 msgr2=0x7f620c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 0x7f620c19aa50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00d950 tx=0x7f61fc00de10 comp rx=0 tx=0).stop 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 shutdown_connections 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f61f80b6700 0x7f61f80b8c10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f620c109e00 0x7f620c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f620c1094b0 0x7f620c19aa50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 --2- 172.21.15.31:0/2406479675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f620c103b70 0x7f620c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.443+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 >> 172.21.15.31:0/2406479675 conn(0x7f620c0ff5e0 msgr2=0x7f620c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.444+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 shutdown_connections 2024-09-21T03:00:49.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:49.444+0000 7f62134ed640 1 -- 172.21.15.31:0/2406479675 wait complete. 2024-09-21T03:00:50.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:00:50.489 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:50.917 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:50 smithi031 ceph-mon[31209]: pgmap v565: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.010+0000 7fd01ad4c640 1 -- 172.21.15.31:0/2086955463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 msgr2=0x7fd01410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.010+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/2086955463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01410d8b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd004009ca0 tx=0x7fd00402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 -- 172.21.15.31:0/2086955463 shutdown_connections 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/2086955463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01410ddf0 0x7fd014110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/2086955463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01410d8b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/2086955463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd014104840 0x7fd014104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 -- 172.21.15.31:0/2086955463 >> 172.21.15.31:0/2086955463 conn(0x7fd014079730 msgr2=0x7fd014079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 -- 172.21.15.31:0/2086955463 shutdown_connections 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 -- 172.21.15.31:0/2086955463 wait complete. 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 Processor -- start 2024-09-21T03:00:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.011+0000 7fd01ad4c640 1 -- start start 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd01ad4c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd014104840 0x7fd01419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd01ad4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd01ad4c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01410ddf0 0x7fd01419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd01ad4c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd014113a70 con 0x7fd014105270 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd01ad4c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141138f0 con 0x7fd014104840 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd01ad4c640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd014113bf0 con 0x7fd01410ddf0 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54084/0 (socket says 172.21.15.31:54084) 2024-09-21T03:00:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 -- 172.21.15.31:0/863090909 learned_addr learned my addr 172.21.15.31:0/863090909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd0192e9640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01410ddf0 0x7fd01419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd018ae8640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd014104840 0x7fd01419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 -- 172.21.15.31:0/863090909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01410ddf0 msgr2=0x7fd01419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01410ddf0 0x7fd01419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 -- 172.21.15.31:0/863090909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd014104840 msgr2=0x7fd01419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd014104840 0x7fd01419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 -- 172.21.15.31:0/863090909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0141a03d0 con 0x7fd014105270 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.012+0000 7fd013fff640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01419aa00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd0040319f0 tx=0x7fd004031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.013+0000 7fd011ffb640 1 -- 172.21.15.31:0/863090909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004003bf0 con 0x7fd014105270 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.013+0000 7fd011ffb640 1 -- 172.21.15.31:0/863090909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd004003d90 con 0x7fd014105270 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.013+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0141a0660 con 0x7fd014105270 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.013+0000 7fd011ffb640 1 -- 172.21.15.31:0/863090909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00402fb70 con 0x7fd014105270 2024-09-21T03:00:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.013+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0141a6e20 con 0x7fd014105270 2024-09-21T03:00:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 Processor -- start 2024-09-21T03:00:51.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 -- start start 2024-09-21T03:00:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddb67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5609ddadcea0 con 0x5609dd7961b0 2024-09-21T03:00:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5609ddacc2d0 con 0x5609dd7bf070 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.016+0000 7fd0212c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5609ddad00e0 con 0x5609dd79e390 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb66be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54090/0 (socket says 172.21.15.31:54090) 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 -- 172.21.15.31:0/194289485 learned_addr learned my addr 172.21.15.31:0/194289485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd011ffb640 1 -- 172.21.15.31:0/863090909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd00402fd10 con 0x7fd014105270 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd0192e9640 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd013fff640 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddb67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 -- 172.21.15.31:0/194289485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 msgr2=0x5609ddb67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddb67120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 -- 172.21.15.31:0/194289485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 msgr2=0x5609ddb666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 -- 172.21.15.31:0/194289485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dd82a650 con 0x5609dd7961b0 2024-09-21T03:00:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd0192e9640 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd018ae8640 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb66be0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00bc90 tx=0x7fcffc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1050a6a202638 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.017+0000 7fd00affd640 1 -- 172.21.15.31:0/194289485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc015070 con 0x5609dd7961b0 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd00affd640 1 -- 172.21.15.31:0/194289485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcffc0035f0 con 0x5609dd7961b0 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd00affd640 1 -- 172.21.15.31:0/194289485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0116a0 con 0x5609dd7961b0 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd011ffb640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0000b6680 0x7fd0000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd0192e9640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0000b6680 0x7fd0000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd011ffb640 1 -- 172.21.15.31:0/863090909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fd004108ba0 con 0x7fd014105270 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd0212c0740 1 -- 172.21.15.31:0/194289485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 msgr2=0x5609ddb66be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd0192e9640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0000b6680 0x7fd0000b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00be70 tx=0x7fd00c00b450 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.018+0000 7fd0212c0740 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb66be0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00bc90 tx=0x7fcffc00f730 comp rx=0 tx=0).stop 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 -- 172.21.15.31:0/194289485 shutdown_connections 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddb67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb66be0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 --2- 172.21.15.31:0/194289485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 -- 172.21.15.31:0/194289485 >> 172.21.15.31:0/194289485 conn(0x5609dd7b3460 msgr2=0x5609ddb64590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 -- 172.21.15.31:0/194289485 shutdown_connections 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 -- 172.21.15.31:0/194289485 wait complete. 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 -- start start 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.019+0000 7fd0212c0740 1 Processor -- start 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd7961b0 0x5609ddb71d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 0x5609ddb722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb7e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5609dd92d700 con 0x5609dd79e390 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5609dda67160 con 0x5609dd7bf070 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5609ddb74e80 con 0x5609dd7961b0 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0212c0740 10 client.0 _ll_register_callbacks cb 0x5609dda809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 0x5609ddb722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 0x5609ddb722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54104/0 (socket says 172.21.15.31:54104) 2024-09-21T03:00:51.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 -- 172.21.15.31:0/240698940 learned_addr learned my addr 172.21.15.31:0/240698940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd018ae8640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd7961b0 0x5609ddb71d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0192e9640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb7e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 -- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd7961b0 msgr2=0x5609ddb71d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd7961b0 0x5609ddb71d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 -- 172.21.15.31:0/240698940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 msgr2=0x5609ddb7e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb7e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 -- 172.21.15.31:0/240698940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dda94c90 con 0x5609dd79e390 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd0192e9640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb7e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fd013fff640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 0x5609ddb722b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd0040ce940 tx=0x7fd004030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7c0490462fa431 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fcff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fcff3fff640 1 -- 172.21.15.31:0/240698940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040ce300 con 0x5609dd79e390 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.020+0000 7fcff3fff640 1 -- 172.21.15.31:0/240698940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fd0040cdc10 con 0x5609dd79e390 2024-09-21T03:00:51.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcffc0035f0 con 0x5609dd79e390 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fcff3fff640 1 -- 172.21.15.31:0/240698940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd0040f2070 con 0x5609dd79e390 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 10 client.17385 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609ddb84f90 con 0x5609dd79e390 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fcff3fff640 1 -- 172.21.15.31:0/240698940 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcffc0035f0 con 0x5609dd79e390 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fcff3fff640 1 -- 172.21.15.31:0/240698940 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd0040cfce0 con 0x5609dd79e390 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 10 client.17385 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 10 client.17385 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 20 client.17385 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd77d9f0 0x5609ddb88510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 4 client.17385 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609ddadedf0 con 0x5609dd77d9f0 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.021+0000 7fd018ae8640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd77d9f0 0x5609ddb88510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.022+0000 7fd018ae8640 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd77d9f0 0x5609ddb88510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aa638a93b0b2451a server_cookie=17b8180e8f86b9ac in_seq=0 out_seq=0 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.022+0000 7fcff3fff640 10 client.17385 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fcff3fff640 1 -- 172.21.15.31:0/240698940 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609ddadedf0 con 0x5609dd77d9f0 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fcff3fff640 10 client.17385 handle_command_reply: tid=0 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 client.17385 shutdown 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 msgr2=0x5609ddb722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 0x5609ddb722b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd0040ce940 tx=0x7fd004030000 comp rx=0 tx=0).stop 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 shutdown_connections 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd77d9f0 0x5609ddb88510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7bf070 0x5609ddb7e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd79e390 0x5609ddb722b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fcff3fff640 0 client.17385 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 --2- 172.21.15.31:0/240698940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd7961b0 0x5609ddb71d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 >> 172.21.15.31:0/240698940 conn(0x5609dd7b3460 msgr2=0x5609ddb69830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:51.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 shutdown_connections 2024-09-21T03:00:51.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 1 -- 172.21.15.31:0/240698940 wait complete. 2024-09-21T03:00:51.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.023+0000 7fd0212c0740 20 client.17385 trim_cache size 0 max 16384 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.036+0000 7fd0212c0740 1 Processor -- start 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.036+0000 7fd0212c0740 1 -- start start 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd77d9f0 0x5609ddb70c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd79e390 0x5609ddb7e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd0212c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ad580 con 0x5609dd7961b0 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd0212c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141af0c0 con 0x5609dd79e390 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd0212c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141adc50 con 0x5609dd77d9f0 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb67de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54110/0 (socket says 172.21.15.31:54110) 2024-09-21T03:00:51.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 -- 172.21.15.31:0/3386734267 learned_addr learned my addr 172.21.15.31:0/3386734267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd018ae8640 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd77d9f0 0x5609ddb70c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 -- 172.21.15.31:0/3386734267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd77d9f0 msgr2=0x5609ddb70c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd77d9f0 0x5609ddb70c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 -- 172.21.15.31:0/3386734267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd79e390 msgr2=0x5609ddb7e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd79e390 0x5609ddb7e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 -- 172.21.15.31:0/3386734267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ddb88a50 con 0x5609dd7961b0 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd018ae8640 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd77d9f0 0x5609ddb70c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd013fff640 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb67de0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd0040d8420 tx=0x7fd004106870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad9ce2f57799e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.037+0000 7fd00a7fc640 1 -- 172.21.15.31:0/3386734267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040ca120 con 0x5609dd7961b0 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd00a7fc640 1 -- 172.21.15.31:0/3386734267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd004036070 con 0x5609dd7961b0 2024-09-21T03:00:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd00a7fc640 1 -- 172.21.15.31:0/3386734267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040ce120 con 0x5609dd7961b0 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 -- 172.21.15.31:0/3386734267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 msgr2=0x5609ddb67de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb67de0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd0040d8420 tx=0x7fd004106870 comp rx=0 tx=0).stop 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 -- 172.21.15.31:0/3386734267 shutdown_connections 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd79e390 0x5609ddb7e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd7961b0 0x5609ddb67de0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3386734267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd77d9f0 0x5609ddb70c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 -- 172.21.15.31:0/3386734267 >> 172.21.15.31:0/3386734267 conn(0x5609dd7b3460 msgr2=0x5609ddb676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 -- 172.21.15.31:0/3386734267 shutdown_connections 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.038+0000 7fd0212c0740 1 -- 172.21.15.31:0/3386734267 wait complete. 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 -- start start 2024-09-21T03:00:51.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 Processor -- start 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 0x5609ddb76440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7961b0 0x5609ddb76980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddbf1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ae9f0 con 0x5609dd77d9f0 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ae320 con 0x5609dd7961b0 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ab560 con 0x5609dd79e390 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd0212c0740 10 client.0 _ll_register_callbacks cb 0x7fd004009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.039+0000 7fd018ae8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 0x5609ddb76440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 0x5609ddb76440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54116/0 (socket says 172.21.15.31:54116) 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 -- 172.21.15.31:0/3733784132 learned_addr learned my addr 172.21.15.31:0/3733784132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd013fff640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7961b0 0x5609ddb76980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd0192e9640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddbf1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 -- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 msgr2=0x5609ddbf1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddbf1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 -- 172.21.15.31:0/3733784132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7961b0 msgr2=0x5609ddb76980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7961b0 0x5609ddb76980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 -- 172.21.15.31:0/3733784132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ddb77270 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd018ae8640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 0x5609ddb76440 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00bbb0 tx=0x7fcffc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a52f90a7185471 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fcff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fcff3fff640 1 -- 172.21.15.31:0/3733784132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc015040 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fcff3fff640 1 -- 172.21.15.31:0/3733784132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fcffc004130 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd004036070 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fcff3fff640 1 -- 172.21.15.31:0/3733784132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcffc004330 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd0212c0740 10 client.17391 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.040+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609ddbf33e0 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fcff3fff640 1 -- 172.21.15.31:0/3733784132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd004036070 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fcff3fff640 1 -- 172.21.15.31:0/3733784132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcffc003960 con 0x5609dd77d9f0 2024-09-21T03:00:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd0212c0740 10 client.17391 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd0212c0740 10 client.17391 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd0212c0740 20 client.17391 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd7bf070 0x5609ddbf5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd0212c0740 4 client.17391 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609ddada8a0 con 0x5609dd7bf070 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.041+0000 7fd013fff640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd7bf070 0x5609ddbf5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fcff3fff640 10 client.17391 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd013fff640 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd7bf070 0x5609ddbf5890 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d7fa97e7aa36c09 server_cookie=6fdb93f93232ebf5 in_seq=0 out_seq=0 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fcff3fff640 1 -- 172.21.15.31:0/3733784132 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609ddada8a0 con 0x5609dd7bf070 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fcff3fff640 10 client.17391 handle_command_reply: tid=0 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 client.17391 shutdown 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 msgr2=0x5609ddb76440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 0x5609ddb76440 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00bbb0 tx=0x7fcffc00b990 comp rx=0 tx=0).stop 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 shutdown_connections 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x5609dd7bf070 0x5609ddbf5890 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609dd79e390 0x5609ddbf1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609dd7961b0 0x5609ddb76980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fcff3fff640 0 client.17391 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 --2- 172.21.15.31:0/3733784132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609dd77d9f0 0x5609ddb76440 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 >> 172.21.15.31:0/3733784132 conn(0x5609dd7b3460 msgr2=0x5609ddb73220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 shutdown_connections 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 1 -- 172.21.15.31:0/3733784132 wait complete. 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.042+0000 7fd0212c0740 20 client.17391 trim_cache size 0 max 16384 2024-09-21T03:00:51.048 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0000b6680 msgr2=0x7fd0000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0000b6680 0x7fd0000b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00be70 tx=0x7fd00c00b450 comp rx=0 tx=0).stop 2024-09-21T03:00:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 msgr2=0x7fd01419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01419aa00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd0040319f0 tx=0x7fd004031a20 comp rx=0 tx=0).stop 2024-09-21T03:00:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 shutdown_connections 2024-09-21T03:00:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd0000b6680 0x7fd0000b8b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd01410ddf0 0x7fd01419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd014105270 0x7fd01419aa00 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 --2- 172.21.15.31:0/863090909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd014104840 0x7fd01419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 >> 172.21.15.31:0/863090909 conn(0x7fd014079730 msgr2=0x7fd014100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 shutdown_connections 2024-09-21T03:00:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:51.043+0000 7fd01ad4c640 1 -- 172.21.15.31:0/863090909 wait complete. 2024-09-21T03:00:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:50 smithi165 ceph-mon[34818]: pgmap v565: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:50 smithi178 ceph-mon[34759]: pgmap v565: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:51.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:00:52.051 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 -- 172.21.15.31:0/3685906257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 msgr2=0x7f7a301087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 --2- 172.21.15.31:0/3685906257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a301087d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009900 tx=0x7f7a2002a410 comp rx=0 tx=0).stop 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 -- 172.21.15.31:0/3685906257 shutdown_connections 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 --2- 172.21.15.31:0/3685906257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30108d10 0x7f7a3010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 --2- 172.21.15.31:0/3685906257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a301087d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 --2- 172.21.15.31:0/3685906257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a30102a80 0x7f7a30102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 -- 172.21.15.31:0/3685906257 >> 172.21.15.31:0/3685906257 conn(0x7f7a30079740 msgr2=0x7f7a30079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 -- 172.21.15.31:0/3685906257 shutdown_connections 2024-09-21T03:00:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.570+0000 7f7a37390640 1 -- 172.21.15.31:0/3685906257 wait complete. 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 Processor -- start 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 -- start start 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30102a80 0x7f7a3019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a3019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a30108d10 0x7f7a301a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a30112980 con 0x7f7a301083c0 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a30112800 con 0x7f7a30102a80 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.571+0000 7f7a37390640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a30112b00 con 0x7f7a30108d10 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a3019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a3019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54132/0 (socket says 172.21.15.31:54132) 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 -- 172.21.15.31:0/1887106969 learned_addr learned my addr 172.21.15.31:0/1887106969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30102a80 0x7f7a3019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3592d640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a30108d10 0x7f7a301a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 -- 172.21.15.31:0/1887106969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a30108d10 msgr2=0x7f7a301a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a30108d10 0x7f7a301a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 -- 172.21.15.31:0/1887106969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30102a80 msgr2=0x7f7a3019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30102a80 0x7f7a3019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 -- 172.21.15.31:0/1887106969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a301a2630 con 0x7f7a301083c0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30102a80 0x7f7a3019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a3492b640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a3019cc60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006010 tx=0x7f7a20004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a267fc640 1 -- 172.21.15.31:0/1887106969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2003f030 con 0x7f7a301083c0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a301a28c0 con 0x7f7a301083c0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a267fc640 1 -- 172.21.15.31:0/1887106969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a20003aa0 con 0x7f7a301083c0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a267fc640 1 -- 172.21.15.31:0/1887106969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20035070 con 0x7f7a301083c0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.572+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a301a9080 con 0x7f7a301083c0 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 Processor -- start 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 -- start start 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 0x557a970916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a97091be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a97092120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557a97007ea0 con 0x557a96cea070 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557a96ff72d0 con 0x557a96cc11b0 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.575+0000 7f7a3d904740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557a96ffb0e0 con 0x557a96cc9390 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 0x557a970916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 0x557a970916a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54144/0 (socket says 172.21.15.31:54144) 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3512c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a97091be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 -- 172.21.15.31:0/2347586919 learned_addr learned my addr 172.21.15.31:0/2347586919 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a267fc640 1 -- 172.21.15.31:0/1887106969 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7a20042030 con 0x7f7a301083c0 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 -- 172.21.15.31:0/2347586919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 msgr2=0x557a97092120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a97092120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 -- 172.21.15.31:0/2347586919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 msgr2=0x557a97091be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a97091be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 -- 172.21.15.31:0/2347586919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a96d55650 con 0x557a96cea070 2024-09-21T03:00:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a3592d640 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 0x557a970916a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009fc0 tx=0x7f7a28010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f30153afec4abe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a1b7fe640 1 -- 172.21.15.31:0/2347586919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28003b70 con 0x557a96cea070 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a1b7fe640 1 -- 172.21.15.31:0/2347586919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a28003d10 con 0x557a96cea070 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.576+0000 7f7a1b7fe640 1 -- 172.21.15.31:0/2347586919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280119e0 con 0x557a96cea070 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a267fc640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a1c0b6680 0x7f7a1c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a1c0b6680 0x7f7a1c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a267fc640 1 -- 172.21.15.31:0/1887106969 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f7a20107c20 con 0x7f7a301083c0 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 -- 172.21.15.31:0/2347586919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 msgr2=0x557a970916a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 0x557a970916a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009fc0 tx=0x7f7a28010860 comp rx=0 tx=0).stop 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a1c0b6680 0x7f7a1c0b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x557a97090320 tx=0x7f7a2c00b5a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 -- 172.21.15.31:0/2347586919 shutdown_connections 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a97092120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a97091be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2347586919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cea070 0x557a970916a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 -- 172.21.15.31:0/2347586919 >> 172.21.15.31:0/2347586919 conn(0x557a96cde460 msgr2=0x557a9708f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 -- 172.21.15.31:0/2347586919 shutdown_connections 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.577+0000 7f7a3d904740 1 -- 172.21.15.31:0/2347586919 wait complete. 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 -- start start 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 Processor -- start 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a9709ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 0x557a9709d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557a96e58700 con 0x557a96cc9390 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557a96f92160 con 0x557a96cc11b0 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.578+0000 7f7a3d904740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557a9709fe80 con 0x557a96cea070 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3d904740 10 client.0 _ll_register_callbacks cb 0x557a96fab9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 0x557a9709d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 0x557a9709d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54146/0 (socket says 172.21.15.31:54146) 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 -- 172.21.15.31:0/1671189901 learned_addr learned my addr 172.21.15.31:0/1671189901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3592d640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a9709ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 -- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 msgr2=0x557a970a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 -- 172.21.15.31:0/1671189901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 msgr2=0x557a9709ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a9709ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 -- 172.21.15.31:0/1671189901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a96fbfc90 con 0x557a96cc9390 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3492b640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 0x557a9709d390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7a20031440 tx=0x7f7a200c9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd27c0aea65485 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a18ff9640 1 -- 172.21.15.31:0/1671189901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a200c9050 con 0x557a96cc9390 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a18ff9640 1 -- 172.21.15.31:0/1671189901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f7a200d2210 con 0x557a96cc9390 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.579+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a28003d10 con 0x557a96cc9390 2024-09-21T03:00:52.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a18ff9640 1 -- 172.21.15.31:0/1671189901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a20049040 con 0x557a96cc9390 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 10 client.17403 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a970abc60 con 0x557a96cc9390 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a18ff9640 1 -- 172.21.15.31:0/1671189901 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a28003d10 con 0x557a96cc9390 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a18ff9640 1 -- 172.21.15.31:0/1671189901 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a200d7000 con 0x557a96cc9390 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 10 client.17403 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 10 client.17403 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 20 client.17403 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96ca89f0 0x557a970af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 4 client.17403 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a97009df0 con 0x557a96ca89f0 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.580+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96ca89f0 0x557a970af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.581+0000 7f7a3512c640 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96ca89f0 0x557a970af080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b740ac0d10d4332 server_cookie=17a34b0f649cfaf in_seq=0 out_seq=0 2024-09-21T03:00:52.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.581+0000 7f7a18ff9640 10 client.17403 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a18ff9640 1 -- 172.21.15.31:0/1671189901 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a97009df0 con 0x557a96ca89f0 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a18ff9640 10 client.17403 handle_command_reply: tid=0 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 client.17403 shutdown 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 msgr2=0x557a9709d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 0x557a9709d390 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7a20031440 tx=0x7f7a200c9a40 comp rx=0 tx=0).stop 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 shutdown_connections 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96ca89f0 0x557a970af080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc9390 0x557a9709d390 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 --2- 172.21.15.31:0/1671189901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc11b0 0x557a9709ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 >> 172.21.15.31:0/1671189901 conn(0x557a96cde460 msgr2=0x557a97094850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a18ff9640 0 client.17403 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:52.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 shutdown_connections 2024-09-21T03:00:52.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 1 -- 172.21.15.31:0/1671189901 wait complete. 2024-09-21T03:00:52.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.582+0000 7f7a3d904740 20 client.17403 trim_cache size 0 max 16384 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 Processor -- start 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 -- start start 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a9708f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc9390 0x557a970a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301b19a0 con 0x557a96cc11b0 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301b34e0 con 0x557a96cc9390 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3d904740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301b2070 con 0x557a96cea070 2024-09-21T03:00:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3512c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a9708f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3512c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a9708f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54158/0 (socket says 172.21.15.31:54158) 2024-09-21T03:00:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3512c640 1 -- 172.21.15.31:0/2802366158 learned_addr learned my addr 172.21.15.31:0/2802366158 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3592d640 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.596+0000 7f7a3492b640 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc9390 0x557a970a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3512c640 1 -- 172.21.15.31:0/2802366158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 msgr2=0x557a970947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3512c640 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3512c640 1 -- 172.21.15.31:0/2802366158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc9390 msgr2=0x557a970a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3512c640 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc9390 0x557a970a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3512c640 1 -- 172.21.15.31:0/2802366158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9709e110 con 0x557a96cc11b0 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3492b640 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc9390 0x557a970a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.597+0000 7f7a3512c640 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a9708f5e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c01f4f0 tx=0x7f7a2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6abe6c76505581 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.598+0000 7f7a1affd640 1 -- 172.21.15.31:0/2802366158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c031670 con 0x557a96cc11b0 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.598+0000 7f7a1affd640 1 -- 172.21.15.31:0/2802366158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c02c070 con 0x557a96cc11b0 2024-09-21T03:00:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.598+0000 7f7a1affd640 1 -- 172.21.15.31:0/2802366158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c020b60 con 0x557a96cc11b0 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 -- 172.21.15.31:0/2802366158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 msgr2=0x557a9708f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a9708f5e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c01f4f0 tx=0x7f7a2c015000 comp rx=0 tx=0).stop 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 -- 172.21.15.31:0/2802366158 shutdown_connections 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cea070 0x557a970947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cc9390 0x557a970a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 --2- 172.21.15.31:0/2802366158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a9708f5e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 -- 172.21.15.31:0/2802366158 >> 172.21.15.31:0/2802366158 conn(0x557a96cde460 msgr2=0x557a9708eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 -- 172.21.15.31:0/2802366158 shutdown_connections 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 -- 172.21.15.31:0/2802366158 wait complete. 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 -- start start 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.599+0000 7f7a3d904740 1 Processor -- start 2024-09-21T03:00:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a970a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a970a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cea070 0x557a970a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301b2e10 con 0x557a96cc11b0 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301b2740 con 0x557a96cea070 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301af980 con 0x557a96cc9390 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3d904740 10 client.0 _ll_register_callbacks cb 0x7f7a20005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a970a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a970a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54166/0 (socket says 172.21.15.31:54166) 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 -- 172.21.15.31:0/837448969 learned_addr learned my addr 172.21.15.31:0/837448969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3492b640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a970a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3592d640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cea070 0x557a970a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 -- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 msgr2=0x557a970a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a970a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 -- 172.21.15.31:0/837448969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cea070 msgr2=0x557a970a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cea070 0x557a970a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3512c640 1 -- 172.21.15.31:0/837448969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a97118bd0 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.600+0000 7f7a3492b640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a970a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3512c640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a970a1300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006010 tx=0x7f7a2c020510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c01dfcdce12e2fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a18ff9640 1 -- 172.21.15.31:0/837448969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c038070 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a18ff9640 1 -- 172.21.15.31:0/837448969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f7a2c020d70 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2c02c070 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a18ff9640 1 -- 172.21.15.31:0/837448969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a2c031470 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 10 client.17409 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a97119e30 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a18ff9640 1 -- 172.21.15.31:0/837448969 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c02c070 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a18ff9640 1 -- 172.21.15.31:0/837448969 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a2c03c6d0 con 0x557a96cc11b0 2024-09-21T03:00:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 10 client.17409 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 10 client.17409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:00:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 20 client.17409 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96c7dda0 0x557a9711c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 4 client.17409 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.601+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a970058a0 con 0x557a96c7dda0 2024-09-21T03:00:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a3492b640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96c7dda0 0x557a9711c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a18ff9640 10 client.17409 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a3492b640 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96c7dda0 0x557a9711c5c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16bf8c0898072cf1 server_cookie=e7a25c3c8a255eb9 in_seq=0 out_seq=0 2024-09-21T03:00:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a18ff9640 1 -- 172.21.15.31:0/837448969 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a970058a0 con 0x557a96c7dda0 2024-09-21T03:00:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a18ff9640 10 client.17409 handle_command_reply: tid=0 2024-09-21T03:00:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a3d904740 1 client.17409 shutdown 2024-09-21T03:00:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 msgr2=0x557a970a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a3d904740 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a970a1300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006010 tx=0x7f7a2c020510 comp rx=0 tx=0).stop 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.602+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 shutdown_connections 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x557a96c7dda0 0x557a9711c5c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557a96cc9390 0x557a970a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a18ff9640 0 client.17409 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a96cea070 0x557a970a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 --2- 172.21.15.31:0/837448969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a96cc11b0 0x557a970a1300 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 >> 172.21.15.31:0/837448969 conn(0x557a96cde460 msgr2=0x557a9709cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 shutdown_connections 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 1 -- 172.21.15.31:0/837448969 wait complete. 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a3d904740 20 client.17409 trim_cache size 0 max 16384 2024-09-21T03:00:52.608 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a1c0b6680 msgr2=0x7f7a1c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a1c0b6680 0x7f7a1c0b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x557a97090320 tx=0x7f7a2c00b5a0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 msgr2=0x7f7a3019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a3019cc60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006010 tx=0x7f7a20004060 comp rx=0 tx=0).stop 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 shutdown_connections 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7a1c0b6680 0x7f7a1c0b8b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a30108d10 0x7f7a301a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a301083c0 0x7f7a3019cc60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 --2- 172.21.15.31:0/1887106969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a30102a80 0x7f7a3019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.603+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 >> 172.21.15.31:0/1887106969 conn(0x7f7a30079740 msgr2=0x7f7a301044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.604+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 shutdown_connections 2024-09-21T03:00:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:52.604+0000 7f7a37390640 1 -- 172.21.15.31:0/1887106969 wait complete. 2024-09-21T03:00:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:52 smithi031 ceph-mon[31209]: pgmap v566: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:52 smithi165 ceph-mon[34818]: pgmap v566: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:53.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:00:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:52 smithi178 ceph-mon[34759]: pgmap v566: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:00:53.454 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 -- 172.21.15.31:0/1618928982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb0109df0 msgr2=0x7fbeb0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 --2- 172.21.15.31:0/1618928982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb0109df0 0x7fbeb0110620 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009ca0 tx=0x7fbea002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 -- 172.21.15.31:0/1618928982 shutdown_connections 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 --2- 172.21.15.31:0/1618928982 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb01094a0 0x7fbeb01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 --2- 172.21.15.31:0/1618928982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb0109df0 0x7fbeb0110620 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 --2- 172.21.15.31:0/1618928982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeb0103b60 0x7fbeb0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 -- 172.21.15.31:0/1618928982 >> 172.21.15.31:0/1618928982 conn(0x7fbeb007b850 msgr2=0x7fbeb007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 -- 172.21.15.31:0/1618928982 shutdown_connections 2024-09-21T03:00:53.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.987+0000 7fbeb759f640 1 -- 172.21.15.31:0/1618928982 wait complete. 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 Processor -- start 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 -- start start 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeb0103b60 0x7fbeb019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 0x7fbeb019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb0109df0 0x7fbeb019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0113ab0 con 0x7fbeb01094a0 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0113930 con 0x7fbeb0103b60 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb759f640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0113c30 con 0x7fbeb0109df0 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 0x7fbeb019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 0x7fbeb019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54186/0 (socket says 172.21.15.31:54186) 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3315234806 learned_addr learned my addr 172.21.15.31:0/3315234806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb533b640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeb0103b60 0x7fbeb019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb0109df0 0x7fbeb019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3315234806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb0109df0 msgr2=0x7fbeb019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb0109df0 0x7fbeb019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3315234806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeb0103b60 msgr2=0x7fbeb019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeb0103b60 0x7fbeb019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.988+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3315234806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb01a03f0 con 0x7fbeb01094a0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.989+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 0x7fbeb019aa20 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbea400d750 tx=0x7fbea400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.989+0000 7fbeae7fc640 1 -- 172.21.15.31:0/3315234806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4017070 con 0x7fbeb01094a0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.989+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb01a06e0 con 0x7fbeb01094a0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.989+0000 7fbeae7fc640 1 -- 172.21.15.31:0/3315234806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea40054d0 con 0x7fbeb01094a0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.989+0000 7fbeae7fc640 1 -- 172.21.15.31:0/3315234806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4004c30 con 0x7fbeb01094a0 2024-09-21T03:00:53.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.989+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb01a6e40 con 0x7fbeb01094a0 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 Processor -- start 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 -- start start 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbeae7fc640 1 -- 172.21.15.31:0/3315234806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbea4004dd0 con 0x7fbeb01094a0 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 0x5588ee6716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a11b0 0x5588ee671be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a9390 0x5588ee672120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5588ee5e7ea0 con 0x5588ee2ca070 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5588ee5d72d0 con 0x5588ee2a11b0 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbebdb13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5588ee5db0e0 con 0x5588ee2a9390 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbeb5b3c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 0x5588ee6716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.992+0000 7fbeb5b3c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 0x5588ee6716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54196/0 (socket says 172.21.15.31:54196) 2024-09-21T03:00:53.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 -- 172.21.15.31:0/884499976 learned_addr learned my addr 172.21.15.31:0/884499976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a9390 0x5588ee672120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb533b640 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a11b0 0x5588ee671be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 -- 172.21.15.31:0/884499976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a9390 msgr2=0x5588ee672120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a9390 0x5588ee672120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 -- 172.21.15.31:0/884499976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a11b0 msgr2=0x5588ee671be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a11b0 0x5588ee671be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 -- 172.21.15.31:0/884499976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ee335650 con 0x5588ee2ca070 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeae7fc640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbe980b6680 0x7fbe980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 0x5588ee6716a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbea002f4d0 tx=0x7fbea0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6b4ab0083f6c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeb533b640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbe980b6680 0x7fbe980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbe977fe640 1 -- 172.21.15.31:0/884499976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0031cf0 con 0x5588ee2ca070 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbeae7fc640 1 -- 172.21.15.31:0/3315234806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fbea40e7140 con 0x7fbeb01094a0 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbe977fe640 1 -- 172.21.15.31:0/884499976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea0040070 con 0x5588ee2ca070 2024-09-21T03:00:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.993+0000 7fbe977fe640 1 -- 172.21.15.31:0/884499976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00042a0 con 0x5588ee2ca070 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 -- 172.21.15.31:0/884499976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 msgr2=0x5588ee6716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 0x5588ee6716a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbea002f4d0 tx=0x7fbea0009800 comp rx=0 tx=0).stop 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbeb533b640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbe980b6680 0x7fbe980b8b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbea8004610 tx=0x7fbea800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 -- 172.21.15.31:0/884499976 shutdown_connections 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a9390 0x5588ee672120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a11b0 0x5588ee671be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 --2- 172.21.15.31:0/884499976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2ca070 0x5588ee6716a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 -- 172.21.15.31:0/884499976 >> 172.21.15.31:0/884499976 conn(0x5588ee2be460 msgr2=0x5588ee66f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.994+0000 7fbebdb13740 1 -- 172.21.15.31:0/884499976 shutdown_connections 2024-09-21T03:00:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.995+0000 7fbebdb13740 1 -- 172.21.15.31:0/884499976 wait complete. 2024-09-21T03:00:53.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.995+0000 7fbebdb13740 1 -- start start 2024-09-21T03:00:53.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.995+0000 7fbebdb13740 1 Processor -- start 2024-09-21T03:00:53.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.995+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee681160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.995+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee68d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbebdb13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5588ee438700 con 0x5588ee2a9390 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbebdb13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5588ee572160 con 0x5588ee2ca070 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbebdb13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5588ee684150 con 0x5588ee2a11b0 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbebdb13740 10 client.0 _ll_register_callbacks cb 0x5588ee58b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6816a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54210/0 (socket says 172.21.15.31:54210) 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/172149151 learned_addr learned my addr 172.21.15.31:0/172149151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb533b640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee681160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee68d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/172149151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 msgr2=0x5588ee681160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:54.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee681160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 msgr2=0x5588ee68d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee68d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/172149151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ee59fc90 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6816a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbea40acf20 tx=0x7fbea400dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdadcfa145a2d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbe94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.996+0000 7fbe94ff9640 1 -- 172.21.15.31:0/172149151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4020070 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbe94ff9640 1 -- 172.21.15.31:0/172149151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fbea40a9200 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea0040070 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbe94ff9640 1 -- 172.21.15.31:0/172149151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbea40ac8e0 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 10 client.17421 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588ee68ff30 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbe94ff9640 1 -- 172.21.15.31:0/172149151 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea0040070 con 0x5588ee2a9390 2024-09-21T03:00:54.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbe94ff9640 1 -- 172.21.15.31:0/172149151 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbea40981a0 con 0x5588ee2a9390 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 10 client.17421 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 10 client.17421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 20 client.17421 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee2889f0 0x5588ee6934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 4 client.17421 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ee5e9df0 con 0x5588ee2889f0 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.997+0000 7fbeb533b640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee2889f0 0x5588ee6934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.998+0000 7fbeb533b640 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee2889f0 0x5588ee6934d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=999de4833395c1a1 server_cookie=2735fa4ff4b1f6a5 in_seq=0 out_seq=0 2024-09-21T03:00:54.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.998+0000 7fbe94ff9640 10 client.17421 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbe94ff9640 1 -- 172.21.15.31:0/172149151 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588ee5e9df0 con 0x5588ee2889f0 2024-09-21T03:00:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbe94ff9640 10 client.17421 handle_command_reply: tid=0 2024-09-21T03:00:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 client.17421 shutdown 2024-09-21T03:00:54.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 msgr2=0x5588ee6816a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6816a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbea40acf20 tx=0x7fbea400dee0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 shutdown_connections 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee2889f0 0x5588ee6934d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee68d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6816a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 --2- 172.21.15.31:0/172149151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee681160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 >> 172.21.15.31:0/172149151 conn(0x5588ee2be460 msgr2=0x5588ee674830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbe94ff9640 0 client.17421 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 shutdown_connections 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 1 -- 172.21.15.31:0/172149151 wait complete. 2024-09-21T03:00:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:53.999+0000 7fbebdb13740 20 client.17421 trim_cache size 0 max 16384 2024-09-21T03:00:54.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.012+0000 7fbebdb13740 1 Processor -- start 2024-09-21T03:00:54.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.012+0000 7fbebdb13740 1 -- start start 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 0x5588ee683040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a9390 0x5588ee672de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2ca070 0x5588ee68d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbebdb13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ad650 con 0x5588ee2a11b0 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbebdb13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01af190 con 0x5588ee2a9390 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbebdb13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01add20 con 0x5588ee2ca070 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 0x5588ee683040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 0x5588ee683040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54214/0 (socket says 172.21.15.31:54214) 2024-09-21T03:00:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 -- 172.21.15.31:0/2996379392 learned_addr learned my addr 172.21.15.31:0/2996379392 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a9390 0x5588ee672de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2ca070 0x5588ee68d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 -- 172.21.15.31:0/2996379392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2ca070 msgr2=0x5588ee68d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2ca070 0x5588ee68d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 -- 172.21.15.31:0/2996379392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a9390 msgr2=0x5588ee672de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a9390 0x5588ee672de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 -- 172.21.15.31:0/2996379392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ee6838a0 con 0x5588ee2a11b0 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2ca070 0x5588ee68d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.013+0000 7fbeb533b640 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 0x5588ee683040 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbea800bfd0 tx=0x7fbea8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c2436bd0a87ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbe96ffd640 1 -- 172.21.15.31:0/2996379392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8005020 con 0x5588ee2a11b0 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbe96ffd640 1 -- 172.21.15.31:0/2996379392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea80047a0 con 0x5588ee2a11b0 2024-09-21T03:00:54.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbe96ffd640 1 -- 172.21.15.31:0/2996379392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8028630 con 0x5588ee2a11b0 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 -- 172.21.15.31:0/2996379392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 msgr2=0x5588ee683040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 0x5588ee683040 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbea800bfd0 tx=0x7fbea8015000 comp rx=0 tx=0).stop 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 -- 172.21.15.31:0/2996379392 shutdown_connections 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2ca070 0x5588ee68d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2a9390 0x5588ee672de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 --2- 172.21.15.31:0/2996379392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a11b0 0x5588ee683040 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 -- 172.21.15.31:0/2996379392 >> 172.21.15.31:0/2996379392 conn(0x5588ee2be460 msgr2=0x5588ee6726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 -- 172.21.15.31:0/2996379392 shutdown_connections 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.014+0000 7fbebdb13740 1 -- 172.21.15.31:0/2996379392 wait complete. 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 -- start start 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 Processor -- start 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee689df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee6859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01aeac0 con 0x5588ee2a9390 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ae3f0 con 0x5588ee2ca070 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01ab630 con 0x5588ee2a11b0 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.015+0000 7fbebdb13740 10 client.0 _ll_register_callbacks cb 0x7fbea40ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54218/0 (socket says 172.21.15.31:54218) 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3482102898 learned_addr learned my addr 172.21.15.31:0/3482102898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb5b3c640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee6859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb533b640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee689df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3482102898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 msgr2=0x5588ee689df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee689df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 msgr2=0x5588ee6859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee6859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 -- 172.21.15.31:0/3482102898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ee6fce70 con 0x5588ee2a9390 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbeb4b3a640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6852e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbea40acc90 tx=0x7fbea40e4b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9d1980f579c29c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbe94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbe94ff9640 1 -- 172.21.15.31:0/3482102898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4020070 con 0x5588ee2a9390 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbe94ff9640 1 -- 172.21.15.31:0/3482102898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fbea40942f0 con 0x5588ee2a9390 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.016+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea80047a0 con 0x5588ee2a9390 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbe94ff9640 1 -- 172.21.15.31:0/3482102898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbea40ac700 con 0x5588ee2a9390 2024-09-21T03:00:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 10 client.17427 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588ee6fdf30 con 0x5588ee2a9390 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbe94ff9640 1 -- 172.21.15.31:0/3482102898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea80047a0 con 0x5588ee2a9390 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbe94ff9640 1 -- 172.21.15.31:0/3482102898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbea405c210 con 0x5588ee2a9390 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 10 client.17427 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 10 client.17427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 20 client.17427 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee25dda0 0x5588ee7006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 4 client.17427 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588ee5e58a0 con 0x5588ee25dda0 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.017+0000 7fbeb533b640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee25dda0 0x5588ee7006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbeb533b640 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee25dda0 0x5588ee7006d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d22b07ffc58bb812 server_cookie=81bdebf9b0a9333f in_seq=0 out_seq=0 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbe94ff9640 10 client.17427 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbe94ff9640 1 -- 172.21.15.31:0/3482102898 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588ee5e58a0 con 0x5588ee25dda0 2024-09-21T03:00:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbe94ff9640 10 client.17427 handle_command_reply: tid=0 2024-09-21T03:00:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 client.17427 shutdown 2024-09-21T03:00:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 msgr2=0x5588ee6852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6852e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbea40acc90 tx=0x7fbea40e4b80 comp rx=0 tx=0).stop 2024-09-21T03:00:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 shutdown_connections 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x5588ee25dda0 0x5588ee7006d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5588ee2ca070 0x5588ee6859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588ee2a9390 0x5588ee6852e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 --2- 172.21.15.31:0/3482102898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5588ee2a11b0 0x5588ee689df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbe94ff9640 0 client.17427 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 >> 172.21.15.31:0/3482102898 conn(0x5588ee2be460 msgr2=0x5588ee673090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 shutdown_connections 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 1 -- 172.21.15.31:0/3482102898 wait complete. 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.018+0000 7fbebdb13740 20 client.17427 trim_cache size 0 max 16384 2024-09-21T03:00:54.024 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbe980b6680 msgr2=0x7fbe980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbe980b6680 0x7fbe980b8b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbea8004610 tx=0x7fbea800b390 comp rx=0 tx=0).stop 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 msgr2=0x7fbeb019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 0x7fbeb019aa20 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbea400d750 tx=0x7fbea400dc10 comp rx=0 tx=0).stop 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 shutdown_connections 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbe980b6680 0x7fbe980b8b90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbeb0109df0 0x7fbeb019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbeb01094a0 0x7fbeb019aa20 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 --2- 172.21.15.31:0/3315234806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbeb0103b60 0x7fbeb019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 >> 172.21.15.31:0/3315234806 conn(0x7fbeb007b850 msgr2=0x7fbeb0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 shutdown_connections 2024-09-21T03:00:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:54.019+0000 7fbeb759f640 1 -- 172.21.15.31:0/3315234806 wait complete. 2024-09-21T03:00:54.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:00:55.003 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:54 smithi031 ceph-mon[31209]: pgmap v567: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:00:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:54 smithi165 ceph-mon[34818]: pgmap v567: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:00:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:54 smithi178 ceph-mon[34759]: pgmap v567: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:00:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 -- 172.21.15.31:0/392698050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d401020e0 msgr2=0x7f5d401024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 --2- 172.21.15.31:0/392698050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d401020e0 0x7f5d401024f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009ca0 tx=0x7f5d3002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 -- 172.21.15.31:0/392698050 shutdown_connections 2024-09-21T03:00:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 --2- 172.21.15.31:0/392698050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d40106c20 0x7f5d40110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 --2- 172.21.15.31:0/392698050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d401062d0 0x7f5d401066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 --2- 172.21.15.31:0/392698050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d401020e0 0x7f5d401024f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.556+0000 7f5d47876640 1 -- 172.21.15.31:0/392698050 >> 172.21.15.31:0/392698050 conn(0x7f5d40079e40 msgr2=0x7f5d4007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.557+0000 7f5d47876640 1 -- 172.21.15.31:0/392698050 shutdown_connections 2024-09-21T03:00:55.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.557+0000 7f5d47876640 1 -- 172.21.15.31:0/392698050 wait complete. 2024-09-21T03:00:55.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.557+0000 7f5d47876640 1 Processor -- start 2024-09-21T03:00:55.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.557+0000 7f5d47876640 1 -- start start 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d47876640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d401020e0 0x7f5d4019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d47876640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d401062d0 0x7f5d4019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d47876640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 0x7f5d4019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d47876640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40113ac0 con 0x7f5d40106c20 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d47876640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40113940 con 0x7f5d401062d0 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d47876640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40113c40 con 0x7f5d401020e0 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 0x7f5d4019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 0x7f5d4019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54226/0 (socket says 172.21.15.31:54226) 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45612640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d401020e0 0x7f5d4019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 -- 172.21.15.31:0/36800907 learned_addr learned my addr 172.21.15.31:0/36800907 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:55.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d44e11640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d401062d0 0x7f5d4019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 -- 172.21.15.31:0/36800907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d401020e0 msgr2=0x7f5d4019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d401020e0 0x7f5d4019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 -- 172.21.15.31:0/36800907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d401062d0 msgr2=0x7f5d4019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d401062d0 0x7f5d4019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 -- 172.21.15.31:0/36800907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d401a04a0 con 0x7f5d40106c20 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d45e13640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 0x7f5d4019fd70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e970 tx=0x7f5d3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.558+0000 7f5d3e7fc640 1 -- 172.21.15.31:0/36800907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800ccc0 con 0x7f5d40106c20 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.559+0000 7f5d3e7fc640 1 -- 172.21.15.31:0/36800907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3800ce60 con 0x7f5d40106c20 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.559+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d401a0790 con 0x7f5d40106c20 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.559+0000 7f5d3e7fc640 1 -- 172.21.15.31:0/36800907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38010430 con 0x7f5d40106c20 2024-09-21T03:00:55.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.559+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d40102f60 con 0x7f5d40106c20 2024-09-21T03:00:55.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.561+0000 7f5d4ddea740 1 Processor -- start 2024-09-21T03:00:55.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.561+0000 7f5d3e7fc640 1 -- 172.21.15.31:0/36800907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5d38004f90 con 0x7f5d40106c20 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.561+0000 7f5d4ddea740 1 -- start start 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 0x56127897c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785ac1b0 0x56127897c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785b4390 0x56127897cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5612788f2ea0 con 0x5612785ac1b0 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5612788e22d0 con 0x5612785d5070 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612788e60e0 con 0x5612785b4390 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d44e11640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 0x56127897c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d44e11640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 0x56127897c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:42160/0 (socket says 172.21.15.31:42160) 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d44e11640 1 -- 172.21.15.31:0/1849515424 learned_addr learned my addr 172.21.15.31:0/1849515424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.562+0000 7f5d44e11640 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785b4390 0x56127897cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d45612640 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785ac1b0 0x56127897c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 -- 172.21.15.31:0/1849515424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785b4390 msgr2=0x56127897cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785b4390 0x56127897cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 -- 172.21.15.31:0/1849515424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785ac1b0 msgr2=0x56127897c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785ac1b0 0x56127897c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 -- 172.21.15.31:0/1849515424 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561278640650 con 0x5612785d5070 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d3e7fc640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d280b6700 0x7f5d280b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d45612640 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785ac1b0 0x56127897c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d280b6700 0x7f5d280b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d3e7fc640 1 -- 172.21.15.31:0/36800907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f5d380e7a20 con 0x7f5d40106c20 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d44e11640 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 0x56127897c2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d34018ac0 tx=0x7f5d3400ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45b5da7a359333b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:55.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d277fe640 1 -- 172.21.15.31:0/1849515424 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401a9a0 con 0x5612785d5070 2024-09-21T03:00:55.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d277fe640 1 -- 172.21.15.31:0/1849515424 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3400cbd0 con 0x5612785d5070 2024-09-21T03:00:55.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.563+0000 7f5d277fe640 1 -- 172.21.15.31:0/1849515424 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401d3c0 con 0x5612785d5070 2024-09-21T03:00:55.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 -- 172.21.15.31:0/1849515424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 msgr2=0x56127897c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d44e11640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d280b6700 0x7f5d280b8c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5d34024000 tx=0x7f5d34020030 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 0x56127897c2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d34018ac0 tx=0x7f5d3400ea90 comp rx=0 tx=0).stop 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 -- 172.21.15.31:0/1849515424 shutdown_connections 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785b4390 0x56127897cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785ac1b0 0x56127897c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/1849515424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785d5070 0x56127897c2c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 -- 172.21.15.31:0/1849515424 >> 172.21.15.31:0/1849515424 conn(0x5612785c9460 msgr2=0x56127897a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 -- 172.21.15.31:0/1849515424 shutdown_connections 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.564+0000 7f5d4ddea740 1 -- 172.21.15.31:0/1849515424 wait complete. 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 -- start start 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 Processor -- start 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x5612789879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x561278987ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x561278993d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561278743700 con 0x5612785d5070 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56127887d160 con 0x5612785b4390 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56127898aaa0 con 0x5612785ac1b0 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.565+0000 7f5d4ddea740 10 client.0 _ll_register_callbacks cb 0x5612788969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x561278993d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x561278993d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54246/0 (socket says 172.21.15.31:54246) 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 -- 172.21.15.31:0/2792051453 learned_addr learned my addr 172.21.15.31:0/2792051453 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45612640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x5612789879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d44e11640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x561278987ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 -- 172.21.15.31:0/2792051453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 msgr2=0x5612789879b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x5612789879b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 -- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 msgr2=0x561278987ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x561278987ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 -- 172.21.15.31:0/2792051453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612788aac90 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d44e11640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x561278987ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x561278993d90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5d380a9e40 tx=0x7f5d380b71c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aeff6c494ccc41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2792051453 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38005280 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2792051453 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f5d380aeba0 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.566+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3400cbd0 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2792051453 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d380055f0 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 10 client.17436 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561278996750 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2792051453 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3400cbd0 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2792051453 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5d380b34e0 con 0x5612785d5070 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 10 client.17436 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:55.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 20 client.17436 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:55.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5612785939f0 0x561278999c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 4 client.17436 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:55.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612788f4df0 con 0x5612785939f0 2024-09-21T03:00:55.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.567+0000 7f5d45612640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5612785939f0 0x561278999c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.568+0000 7f5d24ff9640 10 client.17436 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:55.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.568+0000 7f5d45612640 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5612785939f0 0x561278999c50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c574ae99c9226d6 server_cookie=1fc1b69ece728930 in_seq=0 out_seq=0 2024-09-21T03:00:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2792051453 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612788f4df0 con 0x5612785939f0 2024-09-21T03:00:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d24ff9640 10 client.17436 handle_command_reply: tid=0 2024-09-21T03:00:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 client.17436 shutdown 2024-09-21T03:00:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 msgr2=0x561278993d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x561278993d90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5d380a9e40 tx=0x7f5d380b71c0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 shutdown_connections 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5612785939f0 0x561278999c50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x561278993d90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x561278987ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2792051453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x5612789879b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 >> 172.21.15.31:0/2792051453 conn(0x5612785c9460 msgr2=0x56127897f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d24ff9640 0 client.17436 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 shutdown_connections 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2792051453 wait complete. 2024-09-21T03:00:55.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.569+0000 7f5d4ddea740 20 client.17436 trim_cache size 0 max 16384 2024-09-21T03:00:55.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.582+0000 7f5d4ddea740 1 Processor -- start 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.582+0000 7f5d4ddea740 1 -- start start 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785ac1b0 0x56127897a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 0x561278993c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785d5070 0x56127897f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401ad630 con 0x5612785b4390 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401af170 con 0x5612785ac1b0 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401add00 con 0x5612785d5070 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 0x561278993c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 0x561278993c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54250/0 (socket says 172.21.15.31:54250) 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 -- 172.21.15.31:0/2471111941 learned_addr learned my addr 172.21.15.31:0/2471111941 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d45612640 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785ac1b0 0x56127897a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785d5070 0x56127897f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 -- 172.21.15.31:0/2471111941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785d5070 msgr2=0x56127897f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785d5070 0x56127897f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 -- 172.21.15.31:0/2471111941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785ac1b0 msgr2=0x56127897a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785ac1b0 0x56127897a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 -- 172.21.15.31:0/2471111941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561278988bf0 con 0x5612785b4390 2024-09-21T03:00:55.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.583+0000 7f5d44e11640 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 0x561278993c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5d3401a760 tx=0x7f5d34029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b06f7e3a0fb4fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d26ffd640 1 -- 172.21.15.31:0/2471111941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401d3c0 con 0x5612785b4390 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d26ffd640 1 -- 172.21.15.31:0/2471111941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34037070 con 0x5612785b4390 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d26ffd640 1 -- 172.21.15.31:0/2471111941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401eb40 con 0x5612785b4390 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2471111941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 msgr2=0x561278993c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 0x561278993c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5d3401a760 tx=0x7f5d34029000 comp rx=0 tx=0).stop 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2471111941 shutdown_connections 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785d5070 0x56127897f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785b4390 0x561278993c90 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2471111941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785ac1b0 0x56127897a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.584+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2471111941 >> 172.21.15.31:0/2471111941 conn(0x5612785c9460 msgr2=0x561278979ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2471111941 shutdown_connections 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2471111941 wait complete. 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 -- start start 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 Processor -- start 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x56127898bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x56127898c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x56127898c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401aeaa0 con 0x5612785d5070 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401ae3d0 con 0x5612785b4390 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.585+0000 7f5d4ddea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401ab610 con 0x5612785ac1b0 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d4ddea740 10 client.0 _ll_register_callbacks cb 0x7f5d380048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x56127898c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x56127898c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54262/0 (socket says 172.21.15.31:54262) 2024-09-21T03:00:55.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 -- 172.21.15.31:0/2436674598 learned_addr learned my addr 172.21.15.31:0/2436674598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d44e11640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x56127898c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45612640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x56127898bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 -- 172.21.15.31:0/2436674598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 msgr2=0x56127898bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x56127898bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 -- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 msgr2=0x56127898c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x56127898c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 -- 172.21.15.31:0/2436674598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561278a03620 con 0x5612785d5070 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d45e13640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x56127898c7c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c6a0 tx=0x7f5d380b36f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213439a25297342f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2436674598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380cd060 con 0x5612785d5070 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2436674598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f5d380cf360 con 0x5612785d5070 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.586+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d34037070 con 0x5612785d5070 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2436674598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d38005280 con 0x5612785d5070 2024-09-21T03:00:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 10 client.17442 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561278a048a0 con 0x5612785d5070 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2436674598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34037070 con 0x5612785d5070 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2436674598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5d380b67d0 con 0x5612785d5070 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 10 client.17442 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 20 client.17442 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561278568da0 0x561278a06e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 4 client.17442 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612788f08a0 con 0x561278568da0 2024-09-21T03:00:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.587+0000 7f5d45612640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561278568da0 0x561278a06e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d45612640 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561278568da0 0x561278a06e40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a80ad4c81f3113 server_cookie=c5ea51fa1db1fbc4 in_seq=0 out_seq=0 2024-09-21T03:00:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d24ff9640 10 client.17442 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d24ff9640 1 -- 172.21.15.31:0/2436674598 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612788f08a0 con 0x561278568da0 2024-09-21T03:00:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d24ff9640 10 client.17442 handle_command_reply: tid=0 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 client.17442 shutdown 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 msgr2=0x56127898c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x56127898c7c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c6a0 tx=0x7f5d380b36f0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 shutdown_connections 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561278568da0 0x561278a06e40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612785d5070 0x56127898c7c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612785b4390 0x56127898c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d24ff9640 0 client.17442 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 --2- 172.21.15.31:0/2436674598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612785ac1b0 0x56127898bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 >> 172.21.15.31:0/2436674598 conn(0x5612785c9460 msgr2=0x561278987760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 shutdown_connections 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 1 -- 172.21.15.31:0/2436674598 wait complete. 2024-09-21T03:00:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.588+0000 7f5d4ddea740 20 client.17442 trim_cache size 0 max 16384 2024-09-21T03:00:55.595 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:00:55.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d280b6700 msgr2=0x7f5d280b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d280b6700 0x7f5d280b8c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5d34024000 tx=0x7f5d34020030 comp rx=0 tx=0).stop 2024-09-21T03:00:55.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 msgr2=0x7f5d4019fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 0x7f5d4019fd70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e970 tx=0x7f5d3800ed70 comp rx=0 tx=0).stop 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 shutdown_connections 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f5d280b6700 0x7f5d280b8c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d40106c20 0x7f5d4019fd70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d401062d0 0x7f5d4019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 --2- 172.21.15.31:0/36800907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d401020e0 0x7f5d4019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 >> 172.21.15.31:0/36800907 conn(0x7f5d40079e40 msgr2=0x7f5d40100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 shutdown_connections 2024-09-21T03:00:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:55.589+0000 7f5d47876640 1 -- 172.21.15.31:0/36800907 wait complete. 2024-09-21T03:00:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:56 smithi031 ceph-mon[31209]: pgmap v568: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:56 smithi165 ceph-mon[34818]: pgmap v568: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:56 smithi178 ceph-mon[34759]: pgmap v568: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:00:57.288 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:00:57.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:00:57.563 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 -- 172.21.15.31:0/292690776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44103b60 msgr2=0x7fcf44103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 --2- 172.21.15.31:0/292690776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44103b60 0x7fcf44103f70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009900 tx=0x7fcf3402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 -- 172.21.15.31:0/292690776 shutdown_connections 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 --2- 172.21.15.31:0/292690776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf441094a0 0x7fcf441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 --2- 172.21.15.31:0/292690776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf44109df0 0x7fcf44110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 --2- 172.21.15.31:0/292690776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44103b60 0x7fcf44103f70 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 -- 172.21.15.31:0/292690776 >> 172.21.15.31:0/292690776 conn(0x7fcf4407b850 msgr2=0x7fcf4407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:58.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.058+0000 7fcf4a474640 1 -- 172.21.15.31:0/292690776 shutdown_connections 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.059+0000 7fcf4a474640 1 -- 172.21.15.31:0/292690776 wait complete. 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.059+0000 7fcf4a474640 1 Processor -- start 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.059+0000 7fcf4a474640 1 -- start start 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf4a474640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf44103b60 0x7fcf4419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf4a474640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf441094a0 0x7fcf4419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf4a474640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 0x7fcf4419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf4a474640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44113ab0 con 0x7fcf44109df0 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf4a474640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44113930 con 0x7fcf441094a0 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf4a474640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44113c30 con 0x7fcf44103b60 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 0x7fcf4419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 0x7fcf4419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54278/0 (socket says 172.21.15.31:54278) 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 -- 172.21.15.31:0/3322030667 learned_addr learned my addr 172.21.15.31:0/3322030667 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf44103b60 0x7fcf4419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf441094a0 0x7fcf4419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 -- 172.21.15.31:0/3322030667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf44103b60 msgr2=0x7fcf4419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf44103b60 0x7fcf4419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 -- 172.21.15.31:0/3322030667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf441094a0 msgr2=0x7fcf4419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf441094a0 0x7fcf4419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf48a11640 1 -- 172.21.15.31:0/3322030667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf441a0350 con 0x7fcf44109df0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.060+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf441094a0 0x7fcf4419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.061+0000 7fcf48a11640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 0x7fcf4419fce0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00eaa0 tx=0x7fcf3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.061+0000 7fcf417fa640 1 -- 172.21.15.31:0/3322030667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00cdb0 con 0x7fcf44109df0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.061+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf441a0640 con 0x7fcf44109df0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.061+0000 7fcf417fa640 1 -- 172.21.15.31:0/3322030667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c005020 con 0x7fcf44109df0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.061+0000 7fcf417fa640 1 -- 172.21.15.31:0/3322030667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0116d0 con 0x7fcf44109df0 2024-09-21T03:00:58.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.061+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a6e60 con 0x7fcf44109df0 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf417fa640 1 -- 172.21.15.31:0/3322030667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcf3c011870 con 0x7fcf44109df0 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 Processor -- start 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 -- start start 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5848430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff5848970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5848eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561ff57bee60 con 0x561ff5478490 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ff57ae290 con 0x561ff5480670 2024-09-21T03:00:58.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf509e8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ff57b20a0 con 0x561ff54a1350 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff5848970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff5848970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54282/0 (socket says 172.21.15.31:54282) 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf43fff640 1 -- 172.21.15.31:0/2136213397 learned_addr learned my addr 172.21.15.31:0/2136213397 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf437fe640 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5848430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.064+0000 7fcf437fe640 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5848eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf43fff640 1 -- 172.21.15.31:0/2136213397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 msgr2=0x561ff5848430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf43fff640 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5848430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf417fa640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcf2c0b66b0 0x7fcf2c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf43fff640 1 -- 172.21.15.31:0/2136213397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 msgr2=0x561ff5848eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf43fff640 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5848eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf43fff640 1 -- 172.21.15.31:0/2136213397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff550c930 con 0x561ff5478490 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf437fe640 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5848eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:58.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf48a11640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcf2c0b66b0 0x7fcf2c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf43fff640 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff5848970 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009900 tx=0x7fcf34032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a558502bfc90679 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf417fa640 1 -- 172.21.15.31:0/3322030667 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fcf3c0e7980 con 0x7fcf44109df0 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf2a7fc640 1 -- 172.21.15.31:0/2136213397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34035030 con 0x561ff5478490 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf2a7fc640 1 -- 172.21.15.31:0/2136213397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf340316a0 con 0x561ff5478490 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.065+0000 7fcf2a7fc640 1 -- 172.21.15.31:0/2136213397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf340046c0 con 0x561ff5478490 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 -- 172.21.15.31:0/2136213397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 msgr2=0x561ff5848970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf48a11640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcf2c0b66b0 0x7fcf2c0b8bc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009270 tx=0x7fcf3c07a480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff5848970 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009900 tx=0x7fcf34032ce0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 -- 172.21.15.31:0/2136213397 shutdown_connections 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5848eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff5848970 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 --2- 172.21.15.31:0/2136213397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5848430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 -- 172.21.15.31:0/2136213397 >> 172.21.15.31:0/2136213397 conn(0x561ff5495740 msgr2=0x561ff5846320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:58.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 -- 172.21.15.31:0/2136213397 shutdown_connections 2024-09-21T03:00:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.066+0000 7fcf509e8740 1 -- 172.21.15.31:0/2136213397 wait complete. 2024-09-21T03:00:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 -- start start 2024-09-21T03:00:58.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 Processor -- start 2024-09-21T03:00:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff58583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5858930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5864240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561ff560f890 con 0x561ff5478490 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ff5748d40 con 0x561ff5480670 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ff585aee0 con 0x561ff54a1350 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf509e8740 10 client.0 _ll_register_callbacks cb 0x561ff57acc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff58583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff58583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54290/0 (socket says 172.21.15.31:54290) 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.067+0000 7fcf437fe640 1 -- 172.21.15.31:0/3401398267 learned_addr learned my addr 172.21.15.31:0/3401398267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5858930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf437fe640 1 -- 172.21.15.31:0/3401398267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 msgr2=0x561ff5864240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5864240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf437fe640 1 -- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 msgr2=0x561ff5858930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5858930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf437fe640 1 -- 172.21.15.31:0/3401398267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff5864800 con 0x561ff5478490 2024-09-21T03:00:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5858930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff58583f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf38011860 tx=0x7fcf38011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e260682ac534b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf237fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf237fe640 1 -- 172.21.15.31:0/3401398267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38010ce0 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf237fe640 1 -- 172.21.15.31:0/3401398267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fcf38005020 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf340316a0 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf237fe640 1 -- 172.21.15.31:0/3401398267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf38005220 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf509e8740 10 client.17454 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ff5867150 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.068+0000 7fcf237fe640 1 -- 172.21.15.31:0/3401398267 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf340316a0 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf237fe640 1 -- 172.21.15.31:0/3401398267 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcf380124e0 con 0x561ff5478490 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf509e8740 10 client.17454 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf509e8740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf509e8740 20 client.17454 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff545fcd0 0x561ff586a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf509e8740 4 client.17454 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ff57c0db0 con 0x561ff545fcd0 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.069+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff545fcd0 0x561ff586a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.070+0000 7fcf237fe640 10 client.17454 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.070+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff545fcd0 0x561ff586a620 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89722b61276eb53 server_cookie=58db477ce4f902e0 in_seq=0 out_seq=0 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.070+0000 7fcf237fe640 1 -- 172.21.15.31:0/3401398267 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ff57c0db0 con 0x561ff545fcd0 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.070+0000 7fcf237fe640 10 client.17454 handle_command_reply: tid=0 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.070+0000 7fcf509e8740 1 client.17454 shutdown 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 msgr2=0x561ff58583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff58583f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf38011860 tx=0x7fcf38011d20 comp rx=0 tx=0).stop 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 shutdown_connections 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff545fcd0 0x561ff586a620 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff54a1350 0x561ff5864240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff5858930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3401398267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5478490 0x561ff58583f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 >> 172.21.15.31:0/3401398267 conn(0x561ff5495740 msgr2=0x561ff584b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf237fe640 0 client.17454 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 shutdown_connections 2024-09-21T03:00:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 1 -- 172.21.15.31:0/3401398267 wait complete. 2024-09-21T03:00:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.071+0000 7fcf509e8740 20 client.17454 trim_cache size 0 max 16384 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.088+0000 7fcf509e8740 1 Processor -- start 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 -- start start 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff584d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 0x561ff584dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff54a1350 0x561ff58553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ad450 con 0x561ff5480670 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441aef90 con 0x561ff54a1350 2024-09-21T03:00:58.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf509e8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441adb20 con 0x561ff5478490 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 0x561ff584dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 0x561ff584dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54306/0 (socket says 172.21.15.31:54306) 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 -- 172.21.15.31:0/1828639299 learned_addr learned my addr 172.21.15.31:0/1828639299 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf437fe640 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff54a1350 0x561ff58553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 -- 172.21.15.31:0/1828639299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 msgr2=0x561ff584d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf437fe640 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff584d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff584d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 -- 172.21.15.31:0/1828639299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff54a1350 msgr2=0x561ff58553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff54a1350 0x561ff58553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 -- 172.21.15.31:0/1828639299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff58566f0 con 0x561ff5480670 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf437fe640 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff584d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.089+0000 7fcf43fff640 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 0x561ff584dc60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcf34000c00 tx=0x7fcf34005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef951b4a8e2b5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf29ffb640 1 -- 172.21.15.31:0/1828639299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34035040 con 0x561ff5480670 2024-09-21T03:00:58.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf29ffb640 1 -- 172.21.15.31:0/1828639299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf34005640 con 0x561ff5480670 2024-09-21T03:00:58.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf29ffb640 1 -- 172.21.15.31:0/1828639299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34004e40 con 0x561ff5480670 2024-09-21T03:00:58.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 -- 172.21.15.31:0/1828639299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 msgr2=0x561ff584dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 0x561ff584dc60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcf34000c00 tx=0x7fcf34005220 comp rx=0 tx=0).stop 2024-09-21T03:00:58.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 -- 172.21.15.31:0/1828639299 shutdown_connections 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff54a1350 0x561ff58553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff5480670 0x561ff584dc60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 --2- 172.21.15.31:0/1828639299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff584d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.090+0000 7fcf509e8740 1 -- 172.21.15.31:0/1828639299 >> 172.21.15.31:0/1828639299 conn(0x561ff5495740 msgr2=0x561ff5856a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.091+0000 7fcf509e8740 1 -- 172.21.15.31:0/1828639299 shutdown_connections 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.091+0000 7fcf509e8740 1 -- 172.21.15.31:0/1828639299 wait complete. 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.091+0000 7fcf509e8740 1 -- start start 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.091+0000 7fcf509e8740 1 Processor -- start 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff58603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff58608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 0x561ff585b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ae8c0 con 0x561ff54a1350 2024-09-21T03:00:58.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ae1f0 con 0x561ff5480670 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441ab430 con 0x561ff5478490 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf509e8740 10 client.0 _ll_register_callbacks cb 0x7fcf3800db40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 0x561ff585b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 0x561ff585b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54310/0 (socket says 172.21.15.31:54310) 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 -- 172.21.15.31:0/3028290071 learned_addr learned my addr 172.21.15.31:0/3028290071 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff58603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff58608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 -- 172.21.15.31:0/3028290071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 msgr2=0x561ff58603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff58603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 -- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 msgr2=0x561ff58608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff58608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 -- 172.21.15.31:0/3028290071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff585b750 con 0x561ff54a1350 2024-09-21T03:00:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf437fe640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 0x561ff585b160 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800d400 tx=0x7fcf38011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1313b84d9b03441f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.092+0000 7fcf237fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf237fe640 1 -- 172.21.15.31:0/3028290071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38027040 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf237fe640 1 -- 172.21.15.31:0/3028290071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7fcf38012ad0 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf34005640 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf237fe640 1 -- 172.21.15.31:0/3028290071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf38012cd0 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 10 client.17460 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ff58d57a0 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf237fe640 1 -- 172.21.15.31:0/3028290071 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf34005640 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf237fe640 1 -- 172.21.15.31:0/3028290071 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcf38004c00 con 0x561ff54a1350 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 10 client.17460 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 20 client.17460 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff5435080 0x561ff58d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 4 client.17460 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:00:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.093+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ff57bc860 con 0x561ff5435080 2024-09-21T03:00:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.094+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff5435080 0x561ff58d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.094+0000 7fcf43fff640 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff5435080 0x561ff58d7f50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f4283387274ad1 server_cookie=64d5e5c11a67dc77 in_seq=0 out_seq=0 2024-09-21T03:00:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.094+0000 7fcf237fe640 10 client.17460 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf237fe640 1 -- 172.21.15.31:0/3028290071 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ff57bc860 con 0x561ff5435080 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf237fe640 10 client.17460 handle_command_reply: tid=0 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 client.17460 shutdown 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 msgr2=0x561ff585b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 0x561ff585b160 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800d400 tx=0x7fcf38011d20 comp rx=0 tx=0).stop 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 shutdown_connections 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x561ff5435080 0x561ff58d7f50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561ff54a1350 0x561ff585b160 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ff5480670 0x561ff58608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 --2- 172.21.15.31:0/3028290071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ff5478490 0x561ff58603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 >> 172.21.15.31:0/3028290071 conn(0x561ff5495740 msgr2=0x561ff5845aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf237fe640 0 client.17460 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 shutdown_connections 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 1 -- 172.21.15.31:0/3028290071 wait complete. 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.096+0000 7fcf509e8740 20 client.17460 trim_cache size 0 max 16384 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "db59a392-c804-4e73-833d-e8691cb9fccb", 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T03:00:58.101 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:00:58.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcf2c0b66b0 msgr2=0x7fcf2c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcf2c0b66b0 0x7fcf2c0b8bc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009270 tx=0x7fcf3c07a480 comp rx=0 tx=0).stop 2024-09-21T03:00:58.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 msgr2=0x7fcf4419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:58.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 0x7fcf4419fce0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00eaa0 tx=0x7fcf3c00ef60 comp rx=0 tx=0).stop 2024-09-21T03:00:58.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 shutdown_connections 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcf2c0b66b0 0x7fcf2c0b8bc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf44109df0 0x7fcf4419fce0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf441094a0 0x7fcf4419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 --2- 172.21.15.31:0/3322030667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf44103b60 0x7fcf4419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 >> 172.21.15.31:0/3322030667 conn(0x7fcf4407b850 msgr2=0x7fcf44107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 shutdown_connections 2024-09-21T03:00:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:58.097+0000 7fcf4a474640 1 -- 172.21.15.31:0/3322030667 wait complete. 2024-09-21T03:00:58.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:00:58.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:58 smithi165 ceph-mon[34818]: pgmap v569: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:58.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:58 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T03:00:58.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:58 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:58.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:58 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T03:00:58.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:00:58 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:58 smithi031 ceph-mon[31209]: pgmap v569: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:58 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T03:00:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:58 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:58 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T03:00:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:00:58 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:59.170 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:58 smithi178 ceph-mon[34759]: pgmap v569: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:00:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:58 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T03:00:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:58 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:58 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T03:00:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:00:58 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 -- 172.21.15.31:0/3775308287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d41094b0 msgr2=0x7f23d41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/3775308287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d41094b0 0x7f23d41098c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009900 tx=0x7f23c802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 -- 172.21.15.31:0/3775308287 shutdown_connections 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/3775308287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d4109e00 0x7f23d4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/3775308287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d41094b0 0x7f23d41098c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/3775308287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4103bc0 0x7f23d4103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 -- 172.21.15.31:0/3775308287 >> 172.21.15.31:0/3775308287 conn(0x7f23d407b910 msgr2=0x7f23d407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 -- 172.21.15.31:0/3775308287 shutdown_connections 2024-09-21T03:00:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.697+0000 7f23d9ff7640 1 -- 172.21.15.31:0/3775308287 wait complete. 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 Processor -- start 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 -- start start 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 0x7f23d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 0x7f23d4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4109e00 0x7f23d4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4113ac0 con 0x7f23d4103bc0 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4113940 con 0x7f23d41094b0 2024-09-21T03:00:59.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d9ff7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4113c40 con 0x7f23d4109e00 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 0x7f23d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 0x7f23d4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 0x7f23d4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:59480/0 (socket says 172.21.15.31:59480) 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 -- 172.21.15.31:0/585308335 learned_addr learned my addr 172.21.15.31:0/585308335 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.698+0000 7f23d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 0x7f23d4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59816/0 (socket says 172.21.15.31:59816) 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d3fff640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4109e00 0x7f23d4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 -- 172.21.15.31:0/585308335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4109e00 msgr2=0x7f23d4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4109e00 0x7f23d4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 -- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 msgr2=0x7f23d4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 0x7f23d4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 -- 172.21.15.31:0/585308335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d41a8730 con 0x7f23d41094b0 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d37fe640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 0x7f23d4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:59.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d3fff640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4109e00 0x7f23d4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 0x7f23d4075c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23c802a930 tx=0x7f23c8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d0ff9640 1 -- 172.21.15.31:0/585308335 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c803f030 con 0x7f23d41094b0 2024-09-21T03:00:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d41a89c0 con 0x7f23d41094b0 2024-09-21T03:00:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d0ff9640 1 -- 172.21.15.31:0/585308335 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c8004490 con 0x7f23d41094b0 2024-09-21T03:00:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.699+0000 7f23d0ff9640 1 -- 172.21.15.31:0/585308335 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8035070 con 0x7f23d41094b0 2024-09-21T03:00:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.700+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41a8ef0 con 0x7f23d41094b0 2024-09-21T03:00:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 Processor -- start 2024-09-21T03:00:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 -- start start 2024-09-21T03:00:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23d0ff9640 1 -- 172.21.15.31:0/585308335 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f23c8042030 con 0x7f23d41094b0 2024-09-21T03:00:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9ebe070 0x5604aa265650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 0x5604aa265b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa2660d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5604aa1dbe50 con 0x5604a9e951b0 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604aa1cb280 con 0x5604a9e9d390 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.702+0000 7f23e056b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604aa1cf090 con 0x5604a9ebe070 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 0x5604aa265b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 0x5604aa265b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59830/0 (socket says 172.21.15.31:59830) 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 -- 172.21.15.31:0/1099596768 learned_addr learned my addr 172.21.15.31:0/1099596768 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d3fff640 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9ebe070 0x5604aa265650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa2660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 -- 172.21.15.31:0/1099596768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9ebe070 msgr2=0x5604aa265650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9ebe070 0x5604aa265650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 -- 172.21.15.31:0/1099596768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 msgr2=0x5604aa2660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa2660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 -- 172.21.15.31:0/1099596768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a9f29650 con 0x5604a9e951b0 2024-09-21T03:00:59.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d3fff640 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9ebe070 0x5604aa265650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:59.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d0ff9640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23bc0b6680 0x7f23bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d37fe640 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 0x5604aa265b90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f23c40114f0 tx=0x7f23c40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ee8a4dbdbd5fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d3fff640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23bc0b6680 0x7f23bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23b9ffb640 1 -- 172.21.15.31:0/1099596768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4004020 con 0x5604a9e951b0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23d0ff9640 1 -- 172.21.15.31:0/585308335 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f23c8107a30 con 0x7f23d41094b0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23b9ffb640 1 -- 172.21.15.31:0/1099596768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c40041c0 con 0x5604a9e951b0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.703+0000 7f23b9ffb640 1 -- 172.21.15.31:0/1099596768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c400f840 con 0x5604a9e951b0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 -- 172.21.15.31:0/1099596768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 msgr2=0x5604aa265b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23d3fff640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23bc0b6680 0x7f23bc0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009fd0 tx=0x7f23cc009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 0x5604aa265b90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f23c40114f0 tx=0x7f23c40119b0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 -- 172.21.15.31:0/1099596768 shutdown_connections 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa2660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e951b0 0x5604aa265b90 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 --2- 172.21.15.31:0/1099596768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9ebe070 0x5604aa265650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 -- 172.21.15.31:0/1099596768 >> 172.21.15.31:0/1099596768 conn(0x5604a9eb2460 msgr2=0x5604aa263540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 -- 172.21.15.31:0/1099596768 shutdown_connections 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.704+0000 7f23e056b740 1 -- 172.21.15.31:0/1099596768 wait complete. 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.705+0000 7f23e056b740 1 -- start start 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.705+0000 7f23e056b740 1 Processor -- start 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa270cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa271230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9ebe070 0x5604aa2717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5604aa02c700 con 0x5604a9ebe070 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604aa166110 con 0x5604a9e9d390 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604aa273e30 con 0x5604a9e951b0 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23e056b740 10 client.0 _ll_register_callbacks cb 0x5604aa17f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9ebe070 0x5604aa2717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9ebe070 0x5604aa2717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59832/0 (socket says 172.21.15.31:59832) 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d3fff640 1 -- 172.21.15.31:0/3464329183 learned_addr learned my addr 172.21.15.31:0/3464329183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa271230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d37fe640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa270cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 -- 172.21.15.31:0/3464329183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 msgr2=0x5604aa270cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa270cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 -- 172.21.15.31:0/3464329183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9ebe070 msgr2=0x5604aa2717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9ebe070 0x5604aa2717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 -- 172.21.15.31:0/3464329183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604aa193c40 con 0x5604a9e9d390 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa271230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23c8105180 tx=0x7f23c810b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcee1db0f60fe3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.706+0000 7f23b2ffd640 1 -- 172.21.15.31:0/3464329183 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8107e60 con 0x5604a9e9d390 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23b2ffd640 1 -- 172.21.15.31:0/3464329183 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f23c80c9040 con 0x5604a9e9d390 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23c40041c0 con 0x5604a9e9d390 2024-09-21T03:00:59.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23b2ffd640 1 -- 172.21.15.31:0/3464329183 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23c8049070 con 0x5604a9e9d390 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 10 client.26807 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604aa27fb00 con 0x5604a9e9d390 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23b2ffd640 1 -- 172.21.15.31:0/3464329183 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c40041c0 con 0x5604a9e9d390 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23b2ffd640 1 -- 172.21.15.31:0/3464329183 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f23c80c9280 con 0x5604a9e9d390 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 10 client.26807 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 20 client.26807 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9e7c9f0 0x5604aa282f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 4 client.26807 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604aa1ddda0 con 0x5604a9e7c9f0 2024-09-21T03:00:59.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.707+0000 7f23d37fe640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9e7c9f0 0x5604aa282f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.708+0000 7f23d37fe640 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9e7c9f0 0x5604aa282f80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13312e6fb730270 server_cookie=107c359ad356225e in_seq=0 out_seq=0 2024-09-21T03:00:59.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.708+0000 7f23b2ffd640 10 client.26807 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23b2ffd640 1 -- 172.21.15.31:0/3464329183 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604aa1ddda0 con 0x5604a9e7c9f0 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23b2ffd640 10 client.26807 handle_command_reply: tid=0 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 client.26807 shutdown 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 msgr2=0x5604aa271230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa271230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23c8105180 tx=0x7f23c810b050 comp rx=0 tx=0).stop 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 shutdown_connections 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9e7c9f0 0x5604aa282f80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9ebe070 0x5604aa2717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa271230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23b2ffd640 0 client.26807 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 --2- 172.21.15.31:0/3464329183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa270cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 >> 172.21.15.31:0/3464329183 conn(0x5604a9eb2460 msgr2=0x5604aa2687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 shutdown_connections 2024-09-21T03:00:59.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 1 -- 172.21.15.31:0/3464329183 wait complete. 2024-09-21T03:00:59.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.709+0000 7f23e056b740 20 client.26807 trim_cache size 0 max 16384 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.726+0000 7f23e056b740 1 Processor -- start 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.726+0000 7f23e056b740 1 -- start start 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23e056b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 0x5604aa2711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23e056b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa262e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23e056b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa26dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23e056b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b18b0 con 0x5604a9e7c9f0 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23e056b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b33f0 con 0x5604a9e9d390 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23e056b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b1f80 con 0x5604a9e951b0 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 0x5604aa2711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 0x5604aa2711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59846/0 (socket says 172.21.15.31:59846) 2024-09-21T03:00:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 -- 172.21.15.31:0/3050900113 learned_addr learned my addr 172.21.15.31:0/3050900113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d3fff640 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa26dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa262e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 -- 172.21.15.31:0/3050900113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 msgr2=0x5604aa262e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa262e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 -- 172.21.15.31:0/3050900113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 msgr2=0x5604aa26dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa26dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 -- 172.21.15.31:0/3050900113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604aa26fc50 con 0x5604a9e7c9f0 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.727+0000 7f23d37fe640 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 0x5604aa2711b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f23c4004610 tx=0x7f23c4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d9451ea72e3ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23b97fa640 1 -- 172.21.15.31:0/3050900113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4009ca0 con 0x5604a9e7c9f0 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23b97fa640 1 -- 172.21.15.31:0/3050900113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c4005570 con 0x5604a9e7c9f0 2024-09-21T03:00:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23b97fa640 1 -- 172.21.15.31:0/3050900113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4003bc0 con 0x5604a9e7c9f0 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 -- 172.21.15.31:0/3050900113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 msgr2=0x5604aa2711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 0x5604aa2711b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f23c4004610 tx=0x7f23c4005850 comp rx=0 tx=0).stop 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 -- 172.21.15.31:0/3050900113 shutdown_connections 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e951b0 0x5604aa262e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e9d390 0x5604aa26dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 --2- 172.21.15.31:0/3050900113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e7c9f0 0x5604aa2711b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 -- 172.21.15.31:0/3050900113 >> 172.21.15.31:0/3050900113 conn(0x5604a9eb2460 msgr2=0x5604aa270aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 -- 172.21.15.31:0/3050900113 shutdown_connections 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.728+0000 7f23e056b740 1 -- 172.21.15.31:0/3050900113 wait complete. 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 -- start start 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 Processor -- start 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e7c9f0 0x5604aa2754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e951b0 0x5604aa275a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 0x5604aa2ec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b2d20 con 0x5604a9e9d390 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b2650 con 0x5604a9e951b0 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41af890 con 0x5604a9e7c9f0 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23e056b740 10 client.0 _ll_register_callbacks cb 0x7f23c8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.729+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 0x5604aa2ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 0x5604aa2ec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59848/0 (socket says 172.21.15.31:59848) 2024-09-21T03:00:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e951b0 0x5604aa275a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d3fff640 1 -- 172.21.15.31:0/1243373762 learned_addr learned my addr 172.21.15.31:0/1243373762 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d37fe640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e7c9f0 0x5604aa2754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 -- 172.21.15.31:0/1243373762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e7c9f0 msgr2=0x5604aa2754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e7c9f0 0x5604aa2754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 -- 172.21.15.31:0/1243373762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 msgr2=0x5604aa2ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 0x5604aa2ec940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 -- 172.21.15.31:0/1243373762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604aa2ece80 con 0x5604a9e951b0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d3fff640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 0x5604aa2ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23d2ffd640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e951b0 0x5604aa275a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2814c157b4f05634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23b2ffd640 1 -- 172.21.15.31:0/1243373762 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c80b8060 con 0x5604a9e951b0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23b2ffd640 1 -- 172.21.15.31:0/1243373762 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8528+0+0 (secure 0 0 0) 0x7f23c80c9040 con 0x5604a9e951b0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23c4005570 con 0x5604a9e951b0 2024-09-21T03:00:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23b2ffd640 1 -- 172.21.15.31:0/1243373762 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23c807a050 con 0x5604a9e951b0 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.730+0000 7f23e056b740 10 client.26813 fetch_fsmap learned FSMap version 27 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604aa2edfe0 con 0x5604a9e951b0 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23b2ffd640 1 -- 172.21.15.31:0/1243373762 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c4005570 con 0x5604a9e951b0 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23b2ffd640 1 -- 172.21.15.31:0/1243373762 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f23c8031ac0 con 0x5604a9e951b0 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 10 client.26813 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 20 client.26813 populate_metadata read hostname 'smithi031' 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9ebe070 0x5604aa2f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 4 client.26813 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604aa1d9850 con 0x5604a9ebe070 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.731+0000 7f23d37fe640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9ebe070 0x5604aa2f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23d37fe640 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9ebe070 0x5604aa2f05d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8d6e94fe799eae server_cookie=c2f1451a501d9ff7 in_seq=0 out_seq=0 2024-09-21T03:00:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23b2ffd640 10 client.26813 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23b2ffd640 1 -- 172.21.15.31:0/1243373762 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x5604aa1d9850 con 0x5604a9ebe070 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23b2ffd640 10 client.26813 handle_command_reply: tid=0 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 client.26813 shutdown 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e951b0 msgr2=0x5604aa275a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e951b0 0x5604aa275a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c8009620 comp rx=0 tx=0).stop 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 shutdown_connections 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604a9ebe070 0x5604aa2f05d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604a9e9d390 0x5604aa2ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604a9e951b0 0x5604aa275a70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23b2ffd640 0 client.26813 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 --2- 172.21.15.31:0/1243373762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604a9e7c9f0 0x5604aa2754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 >> 172.21.15.31:0/1243373762 conn(0x5604a9eb2460 msgr2=0x5604aa266680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 shutdown_connections 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 1 -- 172.21.15.31:0/1243373762 wait complete. 2024-09-21T03:00:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.732+0000 7f23e056b740 20 client.26813 trim_cache size 0 max 16384 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "db59a392-c804-4e73-833d-e8691cb9fccb", 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:00:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:58.094612+0000", 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:00:59.742 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.743 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:00:59.746 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:59.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23bc0b6680 msgr2=0x7f23bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23bc0b6680 0x7f23bc0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009fd0 tx=0x7f23cc009500 comp rx=0 tx=0).stop 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 msgr2=0x7f23d4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 0x7f23d4075c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23c802a930 tx=0x7f23c8031a20 comp rx=0 tx=0).stop 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 shutdown_connections 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f23bc0b6680 0x7f23bc0b8b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23d4109e00 0x7f23d4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23d41094b0 0x7f23d4075c50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 --2- 172.21.15.31:0/585308335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d4103bc0 0x7f23d4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 >> 172.21.15.31:0/585308335 conn(0x7f23d407b910 msgr2=0x7f23d4105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 shutdown_connections 2024-09-21T03:00:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:00:59.733+0000 7f23d9ff7640 1 -- 172.21.15.31:0/585308335 wait complete. 2024-09-21T03:01:00.595 INFO:tasks.cephfs.filesystem:scrub status for tag:db59a392-c804-4e73-833d-e8691cb9fccb - {'path': '/', 'tag': 'db59a392-c804-4e73-833d-e8691cb9fccb', 'options': 'recursive,force'} 2024-09-21T03:01:00.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:00 smithi031 ceph-mon[31209]: pgmap v570: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:00.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:00 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T03:01:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:00 smithi165 ceph-mon[34818]: pgmap v570: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:00 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T03:01:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:00 smithi178 ceph-mon[34759]: pgmap v570: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:00 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T03:01:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:02 smithi031 ceph-mon[31209]: pgmap v571: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:01:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:02 smithi165 ceph-mon[34818]: pgmap v571: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:01:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:02 smithi178 ceph-mon[34759]: pgmap v571: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:01:05.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:04 smithi031 ceph-mon[31209]: pgmap v572: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:01:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:04 smithi165 ceph-mon[34818]: pgmap v572: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:01:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:04 smithi178 ceph-mon[34759]: pgmap v572: 97 pgs: 97 active+clean; 259 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:01:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:06 smithi031 ceph-mon[31209]: pgmap v573: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:01:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:06 smithi165 ceph-mon[34818]: pgmap v573: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:01:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:06 smithi178 ceph-mon[34759]: pgmap v573: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:01:09.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:08 smithi031 ceph-mon[31209]: pgmap v574: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:01:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:08 smithi165 ceph-mon[34818]: pgmap v574: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:01:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:08 smithi178 ceph-mon[34759]: pgmap v574: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:01:11.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:10 smithi031 ceph-mon[31209]: pgmap v575: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:01:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:10 smithi165 ceph-mon[34818]: pgmap v575: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:01:11.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:10 smithi178 ceph-mon[34759]: pgmap v575: 97 pgs: 97 active+clean; 259 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:01:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:11 smithi031 ceph-mon[31209]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:01:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:11 smithi165 ceph-mon[34818]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:01:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:11 smithi178 ceph-mon[34759]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:01:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:12 smithi031 ceph-mon[31209]: pgmap v577: 97 pgs: 97 active+clean; 226 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:01:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:12 smithi165 ceph-mon[34818]: pgmap v577: 97 pgs: 97 active+clean; 226 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:01:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:12 smithi178 ceph-mon[34759]: pgmap v577: 97 pgs: 97 active+clean; 226 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:01:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:14 smithi031 ceph-mon[31209]: pgmap v578: 97 pgs: 97 active+clean; 178 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:01:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:14 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:01:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:14 smithi165 ceph-mon[34818]: pgmap v578: 97 pgs: 97 active+clean; 178 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:01:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:14 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:01:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:14 smithi178 ceph-mon[34759]: pgmap v578: 97 pgs: 97 active+clean; 178 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:01:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:14 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:01:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:01:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:01:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:01:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:01:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:01:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:01:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:01:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:01:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:01:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:16 smithi031 ceph-mon[31209]: pgmap v579: 97 pgs: 97 active+clean; 10 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:16 smithi165 ceph-mon[34818]: pgmap v579: 97 pgs: 97 active+clean; 10 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:16 smithi178 ceph-mon[34759]: pgmap v579: 97 pgs: 97 active+clean; 10 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:18 smithi031 ceph-mon[31209]: pgmap v580: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:18 smithi165 ceph-mon[34818]: pgmap v580: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:18 smithi178 ceph-mon[34759]: pgmap v580: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:21.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:20 smithi031 ceph-mon[31209]: pgmap v581: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:20 smithi165 ceph-mon[34818]: pgmap v581: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:21.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:20 smithi178 ceph-mon[34759]: pgmap v581: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:01:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:22 smithi165 ceph-mon[34818]: pgmap v582: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 193 B/s wr, 10 op/s 2024-09-21T03:01:22.941 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:22 smithi165 ceph-mon[34818]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:01:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:22 smithi178 ceph-mon[34759]: pgmap v582: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 193 B/s wr, 10 op/s 2024-09-21T03:01:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:22 smithi178 ceph-mon[34759]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:01:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:22 smithi031 ceph-mon[31209]: pgmap v582: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 193 B/s wr, 10 op/s 2024-09-21T03:01:23.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:22 smithi031 ceph-mon[31209]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:01:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:24 smithi178 ceph-mon[34759]: pgmap v584: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:01:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:24 smithi031 ceph-mon[31209]: pgmap v584: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:01:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:24 smithi165 ceph-mon[34818]: pgmap v584: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:01:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:26 smithi178 ceph-mon[34759]: pgmap v585: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:01:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:26 smithi031 ceph-mon[31209]: pgmap v585: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:01:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:26 smithi165 ceph-mon[34818]: pgmap v585: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:01:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:28 smithi178 ceph-mon[34759]: pgmap v586: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:28.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:28 smithi165 ceph-mon[34818]: pgmap v586: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:28 smithi031 ceph-mon[31209]: pgmap v586: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:30.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:01:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:30 smithi178 ceph-mon[34759]: pgmap v587: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:30.991 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:30 smithi031 ceph-mon[31209]: pgmap v587: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:30 smithi165 ceph-mon[34818]: pgmap v587: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 -- 172.21.15.31:0/2452327335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff210108cf0 msgr2=0x7ff21010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 --2- 172.21.15.31:0/2452327335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff210108cf0 0x7ff21010f520 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff208009ca0 tx=0x7ff20802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 -- 172.21.15.31:0/2452327335 shutdown_connections 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 --2- 172.21.15.31:0/2452327335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff210108cf0 0x7ff21010f520 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 --2- 172.21.15.31:0/2452327335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2101083a0 0x7ff2101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 --2- 172.21.15.31:0/2452327335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff210102a60 0x7ff210102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 -- 172.21.15.31:0/2452327335 >> 172.21.15.31:0/2452327335 conn(0x7ff210079730 msgr2=0x7ff210079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 -- 172.21.15.31:0/2452327335 shutdown_connections 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.383+0000 7ff2161c5640 1 -- 172.21.15.31:0/2452327335 wait complete. 2024-09-21T03:01:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 Processor -- start 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 -- start start 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff210102a60 0x7ff21010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 0x7ff2101104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff210108cf0 0x7ff210110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff210113a80 con 0x7ff2101083a0 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff210113900 con 0x7ff210108cf0 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff2161c5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff210113c00 con 0x7ff210102a60 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff207fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 0x7ff2101104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff207fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 0x7ff2101104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42232/0 (socket says 172.21.15.31:42232) 2024-09-21T03:01:31.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff207fff640 1 -- 172.21.15.31:0/1099270423 learned_addr learned my addr 172.21.15.31:0/1099270423 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.384+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff210102a60 0x7ff21010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff20ffff640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff210108cf0 0x7ff210110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff207fff640 1 -- 172.21.15.31:0/1099270423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff210102a60 msgr2=0x7ff21010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff207fff640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff210102a60 0x7ff21010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff207fff640 1 -- 172.21.15.31:0/1099270423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff210108cf0 msgr2=0x7ff210110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff207fff640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff210108cf0 0x7ff210110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff207fff640 1 -- 172.21.15.31:0/1099270423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2101a8750 con 0x7ff2101083a0 2024-09-21T03:01:31.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff207fff640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 0x7ff2101104f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800e9c0 tx=0x7ff1f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff20d7fa640 1 -- 172.21.15.31:0/1099270423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800cd80 con 0x7ff2101083a0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2101a8a40 con 0x7ff2101083a0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff20d7fa640 1 -- 172.21.15.31:0/1099270423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f80054d0 con 0x7ff2101083a0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff20d7fa640 1 -- 172.21.15.31:0/1099270423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8010430 con 0x7ff2101083a0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.385+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2101a8fd0 con 0x7ff2101083a0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.388+0000 7ff20d7fa640 1 -- 172.21.15.31:0/1099270423 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff1f8004fd0 con 0x7ff2101083a0 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff20d7fa640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1fc0b6680 0x7ff1fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 Processor -- start 2024-09-21T03:01:31.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 -- start start 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1fc0b6680 0x7ff1fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff20d7fa640 1 -- 172.21.15.31:0/1099270423 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7ff1f80e7bc0 con 0x7ff2101083a0 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 0x564e80c98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80c987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c98cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564e80c0ee50 con 0x564e808f1070 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e80bfe280 con 0x564e808d0390 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.389+0000 7ff21c739740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e80c02090 con 0x564e808c81b0 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 0x564e80c98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 0x564e80c98270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42242/0 (socket says 172.21.15.31:42242) 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 -- 172.21.15.31:0/995251162 learned_addr learned my addr 172.21.15.31:0/995251162 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20ffff640 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c98cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1fc0b6680 0x7ff1fc0b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff210111a00 tx=0x7ff200005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff207fff640 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80c987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 -- 172.21.15.31:0/995251162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 msgr2=0x564e80c987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80c987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 -- 172.21.15.31:0/995251162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 msgr2=0x564e80c98cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c98cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 -- 172.21.15.31:0/995251162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8095c650 con 0x564e808f1070 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 0x564e80c98270 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e620 tx=0x7ff200014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1e59e586ca23d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff205ffb640 1 -- 172.21.15.31:0/995251162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200026430 con 0x564e808f1070 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff205ffb640 1 -- 172.21.15.31:0/995251162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20001d8c0 con 0x564e808f1070 2024-09-21T03:01:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.390+0000 7ff205ffb640 1 -- 172.21.15.31:0/995251162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200025890 con 0x564e808f1070 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 -- 172.21.15.31:0/995251162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 msgr2=0x564e80c98270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 0x564e80c98270 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e620 tx=0x7ff200014000 comp rx=0 tx=0).stop 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 -- 172.21.15.31:0/995251162 shutdown_connections 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80c987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c98cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 --2- 172.21.15.31:0/995251162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808f1070 0x564e80c98270 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 -- 172.21.15.31:0/995251162 >> 172.21.15.31:0/995251162 conn(0x564e808e5460 msgr2=0x564e80c96160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.391+0000 7ff21c739740 1 -- 172.21.15.31:0/995251162 shutdown_connections 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.392+0000 7ff21c739740 1 -- 172.21.15.31:0/995251162 wait complete. 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.392+0000 7ff21c739740 1 -- start start 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.392+0000 7ff21c739740 1 Processor -- start 2024-09-21T03:01:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80cac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cacb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564e80a5f700 con 0x564e808d0390 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e80b99110 con 0x564e808f1070 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e80caf180 con 0x564e808c81b0 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff21c739740 10 client.0 _ll_register_callbacks cb 0x564e80bb2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42254/0 (socket says 172.21.15.31:42254) 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 -- 172.21.15.31:0/895029455 learned_addr learned my addr 172.21.15.31:0/895029455 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80cac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff20ffff640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cacb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 -- 172.21.15.31:0/895029455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 msgr2=0x564e80cac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80cac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 -- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 msgr2=0x564e80cacb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cacb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.393+0000 7ff207fff640 1 -- 172.21.15.31:0/895029455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e80bc6c40 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff207fff640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cac5f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80ad9a0 tx=0x7ff1f80ad3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821c5fbd2133b4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff1eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff1eeffd640 1 -- 172.21.15.31:0/895029455 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80046c0 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff1eeffd640 1 -- 172.21.15.31:0/895029455 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7ff1f80b21e0 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff20001d8c0 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff1eeffd640 1 -- 172.21.15.31:0/895029455 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1f80ad520 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff21c739740 10 client.17490 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e80cbaec0 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.394+0000 7ff1eeffd640 1 -- 172.21.15.31:0/895029455 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20001d8c0 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff1eeffd640 1 -- 172.21.15.31:0/895029455 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1f80ac6e0 con 0x564e808d0390 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff21c739740 10 client.17490 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff21c739740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff21c739740 20 client.17490 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff21c739740 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e808af9f0 0x564e80cbe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff21c739740 4 client.17490 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e80c10da0 con 0x564e808af9f0 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.395+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e808af9f0 0x564e80cbe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.396+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e808af9f0 0x564e80cbe220 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a4f4092890d92f server_cookie=356a1b4d3ca546fe in_seq=0 out_seq=0 2024-09-21T03:01:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.396+0000 7ff1eeffd640 10 client.17490 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.396+0000 7ff1eeffd640 1 -- 172.21.15.31:0/895029455 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e80c10da0 con 0x564e808af9f0 2024-09-21T03:01:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.396+0000 7ff1eeffd640 10 client.17490 handle_command_reply: tid=0 2024-09-21T03:01:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.396+0000 7ff21c739740 1 client.17490 shutdown 2024-09-21T03:01:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 msgr2=0x564e80cac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cac5f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80ad9a0 tx=0x7ff1f80ad3d0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 shutdown_connections 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e808af9f0 0x564e80cbe220 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cacb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cac5f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 --2- 172.21.15.31:0/895029455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808c81b0 0x564e80cac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 >> 172.21.15.31:0/895029455 conn(0x564e808e5460 msgr2=0x564e80c94700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff1eeffd640 0 client.17490 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 shutdown_connections 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 1 -- 172.21.15.31:0/895029455 wait complete. 2024-09-21T03:01:31.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.397+0000 7ff21c739740 20 client.17490 trim_cache size 0 max 16384 2024-09-21T03:01:31.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 Processor -- start 2024-09-21T03:01:31.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 -- start start 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 0x564e80cac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c95a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808f1070 0x564e80c95fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ad630 con 0x564e808c81b0 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101af170 con 0x564e808d0390 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff21c739740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101add00 con 0x564e808f1070 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff20f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 0x564e80cac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff20f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 0x564e80cac570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42260/0 (socket says 172.21.15.31:42260) 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff20f7fe640 1 -- 172.21.15.31:0/398762086 learned_addr learned my addr 172.21.15.31:0/398762086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:31.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff207fff640 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.415+0000 7ff20ffff640 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808f1070 0x564e80c95fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20f7fe640 1 -- 172.21.15.31:0/398762086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808f1070 msgr2=0x564e80c95fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808f1070 0x564e80c95fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20f7fe640 1 -- 172.21.15.31:0/398762086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 msgr2=0x564e80c95a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c95a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20f7fe640 1 -- 172.21.15.31:0/398762086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e80cad370 con 0x564e808c81b0 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20ffff640 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808f1070 0x564e80c95fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 0x564e80cac570 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff200025440 tx=0x7ff200016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df23f609e792e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff2057fa640 1 -- 172.21.15.31:0/398762086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000ed50 con 0x564e808c81b0 2024-09-21T03:01:31.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff2057fa640 1 -- 172.21.15.31:0/398762086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff200005570 con 0x564e808c81b0 2024-09-21T03:01:31.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.416+0000 7ff2057fa640 1 -- 172.21.15.31:0/398762086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000035f0 con 0x564e808c81b0 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 -- 172.21.15.31:0/398762086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 msgr2=0x564e80cac570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 0x564e80cac570 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff200025440 tx=0x7ff200016000 comp rx=0 tx=0).stop 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 -- 172.21.15.31:0/398762086 shutdown_connections 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e808f1070 0x564e80c95fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808d0390 0x564e80c95a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 --2- 172.21.15.31:0/398762086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808c81b0 0x564e80cac570 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 -- 172.21.15.31:0/398762086 >> 172.21.15.31:0/398762086 conn(0x564e808e5460 msgr2=0x564e80cabe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 -- 172.21.15.31:0/398762086 shutdown_connections 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.417+0000 7ff21c739740 1 -- 172.21.15.31:0/398762086 wait complete. 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 -- start start 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 Processor -- start 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cb4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cb0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e80884da0 0x564e80cb0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101aeaa0 con 0x564e808d0390 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ae3d0 con 0x564e808f1070 2024-09-21T03:01:31.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101ab610 con 0x564e80884da0 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff21c739740 10 client.0 _ll_register_callbacks cb 0x7ff1f80aa0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff20f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cb4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff20f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cb4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42266/0 (socket says 172.21.15.31:42266) 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.418+0000 7ff20f7fe640 1 -- 172.21.15.31:0/4064583185 learned_addr learned my addr 172.21.15.31:0/4064583185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20ffff640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e80884da0 0x564e80cb0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff207fff640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cb0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20f7fe640 1 -- 172.21.15.31:0/4064583185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e80884da0 msgr2=0x564e80cb0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e80884da0 0x564e80cb0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20f7fe640 1 -- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 msgr2=0x564e80cb0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cb0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20f7fe640 1 -- 172.21.15.31:0/4064583185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e80d27bc0 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff20f7fe640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cb4dd0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e5f0 tx=0x7ff20000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae458c07f6f89c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff1eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff1eeffd640 1 -- 172.21.15.31:0/4064583185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20003e020 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff1eeffd640 1 -- 172.21.15.31:0/4064583185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7ff2000050d0 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff200005570 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.419+0000 7ff1eeffd640 1 -- 172.21.15.31:0/4064583185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff200049050 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 10 client.17496 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e80d28cf0 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff1eeffd640 1 -- 172.21.15.31:0/4064583185 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff200005570 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff1eeffd640 1 -- 172.21.15.31:0/4064583185 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff200003890 con 0x564e808d0390 2024-09-21T03:01:31.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 10 client.17496 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 20 client.17496 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e80d290d0 0x564e80d2b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 4 client.17496 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e80c0c850 con 0x564e80d290d0 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.420+0000 7ff207fff640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e80d290d0 0x564e80d2b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff207fff640 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e80d290d0 0x564e80d2b4f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b4e80af70a783d server_cookie=6ec39dd69e521d5d in_seq=0 out_seq=0 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff1eeffd640 10 client.17496 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff1eeffd640 1 -- 172.21.15.31:0/4064583185 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x564e80c0c850 con 0x564e80d290d0 2024-09-21T03:01:31.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff1eeffd640 10 client.17496 handle_command_reply: tid=0 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff21c739740 1 client.17496 shutdown 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 msgr2=0x564e80cb4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.421+0000 7ff21c739740 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cb4dd0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff20000e5f0 tx=0x7ff20000ee00 comp rx=0 tx=0).stop 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 shutdown_connections 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x564e80d290d0 0x564e80d2b4f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff1eeffd640 0 client.17496 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e80884da0 0x564e80cb0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e808f1070 0x564e80cb0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 --2- 172.21.15.31:0/4064583185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e808d0390 0x564e80cb4dd0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 >> 172.21.15.31:0/4064583185 conn(0x564e808e5460 msgr2=0x564e80c959b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 shutdown_connections 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 1 -- 172.21.15.31:0/4064583185 wait complete. 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff21c739740 20 client.17496 trim_cache size 0 max 16384 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T03:01:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:58.094612+0000", 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:01:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:01:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:01:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:01:31.440 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1fc0b6680 msgr2=0x7ff1fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1fc0b6680 0x7ff1fc0b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff210111a00 tx=0x7ff200005f90 comp rx=0 tx=0).stop 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 msgr2=0x7ff2101104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 0x7ff2101104f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800e9c0 tx=0x7ff1f800ee80 comp rx=0 tx=0).stop 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 shutdown_connections 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff1fc0b6680 0x7ff1fc0b8b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff210108cf0 0x7ff210110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2101083a0 0x7ff2101104f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.422+0000 7ff2161c5640 1 --2- 172.21.15.31:0/1099270423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff210102a60 0x7ff21010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.423+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 >> 172.21.15.31:0/1099270423 conn(0x7ff210079730 msgr2=0x7ff210104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.423+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 shutdown_connections 2024-09-21T03:01:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:31.423+0000 7ff2161c5640 1 -- 172.21.15.31:0/1099270423 wait complete. 2024-09-21T03:01:32.205 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:01:32.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:01:32.468 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:32.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:32 smithi031 ceph-mon[31209]: pgmap v588: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:32 smithi178 ceph-mon[34759]: pgmap v588: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.965+0000 7f609fe52640 1 -- 172.21.15.31:0/2931328857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 msgr2=0x7f6098102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.965+0000 7f609fe52640 1 --2- 172.21.15.31:0/2931328857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f6098102ef0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6094009900 tx=0x7f609402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.965+0000 7f609fe52640 1 -- 172.21.15.31:0/2931328857 shutdown_connections 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.965+0000 7f609fe52640 1 --2- 172.21.15.31:0/2931328857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60981083d0 0x7f60981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.965+0000 7f609fe52640 1 --2- 172.21.15.31:0/2931328857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 0x7f609810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.965+0000 7f609fe52640 1 --2- 172.21.15.31:0/2931328857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f6098102ef0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.966+0000 7f609fe52640 1 -- 172.21.15.31:0/2931328857 >> 172.21.15.31:0/2931328857 conn(0x7f60980797b0 msgr2=0x7f6098079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.966+0000 7f609fe52640 1 -- 172.21.15.31:0/2931328857 shutdown_connections 2024-09-21T03:01:32.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.966+0000 7f609fe52640 1 -- 172.21.15.31:0/2931328857 wait complete. 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.966+0000 7f609fe52640 1 Processor -- start 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.966+0000 7f609fe52640 1 -- start start 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609fe52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f60981993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609fe52640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60981083d0 0x7f6098199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609fe52640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 0x7f609819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609fe52640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60981129e0 con 0x7f6098102ae0 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609fe52640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112860 con 0x7f6098108d20 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609fe52640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112b60 con 0x7f60981083d0 2024-09-21T03:01:32.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f60981993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f60981993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42286/0 (socket says 172.21.15.31:42286) 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 -- 172.21.15.31:0/1974507178 learned_addr learned my addr 172.21.15.31:0/1974507178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609d3ed640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60981083d0 0x7f6098199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609e3ef640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 0x7f609819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 -- 172.21.15.31:0/1974507178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60981083d0 msgr2=0x7f6098199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60981083d0 0x7f6098199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 -- 172.21.15.31:0/1974507178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 msgr2=0x7f609819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 0x7f609819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609dbee640 1 -- 172.21.15.31:0/1974507178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609819f260 con 0x7f6098102ae0 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.967+0000 7f609e3ef640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 0x7f609819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.968+0000 7f609dbee640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f60981993b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6094006010 tx=0x7f60940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:32.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.968+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609403f030 con 0x7f6098102ae0 2024-09-21T03:01:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.968+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609819f4f0 con 0x7f6098102ae0 2024-09-21T03:01:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.968+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6094003800 con 0x7f6098102ae0 2024-09-21T03:01:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.968+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094035070 con 0x7f6098102ae0 2024-09-21T03:01:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.968+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609819fa20 con 0x7f6098102ae0 2024-09-21T03:01:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.969+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6098102ef0 con 0x7f6098102ae0 2024-09-21T03:01:32.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.971+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6094003cb0 con 0x7f6098102ae0 2024-09-21T03:01:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.971+0000 7f608effd640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60840b66b0 0x7f60840b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.971+0000 7f609d3ed640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60840b66b0 0x7f60840b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.972+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f60941083a0 con 0x7f6098102ae0 2024-09-21T03:01:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.972+0000 7f609d3ed640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60840b66b0 0x7f60840b8bc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6088005e10 tx=0x7f6088005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:32.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:32.975+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f609819fa20 con 0x7f6098102ae0 2024-09-21T03:01:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:32 smithi165 ceph-mon[34818]: pgmap v588: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.201+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f609819a4e0 con 0x7f6098102ae0 2024-09-21T03:01:33.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.202+0000 7f608effd640 1 -- 172.21.15.31:0/1974507178 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f60940ce8b0 con 0x7f6098102ae0 2024-09-21T03:01:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:01:33.204 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:01:33.204 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60840b66b0 msgr2=0x7f60840b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60840b66b0 0x7f60840b8bc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6088005e10 tx=0x7f6088005da0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 msgr2=0x7f60981993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f60981993b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6094006010 tx=0x7f60940042f0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 shutdown_connections 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f60840b66b0 0x7f60840b8bc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60981083d0 0x7f6098199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6098108d20 0x7f609819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 --2- 172.21.15.31:0/1974507178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6098102ae0 0x7f60981993b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 >> 172.21.15.31:0/1974507178 conn(0x7f60980797b0 msgr2=0x7f6098106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 shutdown_connections 2024-09-21T03:01:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:33.204+0000 7f609fe52640 1 -- 172.21.15.31:0/1974507178 wait complete. 2024-09-21T03:01:33.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:33 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1974507178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:01:33.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:01:33.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:33 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1974507178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:01:34.126 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:34.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:33 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1974507178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:01:34.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 -- 172.21.15.31:0/562170757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8108d20 msgr2=0x7fe5a810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 --2- 172.21.15.31:0/562170757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8108d20 0x7fe5a810f550 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009ca0 tx=0x7fe5a002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 -- 172.21.15.31:0/562170757 shutdown_connections 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 --2- 172.21.15.31:0/562170757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8108d20 0x7fe5a810f550 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 --2- 172.21.15.31:0/562170757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a81083d0 0x7fe5a81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 --2- 172.21.15.31:0/562170757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8102ae0 0x7fe5a8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 -- 172.21.15.31:0/562170757 >> 172.21.15.31:0/562170757 conn(0x7fe5a8079760 msgr2=0x7fe5a8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.628+0000 7fe5ac933640 1 -- 172.21.15.31:0/562170757 shutdown_connections 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.629+0000 7fe5ac933640 1 -- 172.21.15.31:0/562170757 wait complete. 2024-09-21T03:01:34.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.629+0000 7fe5ac933640 1 Processor -- start 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.629+0000 7fe5ac933640 1 -- start start 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5ac933640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 0x7fe5a819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5ac933640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a81083d0 0x7fe5a819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5ac933640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a8108d20 0x7fe5a819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5ac933640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8112990 con 0x7fe5a8102ae0 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5ac933640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8112810 con 0x7fe5a8108d20 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5ac933640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8112b10 con 0x7fe5a81083d0 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 0x7fe5a819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 0x7fe5a819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42302/0 (socket says 172.21.15.31:42302) 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 -- 172.21.15.31:0/940552312 learned_addr learned my addr 172.21.15.31:0/940552312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:34.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a81083d0 0x7fe5a819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a8108d20 0x7fe5a819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 -- 172.21.15.31:0/940552312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a81083d0 msgr2=0x7fe5a819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a81083d0 0x7fe5a819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 -- 172.21.15.31:0/940552312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a8108d20 msgr2=0x7fe5a819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a8108d20 0x7fe5a819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 -- 172.21.15.31:0/940552312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a0360 con 0x7fe5a8102ae0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a8108d20 0x7fe5a819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.630+0000 7fe5a659c640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 0x7fe5a819a3f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.631+0000 7fe5977fe640 1 -- 172.21.15.31:0/940552312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800cd80 con 0x7fe5a8102ae0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.631+0000 7fe5977fe640 1 -- 172.21.15.31:0/940552312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5980054d0 con 0x7fe5a8102ae0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.631+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a0650 con 0x7fe5a8102ae0 2024-09-21T03:01:34.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.631+0000 7fe5977fe640 1 -- 172.21.15.31:0/940552312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598010690 con 0x7fe5a8102ae0 2024-09-21T03:01:34.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.631+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81a6db0 con 0x7fe5a8102ae0 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.633+0000 7fe5b2ea7740 1 Processor -- start 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 -- start start 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5977fe640 1 -- 172.21.15.31:0/940552312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe598010830 con 0x7fe5a8102ae0 2024-09-21T03:01:34.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650bd958ea0 con 0x5650bd61a390 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650bd9482d0 con 0x5650bd6121b0 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5650bd94c0e0 con 0x5650bd63b070 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9e3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42310/0 (socket says 172.21.15.31:42310) 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/312548339 learned_addr learned my addr 172.21.15.31:0/312548339 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a659c640 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/312548339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 msgr2=0x5650bd9e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/312548339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 msgr2=0x5650bd9e2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9e2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.634+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/312548339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bd6a6650 con 0x5650bd61a390 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9e3120 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009c60 tx=0x7fe59c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4932468ee2a4e404 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe594ff9640 1 -- 172.21.15.31:0/312548339 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004230 con 0x5650bd61a390 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe5977fe640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5900b6680 0x7fe5900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe594ff9640 1 -- 172.21.15.31:0/312548339 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c0043d0 con 0x5650bd61a390 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe594ff9640 1 -- 172.21.15.31:0/312548339 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004c80 con 0x5650bd61a390 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5900b6680 0x7fe5900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.635+0000 7fe5977fe640 1 -- 172.21.15.31:0/940552312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fe5980e7c20 con 0x7fe5a8102ae0 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/312548339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 msgr2=0x5650bd9e3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5900b6680 0x7fe5900b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004af0 tx=0x7fe5a0005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9e3120 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009c60 tx=0x7fe59c011810 comp rx=0 tx=0).stop 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/312548339 shutdown_connections 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9e3120 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/312548339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/312548339 >> 172.21.15.31:0/312548339 conn(0x5650bd62f460 msgr2=0x5650bd9e0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/312548339 shutdown_connections 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/312548339 wait complete. 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.636+0000 7fe5b2ea7740 1 -- start start 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 Processor -- start 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd6121b0 0x5650bd9edca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 0x5650bd9ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650bd7a9700 con 0x5650bd63b070 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650bd8e3160 con 0x5650bd61a390 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5650bd9f0e80 con 0x5650bd6121b0 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5b2ea7740 10 client.0 _ll_register_callbacks cb 0x5650bd8fc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 0x5650bd9ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd6121b0 0x5650bd9edca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 0x5650bd9ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42324/0 (socket says 172.21.15.31:42324) 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5a6d9d640 1 -- 172.21.15.31:0/3465159992 learned_addr learned my addr 172.21.15.31:0/3465159992 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.637+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a659c640 1 -- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 msgr2=0x5650bd9ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a659c640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a659c640 1 -- 172.21.15.31:0/3465159992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 msgr2=0x5650bd9ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a659c640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 0x5650bd9ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a659c640 1 -- 172.21.15.31:0/3465159992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bd910c90 con 0x5650bd6121b0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 0x5650bd9ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5a659c640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd6121b0 0x5650bd9edca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5980e5370 tx=0x7fe5980eb240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8ec62cf0b214cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe58dffb640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe58dffb640 1 -- 172.21.15.31:0/3465159992 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980a9040 con 0x5650bd6121b0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c0043d0 con 0x5650bd6121b0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe58dffb640 1 -- 172.21.15.31:0/3465159992 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fe5980aced0 con 0x5650bd6121b0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe58dffb640 1 -- 172.21.15.31:0/3465159992 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5980ad840 con 0x5650bd6121b0 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5b2ea7740 10 client.27013 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:34.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650bd9fcb60 con 0x5650bd6121b0 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.638+0000 7fe58dffb640 1 -- 172.21.15.31:0/3465159992 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c0043d0 con 0x5650bd6121b0 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe58dffb640 1 -- 172.21.15.31:0/3465159992 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5980a93c0 con 0x5650bd6121b0 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5b2ea7740 10 client.27013 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5b2ea7740 10 client.27013 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5b2ea7740 20 client.27013 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5f99f0 0x5650bd9fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5b2ea7740 4 client.27013 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650bd95adf0 con 0x5650bd5f99f0 2024-09-21T03:01:34.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.639+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5f99f0 0x5650bd9fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5f99f0 0x5650bd9fff70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9bfcfa8a0b577570 server_cookie=cf74ba1b32fd0217 in_seq=0 out_seq=0 2024-09-21T03:01:34.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe58dffb640 10 client.27013 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:01:34.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe58dffb640 1 -- 172.21.15.31:0/3465159992 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650bd95adf0 con 0x5650bd5f99f0 2024-09-21T03:01:34.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe58dffb640 10 client.27013 handle_command_reply: tid=0 2024-09-21T03:01:34.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe5b2ea7740 1 client.27013 shutdown 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd6121b0 msgr2=0x5650bd9edca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd6121b0 0x5650bd9edca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5980e5370 tx=0x7fe5980eb240 comp rx=0 tx=0).stop 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.640+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 shutdown_connections 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd63b070 0x5650bd9ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5f99f0 0x5650bd9fff70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/3465159992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd6121b0 0x5650bd9edca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe58dffb640 0 client.27013 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 >> 172.21.15.31:0/3465159992 conn(0x5650bd62f460 msgr2=0x5650bd9e5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 shutdown_connections 2024-09-21T03:01:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/3465159992 wait complete. 2024-09-21T03:01:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.641+0000 7fe5b2ea7740 20 client.27013 trim_cache size 0 max 16384 2024-09-21T03:01:34.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 Processor -- start 2024-09-21T03:01:34.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 -- start start 2024-09-21T03:01:34.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 0x5650bd9efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81af700 con 0x5650bd6121b0 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b1240 con 0x5650bd61a390 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81afdd0 con 0x5650bd63b070 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.654+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 0x5650bd9efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 0x5650bd9efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42330/0 (socket says 172.21.15.31:42330) 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 -- 172.21.15.31:0/1934539647 learned_addr learned my addr 172.21.15.31:0/1934539647 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 -- 172.21.15.31:0/1934539647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 msgr2=0x5650bd9ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 -- 172.21.15.31:0/1934539647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 msgr2=0x5650bd9e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 -- 172.21.15.31:0/1934539647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bd9ec1a0 con 0x5650bd6121b0 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe5a659c640 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 0x5650bd9efb80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe5980ad770 tx=0x7fe5980e7f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cbed539d66d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe58ffff640 1 -- 172.21.15.31:0/1934539647 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980a9040 con 0x5650bd6121b0 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe58ffff640 1 -- 172.21.15.31:0/1934539647 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5980ad580 con 0x5650bd6121b0 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.655+0000 7fe58ffff640 1 -- 172.21.15.31:0/1934539647 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980a91e0 con 0x5650bd6121b0 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/1934539647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 msgr2=0x5650bd9efb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 0x5650bd9efb80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe5980ad770 tx=0x7fe5980e7f30 comp rx=0 tx=0).stop 2024-09-21T03:01:34.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/1934539647 shutdown_connections 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd61a390 0x5650bd9e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/1934539647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd6121b0 0x5650bd9efb80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/1934539647 >> 172.21.15.31:0/1934539647 conn(0x5650bd62f460 msgr2=0x5650bd9eda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/1934539647 shutdown_connections 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.656+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/1934539647 wait complete. 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 -- start start 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 Processor -- start 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b0b70 con 0x5650bd61a390 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81b04a0 con 0x5650bd6121b0 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a81ad6e0 con 0x5650bd63b070 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5b2ea7740 10 client.0 _ll_register_callbacks cb 0x7fe5980eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42346/0 (socket says 172.21.15.31:42346) 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/4158432467 learned_addr learned my addr 172.21.15.31:0/4158432467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a659c640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/4158432467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 msgr2=0x5650bd9f25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9f25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 msgr2=0x5650bd9f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.657+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe5a5d9b640 1 -- 172.21.15.31:0/4158432467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bd9f2d30 con 0x5650bd61a390 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe5a6d9d640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe5a5d9b640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9f1f00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009c60 tx=0x7fe59c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c678eecf412c50e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe58dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe58dffb640 1 -- 172.21.15.31:0/4158432467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004260 con 0x5650bd61a390 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe58dffb640 1 -- 172.21.15.31:0/4158432467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fe59c004400 con 0x5650bd61a390 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5980ad580 con 0x5650bd61a390 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe58dffb640 1 -- 172.21.15.31:0/4158432467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe59c03a050 con 0x5650bd61a390 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe5b2ea7740 10 client.17520 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650bda6aa40 con 0x5650bd61a390 2024-09-21T03:01:34.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.658+0000 7fe58dffb640 1 -- 172.21.15.31:0/4158432467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5980ad580 con 0x5650bd61a390 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe58dffb640 1 -- 172.21.15.31:0/4158432467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe59c035030 con 0x5650bd61a390 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5b2ea7740 10 client.17520 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5b2ea7740 10 client.17520 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5b2ea7740 20 client.17520 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5ceda0 0x5650bda6d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5b2ea7740 4 client.17520 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650bd9568a0 con 0x5650bd5ceda0 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5a659c640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5ceda0 0x5650bda6d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe58dffb640 10 client.17520 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe5a659c640 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5ceda0 0x5650bda6d0f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=996200427c788399 server_cookie=f0a081b8b0cab1ca in_seq=0 out_seq=0 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.659+0000 7fe58dffb640 1 -- 172.21.15.31:0/4158432467 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650bd9568a0 con 0x5650bd5ceda0 2024-09-21T03:01:34.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe58dffb640 10 client.17520 handle_command_reply: tid=0 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 client.17520 shutdown 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 msgr2=0x5650bd9f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9f1f00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009c60 tx=0x7fe59c005da0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 shutdown_connections 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5650bd5ceda0 0x5650bda6d0f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5650bd63b070 0x5650bd9f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650bd61a390 0x5650bd9f1f00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe58dffb640 0 client.17520 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 --2- 172.21.15.31:0/4158432467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650bd6121b0 0x5650bd9f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 >> 172.21.15.31:0/4158432467 conn(0x5650bd62f460 msgr2=0x5650bd9ef8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 shutdown_connections 2024-09-21T03:01:34.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 1 -- 172.21.15.31:0/4158432467 wait complete. 2024-09-21T03:01:34.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5b2ea7740 20 client.17520 trim_cache size 0 max 16384 2024-09-21T03:01:34.669 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5900b6680 msgr2=0x7fe5900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5900b6680 0x7fe5900b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004af0 tx=0x7fe5a0005c30 comp rx=0 tx=0).stop 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 msgr2=0x7fe5a819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 0x7fe5a819a3f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 shutdown_connections 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe5900b6680 0x7fe5900b8b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a81083d0 0x7fe5a819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.660+0000 7fe5ac933640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5a8108d20 0x7fe5a819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.661+0000 7fe5ac933640 1 --2- 172.21.15.31:0/940552312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5a8102ae0 0x7fe5a819a3f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.661+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 >> 172.21.15.31:0/940552312 conn(0x7fe5a8079760 msgr2=0x7fe5a8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.661+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 shutdown_connections 2024-09-21T03:01:34.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:34.661+0000 7fe5ac933640 1 -- 172.21.15.31:0/940552312 wait complete. 2024-09-21T03:01:34.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:34 smithi031 ceph-mon[31209]: pgmap v589: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:01:34.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:34 smithi178 ceph-mon[34759]: pgmap v589: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:01:35.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:34 smithi165 ceph-mon[34818]: pgmap v589: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:01:35.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:01:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:36.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 -- 172.21.15.31:0/4205864381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc10ddf0 msgr2=0x7f1dbc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/4205864381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc10ddf0 0x7f1dbc110300 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ca0 tx=0x7f1db402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 -- 172.21.15.31:0/4205864381 shutdown_connections 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/4205864381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc10ddf0 0x7f1dbc110300 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/4205864381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc105270 0x7f1dbc10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/4205864381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dbc104840 0x7f1dbc104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.142+0000 7f1dc0ead640 1 -- 172.21.15.31:0/4205864381 >> 172.21.15.31:0/4205864381 conn(0x7f1dbc079730 msgr2=0x7f1dbc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.143+0000 7f1dc0ead640 1 -- 172.21.15.31:0/4205864381 shutdown_connections 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.143+0000 7f1dc0ead640 1 -- 172.21.15.31:0/4205864381 wait complete. 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.143+0000 7f1dc0ead640 1 Processor -- start 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.143+0000 7f1dc0ead640 1 -- start start 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dc0ead640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc104840 0x7f1dbc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dc0ead640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 0x7f1dbc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dc0ead640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dbc10ddf0 0x7f1dbc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dc0ead640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc113a90 con 0x7f1dbc105270 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dc0ead640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc113910 con 0x7f1dbc104840 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dc0ead640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc113c10 con 0x7f1dbc10ddf0 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 0x7f1dbc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 0x7f1dbc19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42366/0 (socket says 172.21.15.31:42366) 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3536594440 learned_addr learned my addr 172.21.15.31:0/3536594440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dbc10ddf0 0x7f1dbc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dba59c640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc104840 0x7f1dbc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3536594440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dbc10ddf0 msgr2=0x7f1dbc19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dbc10ddf0 0x7f1dbc19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3536594440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc104840 msgr2=0x7f1dbc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc104840 0x7f1dbc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3536594440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc1a0310 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dba59c640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc104840 0x7f1dbc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 0x7f1dbc19aa00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c8c0 tx=0x7f1db000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.144+0000 7f1dab7fe640 1 -- 172.21.15.31:0/3536594440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0007bb0 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.145+0000 7f1dab7fe640 1 -- 172.21.15.31:0/3536594440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db0007d50 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.145+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc1a0600 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.145+0000 7f1dab7fe640 1 -- 172.21.15.31:0/3536594440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00187c0 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.145+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc1a6e20 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.147+0000 7f1dc7421740 1 Processor -- start 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 -- start start 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dab7fe640 1 -- 172.21.15.31:0/3536594440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f1db000fa20 con 0x7f1dbc105270 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc074f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc074fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc0387390 0x563fc0750120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563fc06c5ea0 con 0x563fc037f1b0 2024-09-21T03:01:36.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563fc06b52d0 con 0x563fc03a8070 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dc7421740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fc06b90e0 con 0x563fc0387390 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc074fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc074fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42374/0 (socket says 172.21.15.31:42374) 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dba59c640 1 -- 172.21.15.31:0/1474157777 learned_addr learned my addr 172.21.15.31:0/1474157777 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc074f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.148+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc0387390 0x563fc0750120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dba59c640 1 -- 172.21.15.31:0/1474157777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc0387390 msgr2=0x563fc0750120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dba59c640 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc0387390 0x563fc0750120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dba59c640 1 -- 172.21.15.31:0/1474157777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 msgr2=0x563fc074f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dba59c640 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc074f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dba59c640 1 -- 172.21.15.31:0/1474157777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc0413650 con 0x563fc037f1b0 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dba59c640 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc074fbe0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009c30 tx=0x7f1dac00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712e9d8b921aafd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dab7fe640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1da40b6730 0x7f1da40b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1da40b6730 0x7f1da40b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1da8ff9640 1 -- 172.21.15.31:0/1474157777 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac016070 con 0x563fc037f1b0 2024-09-21T03:01:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1dab7fe640 1 -- 172.21.15.31:0/3536594440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f1db00e7770 con 0x7f1dbc105270 2024-09-21T03:01:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1da8ff9640 1 -- 172.21.15.31:0/1474157777 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac00ee90 con 0x563fc037f1b0 2024-09-21T03:01:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.149+0000 7f1da8ff9640 1 -- 172.21.15.31:0/1474157777 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac011470 con 0x563fc037f1b0 2024-09-21T03:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 -- 172.21.15.31:0/1474157777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 msgr2=0x563fc074fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc074fbe0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009c30 tx=0x7f1dac00e6c0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1da40b6730 0x7f1da40b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1db4006010 tx=0x7f1db402f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 -- 172.21.15.31:0/1474157777 shutdown_connections 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc0387390 0x563fc0750120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc074fbe0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 --2- 172.21.15.31:0/1474157777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc074f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 -- 172.21.15.31:0/1474157777 >> 172.21.15.31:0/1474157777 conn(0x563fc039c460 msgr2=0x563fc074d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 -- 172.21.15.31:0/1474157777 shutdown_connections 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.150+0000 7f1dc7421740 1 -- 172.21.15.31:0/1474157777 wait complete. 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 -- start start 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 Processor -- start 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc037f1b0 0x563fc075f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc075f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc076b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563fc0516700 con 0x563fc0387390 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563fc0650160 con 0x563fc03a8070 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fc07621e0 con 0x563fc037f1b0 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1dc7421740 10 client.0 _ll_register_callbacks cb 0x563fc06699e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1db9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc075f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1db9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc075f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42378/0 (socket says 172.21.15.31:42378) 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.151+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3318266604 learned_addr learned my addr 172.21.15.31:0/3318266604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1dba59c640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc037f1b0 0x563fc075f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc076b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc037f1b0 msgr2=0x563fc075f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc037f1b0 0x563fc075f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3318266604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 msgr2=0x563fc076b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc076b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1db9d9b640 1 -- 172.21.15.31:0/3318266604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc067dc90 con 0x563fc0387390 2024-09-21T03:01:36.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc076b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc075f680 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1db0000c00 tx=0x7f1db00b6e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a081d45255300788 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1da1ffb640 1 -- 172.21.15.31:0/3318266604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00ed020 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1da1ffb640 1 -- 172.21.15.31:0/3318266604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f1db00ac820 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac00ee90 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1da1ffb640 1 -- 172.21.15.31:0/3318266604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1db0096e60 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1dc7421740 10 client.17532 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563fc076df30 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.152+0000 7f1da1ffb640 1 -- 172.21.15.31:0/3318266604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac00ee90 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1da1ffb640 1 -- 172.21.15.31:0/3318266604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1db00ac290 con 0x563fc0387390 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dc7421740 10 client.17532 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dc7421740 10 client.17532 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:01:36.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dc7421740 20 client.17532 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dc7421740 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03669f0 0x563fc07713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dc7421740 4 client.17532 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc06c7df0 con 0x563fc03669f0 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.153+0000 7f1dba59c640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03669f0 0x563fc07713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1da1ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1dba59c640 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03669f0 0x563fc07713a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88e703cb49ba51a6 server_cookie=5db1c26d0be1e49 in_seq=0 out_seq=0 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1da1ffb640 1 -- 172.21.15.31:0/3318266604 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563fc06c7df0 con 0x563fc03669f0 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1da1ffb640 10 client.17532 handle_command_reply: tid=0 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1dc7421740 1 client.17532 shutdown 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 msgr2=0x563fc075f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1dc7421740 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc075f680 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1db0000c00 tx=0x7f1db00b6e90 comp rx=0 tx=0).stop 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 shutdown_connections 2024-09-21T03:01:36.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.154+0000 7f1dc7421740 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03669f0 0x563fc07713a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1da1ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc03a8070 0x563fc076b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc075f680 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 1 --2- 172.21.15.31:0/3318266604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc037f1b0 0x563fc075f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 >> 172.21.15.31:0/3318266604 conn(0x563fc039c460 msgr2=0x563fc074bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 shutdown_connections 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 1 -- 172.21.15.31:0/3318266604 wait complete. 2024-09-21T03:01:36.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.155+0000 7f1dc7421740 20 client.17532 trim_cache size 0 max 16384 2024-09-21T03:01:36.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 Processor -- start 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 -- start start 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0761020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc076b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc0387390 0x563fc0750de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ad460 con 0x563fc037f1b0 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1aefa0 con 0x563fc0387390 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1dc7421740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1adb30 con 0x563fc03669f0 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1db9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc076b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1db9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc076b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42384/0 (socket says 172.21.15.31:42384) 2024-09-21T03:01:36.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.168+0000 7f1db9d9b640 1 -- 172.21.15.31:0/4151575721 learned_addr learned my addr 172.21.15.31:0/4151575721 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1dba59c640 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0761020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc0387390 0x563fc0750de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1db9d9b640 1 -- 172.21.15.31:0/4151575721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 msgr2=0x563fc0761020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0761020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1db9d9b640 1 -- 172.21.15.31:0/4151575721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc0387390 msgr2=0x563fc0750de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc0387390 0x563fc0750de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1db9d9b640 1 -- 172.21.15.31:0/4151575721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc0751fb0 con 0x563fc037f1b0 2024-09-21T03:01:36.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc0387390 0x563fc0750de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc076b410 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1db001f000 tx=0x7f1db00e4fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932c22c7a09086ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1da3fff640 1 -- 172.21.15.31:0/4151575721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00d4070 con 0x563fc037f1b0 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1da3fff640 1 -- 172.21.15.31:0/4151575721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db00ad0d0 con 0x563fc037f1b0 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.169+0000 7f1da3fff640 1 -- 172.21.15.31:0/4151575721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00a9150 con 0x563fc037f1b0 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 -- 172.21.15.31:0/4151575721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 msgr2=0x563fc076b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc076b410 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1db001f000 tx=0x7f1db00e4fc0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 -- 172.21.15.31:0/4151575721 shutdown_connections 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc0387390 0x563fc0750de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc037f1b0 0x563fc076b410 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 --2- 172.21.15.31:0/4151575721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0761020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 -- 172.21.15.31:0/4151575721 >> 172.21.15.31:0/4151575721 conn(0x563fc039c460 msgr2=0x563fc07506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 -- 172.21.15.31:0/4151575721 shutdown_connections 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 -- 172.21.15.31:0/4151575721 wait complete. 2024-09-21T03:01:36.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 -- start start 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.170+0000 7f1dc7421740 1 Processor -- start 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0763790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc037f1b0 0x563fc0763cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc07dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ae8d0 con 0x563fc0387390 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ae200 con 0x563fc037f1b0 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1ab460 con 0x563fc03669f0 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dc7421740 10 client.0 _ll_register_callbacks cb 0x7f1db0007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc07dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc07dad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42390/0 (socket says 172.21.15.31:42390) 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 -- 172.21.15.31:0/559051862 learned_addr learned my addr 172.21.15.31:0/559051862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dba59c640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0763790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1db9d9b640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc037f1b0 0x563fc0763cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 -- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 msgr2=0x563fc0763790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0763790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 -- 172.21.15.31:0/559051862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc037f1b0 msgr2=0x563fc0763cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc037f1b0 0x563fc0763cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.171+0000 7f1dbad9d640 1 -- 172.21.15.31:0/559051862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc07524d0 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dbad9d640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc07dad40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1db40316d0 tx=0x7f1db402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff59d221c681fbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1da1ffb640 1 -- 172.21.15.31:0/559051862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4057060 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1da1ffb640 1 -- 172.21.15.31:0/559051862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f1db4004040 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db00ad0d0 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1da1ffb640 1 -- 172.21.15.31:0/559051862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1db4004240 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 10 client.17538 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563fc07dc0a0 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1da1ffb640 1 -- 172.21.15.31:0/559051862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db00ad0d0 con 0x563fc0387390 2024-09-21T03:01:36.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1da1ffb640 1 -- 172.21.15.31:0/559051862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1db402fd30 con 0x563fc0387390 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 10 client.17538 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 10 client.17538 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 20 client.17538 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.172+0000 7f1dc7421740 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03a8070 0x563fc07de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.173+0000 7f1dc7421740 4 client.17538 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.173+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc06c38a0 con 0x563fc03a8070 2024-09-21T03:01:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.173+0000 7f1dba59c640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03a8070 0x563fc07de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.173+0000 7f1dba59c640 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03a8070 0x563fc07de8c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3889affb4cdb4e7e server_cookie=b1f2cd51af42d1c7 in_seq=0 out_seq=0 2024-09-21T03:01:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.173+0000 7f1da1ffb640 10 client.17538 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:01:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.173+0000 7f1da1ffb640 1 -- 172.21.15.31:0/559051862 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc06c38a0 con 0x563fc03a8070 2024-09-21T03:01:36.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1da1ffb640 10 client.17538 handle_command_reply: tid=0 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 client.17538 shutdown 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 msgr2=0x563fc07dad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc07dad40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1db40316d0 tx=0x7f1db402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 shutdown_connections 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x563fc03a8070 0x563fc07de8c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563fc0387390 0x563fc07dad40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc037f1b0 0x563fc0763cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 --2- 172.21.15.31:0/559051862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc03669f0 0x563fc0763790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 >> 172.21.15.31:0/559051862 conn(0x563fc039c460 msgr2=0x563fc0760d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1da1ffb640 0 client.17538 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 shutdown_connections 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 1 -- 172.21.15.31:0/559051862 wait complete. 2024-09-21T03:01:36.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc7421740 20 client.17538 trim_cache size 0 max 16384 2024-09-21T03:01:36.181 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:01:36.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1da40b6730 msgr2=0x7f1da40b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1da40b6730 0x7f1da40b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1db4006010 tx=0x7f1db402f310 comp rx=0 tx=0).stop 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 msgr2=0x7f1dbc19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.174+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 0x7f1dbc19aa00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c8c0 tx=0x7f1db000cd80 comp rx=0 tx=0).stop 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 shutdown_connections 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f1da40b6730 0x7f1da40b8c40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dbc10ddf0 0x7f1dbc19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dbc105270 0x7f1dbc19aa00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 --2- 172.21.15.31:0/3536594440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dbc104840 0x7f1dbc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 >> 172.21.15.31:0/3536594440 conn(0x7f1dbc079730 msgr2=0x7f1dbc107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 shutdown_connections 2024-09-21T03:01:36.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:36.175+0000 7f1dc0ead640 1 -- 172.21.15.31:0/3536594440 wait complete. 2024-09-21T03:01:36.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:01:36.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:36 smithi178 ceph-mon[34759]: pgmap v590: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:36 smithi031 ceph-mon[31209]: pgmap v590: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:36 smithi165 ceph-mon[34818]: pgmap v590: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 -- 172.21.15.31:0/636797764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f98103bb0 msgr2=0x7f6f98103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/636797764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f98103bb0 0x7f6f98103fc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009900 tx=0x7f6f8802a410 comp rx=0 tx=0).stop 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 -- 172.21.15.31:0/636797764 shutdown_connections 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/636797764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f981094f0 0x7f6f98109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/636797764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f98109e40 0x7f6f98110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/636797764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f98103bb0 0x7f6f98103fc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 -- 172.21.15.31:0/636797764 >> 172.21.15.31:0/636797764 conn(0x7f6f980ff600 msgr2=0x7f6f98101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 -- 172.21.15.31:0/636797764 shutdown_connections 2024-09-21T03:01:37.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.602+0000 7f6f9ebce640 1 -- 172.21.15.31:0/636797764 wait complete. 2024-09-21T03:01:37.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 Processor -- start 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 -- start start 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f98103bb0 0x7f6f9819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 0x7f6f9819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f98109e40 0x7f6f981a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98113b00 con 0x7f6f981094f0 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98113980 con 0x7f6f98103bb0 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9ebce640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98113c80 con 0x7f6f98109e40 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 0x7f6f9819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 0x7f6f9819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42408/0 (socket says 172.21.15.31:42408) 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f98109e40 0x7f6f981a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f97fff640 1 -- 172.21.15.31:0/180911567 learned_addr learned my addr 172.21.15.31:0/180911567 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.603+0000 7f6f9c96a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f98103bb0 0x7f6f9819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f97fff640 1 -- 172.21.15.31:0/180911567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f98109e40 msgr2=0x7f6f981a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f97fff640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f98109e40 0x7f6f981a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f97fff640 1 -- 172.21.15.31:0/180911567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f98103bb0 msgr2=0x7f6f9819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f97fff640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f98103bb0 0x7f6f9819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f97fff640 1 -- 172.21.15.31:0/180911567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f981a2640 con 0x7f6f981094f0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f97fff640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 0x7f6f9819cc70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000cce0 tx=0x7f6f8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f95ffb640 1 -- 172.21.15.31:0/180911567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80004b20 con 0x7f6f981094f0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f95ffb640 1 -- 172.21.15.31:0/180911567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f80004cc0 con 0x7f6f981094f0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f981a2930 con 0x7f6f981094f0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f95ffb640 1 -- 172.21.15.31:0/180911567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000f6b0 con 0x7f6f981094f0 2024-09-21T03:01:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.604+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f981a9090 con 0x7f6f981094f0 2024-09-21T03:01:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 Processor -- start 2024-09-21T03:01:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6f95ffb640 1 -- 172.21.15.31:0/180911567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6f8000f850 con 0x7f6f981094f0 2024-09-21T03:01:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 -- start start 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d21be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def8959390 0x55def8d22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55def8c97ea0 con 0x55def897a070 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55def8c872d0 con 0x55def8959390 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6fa5142740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55def8c8b0e0 con 0x55def89511b0 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42424/0 (socket says 172.21.15.31:42424) 2024-09-21T03:01:37.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.607+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1277448470 learned_addr learned my addr 172.21.15.31:0/1277448470 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f97fff640 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def8959390 0x55def8d22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d21be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1277448470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 msgr2=0x55def8d21be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d21be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1277448470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def8959390 msgr2=0x55def8d22120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def8959390 0x55def8d22120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1277448470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55def89e5650 con 0x55def897a070 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f97fff640 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def8959390 0x55def8d22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f95ffb640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6f840b6680 0x7f6f840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d216a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000ab80 tx=0x7f6f9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3181e447c7881c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6f840b6680 0x7f6f840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f8effd640 1 -- 172.21.15.31:0/1277448470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001d070 con 0x55def897a070 2024-09-21T03:01:37.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f95ffb640 1 -- 172.21.15.31:0/180911567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f6f800e7680 con 0x7f6f981094f0 2024-09-21T03:01:37.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f8effd640 1 -- 172.21.15.31:0/1277448470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f90018af0 con 0x55def897a070 2024-09-21T03:01:37.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.608+0000 7f6f8effd640 1 -- 172.21.15.31:0/1277448470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90010970 con 0x55def897a070 2024-09-21T03:01:37.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 -- 172.21.15.31:0/1277448470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 msgr2=0x55def8d216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6f840b6680 0x7f6f840b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f88031440 tx=0x7f6f88009650 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d216a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000ab80 tx=0x7f6f9000f790 comp rx=0 tx=0).stop 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 -- 172.21.15.31:0/1277448470 shutdown_connections 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d21be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def8959390 0x55def8d22120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1277448470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d216a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 -- 172.21.15.31:0/1277448470 >> 172.21.15.31:0/1277448470 conn(0x55def896e460 msgr2=0x55def8d1f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 -- 172.21.15.31:0/1277448470 shutdown_connections 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.609+0000 7f6fa5142740 1 -- 172.21.15.31:0/1277448470 wait complete. 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 -- start start 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 Processor -- start 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d2cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d2d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d39170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55def8ae8700 con 0x55def897a070 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55def8c22160 con 0x55def89511b0 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55def8d2fe80 con 0x55def8959390 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.610+0000 7f6fa5142740 10 client.0 _ll_register_callbacks cb 0x55def8c3b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d39170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d39170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42436/0 (socket says 172.21.15.31:42436) 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 -- 172.21.15.31:0/490206756 learned_addr learned my addr 172.21.15.31:0/490206756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f97fff640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d2d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d2cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 -- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 msgr2=0x55def8d2d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d2d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 -- 172.21.15.31:0/490206756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 msgr2=0x55def8d2cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d2cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 -- 172.21.15.31:0/490206756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55def8c4fc90 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f97fff640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d39170 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6f90010510 tx=0x7f6f90018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9122a79cb10f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f77fff640 1 -- 172.21.15.31:0/490206756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001d050 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6f77fff640 1 -- 172.21.15.31:0/490206756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f6f90005040 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.611+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f90018af0 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6f77fff640 1 -- 172.21.15.31:0/490206756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f90031050 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 10 client.17550 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55def8d3bb70 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6f77fff640 1 -- 172.21.15.31:0/490206756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f90018af0 con 0x55def897a070 2024-09-21T03:01:37.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6f77fff640 1 -- 172.21.15.31:0/490206756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6f90010ce0 con 0x55def897a070 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 10 client.17550 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 10 client.17550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 20 client.17550 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def89389f0 0x55def8d3ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 4 client.17550 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55def8c99df0 con 0x55def89389f0 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.612+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def89389f0 0x55def8d3ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.613+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def89389f0 0x55def8d3ef80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0ed0cfd522ef0fc server_cookie=645a1ed69fd65caf in_seq=0 out_seq=0 2024-09-21T03:01:37.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.613+0000 7f6f77fff640 10 client.17550 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:01:37.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6f77fff640 1 -- 172.21.15.31:0/490206756 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55def8c99df0 con 0x55def89389f0 2024-09-21T03:01:37.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6f77fff640 10 client.17550 handle_command_reply: tid=0 2024-09-21T03:01:37.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 client.17550 shutdown 2024-09-21T03:01:37.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 msgr2=0x55def8d39170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d39170 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6f90010510 tx=0x7f6f90018670 comp rx=0 tx=0).stop 2024-09-21T03:01:37.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 shutdown_connections 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def89389f0 0x55def8d3ef80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8d39170 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6f77fff640 0 client.17550 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 --2- 172.21.15.31:0/490206756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d2cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 >> 172.21.15.31:0/490206756 conn(0x55def896e460 msgr2=0x55def8d24880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 shutdown_connections 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 1 -- 172.21.15.31:0/490206756 wait complete. 2024-09-21T03:01:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.614+0000 7f6fa5142740 20 client.17550 trim_cache size 0 max 16384 2024-09-21T03:01:37.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 Processor -- start 2024-09-21T03:01:37.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 -- start start 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d2d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 0x55def8d390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def897a070 0x55def8d29f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b1a10 con 0x55def8959390 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b3550 con 0x55def897a070 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6fa5142740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b20e0 con 0x55def89511b0 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 0x55def8d390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 0x55def8d390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42442/0 (socket says 172.21.15.31:42442) 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6f9c96a640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d2d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6f97fff640 1 -- 172.21.15.31:0/3923944260 learned_addr learned my addr 172.21.15.31:0/3923944260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.631+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def897a070 0x55def8d29f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f97fff640 1 -- 172.21.15.31:0/3923944260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 msgr2=0x55def8d2d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f97fff640 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d2d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f97fff640 1 -- 172.21.15.31:0/3923944260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def897a070 msgr2=0x55def8d29f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f97fff640 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def897a070 0x55def8d29f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f97fff640 1 -- 172.21.15.31:0/3923944260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55def8d23990 con 0x55def8959390 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d2d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f97fff640 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 0x55def8d390e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6f800a9b00 tx=0x7f6f80004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2afbbdd93f16b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f8e7fc640 1 -- 172.21.15.31:0/3923944260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8001d030 con 0x55def8959390 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f8e7fc640 1 -- 172.21.15.31:0/3923944260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f800ad090 con 0x55def8959390 2024-09-21T03:01:37.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.632+0000 7f6f8e7fc640 1 -- 172.21.15.31:0/3923944260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800e7ab0 con 0x55def8959390 2024-09-21T03:01:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 -- 172.21.15.31:0/3923944260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 msgr2=0x55def8d390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 0x55def8d390e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6f800a9b00 tx=0x7f6f80004f40 comp rx=0 tx=0).stop 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 -- 172.21.15.31:0/3923944260 shutdown_connections 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def897a070 0x55def8d29f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def8959390 0x55def8d390e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 --2- 172.21.15.31:0/3923944260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def89511b0 0x55def8d2d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 -- 172.21.15.31:0/3923944260 >> 172.21.15.31:0/3923944260 conn(0x55def896e460 msgr2=0x55def8d2caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 -- 172.21.15.31:0/3923944260 shutdown_connections 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 -- 172.21.15.31:0/3923944260 wait complete. 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.633+0000 7f6fa5142740 1 -- start start 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 Processor -- start 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8da8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b2e80 con 0x55def897a070 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b27b0 con 0x55def89511b0 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981af9f0 con 0x55def8959390 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6fa5142740 10 client.0 _ll_register_callbacks cb 0x7f6f9001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8da8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6f9d16b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8da8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42448/0 (socket says 172.21.15.31:42448) 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6f9c96a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1782905242 learned_addr learned my addr 172.21.15.31:0/1782905242 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.634+0000 7f6f97fff640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 msgr2=0x55def8d409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1782905242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 msgr2=0x55def8d403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9d16b640 1 -- 172.21.15.31:0/1782905242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55def8da8c40 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f97fff640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f9d16b640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8da8700 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6f900048b0 tx=0x7f6f90004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cd1ef0b02058fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f77fff640 1 -- 172.21.15.31:0/1782905242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001d040 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f77fff640 1 -- 172.21.15.31:0/1782905242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f6f90018c80 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f800ad090 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f77fff640 1 -- 172.21.15.31:0/1782905242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f9000ae80 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6fa5142740 10 client.17556 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55def8da9dd0 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.635+0000 7f6f77fff640 1 -- 172.21.15.31:0/1782905242 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f800ad090 con 0x55def897a070 2024-09-21T03:01:37.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6f77fff640 1 -- 172.21.15.31:0/1782905242 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6f90010540 con 0x55def897a070 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6fa5142740 10 client.17556 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6fa5142740 10 client.17556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6fa5142740 20 client.17556 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def890dda0 0x55def8dac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6fa5142740 4 client.17556 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55def8c958a0 con 0x55def890dda0 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.636+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def890dda0 0x55def8dac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6f77fff640 10 client.17556 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6f9c96a640 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def890dda0 0x55def8dac500 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c97f6f6a74f3e63 server_cookie=1610896ed71bdf5f in_seq=0 out_seq=0 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6f77fff640 1 -- 172.21.15.31:0/1782905242 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55def8c958a0 con 0x55def890dda0 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6f77fff640 10 client.17556 handle_command_reply: tid=0 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 client.17556 shutdown 2024-09-21T03:01:37.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 msgr2=0x55def8da8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8da8700 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6f900048b0 tx=0x7f6f90004ce0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 shutdown_connections 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55def890dda0 0x55def8dac500 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55def8959390 0x55def8d409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55def897a070 0x55def8da8700 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 --2- 172.21.15.31:0/1782905242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55def89511b0 0x55def8d403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6f77fff640 0 client.17556 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 >> 172.21.15.31:0/1782905242 conn(0x55def896e460 msgr2=0x55def8d22660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 shutdown_connections 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 1 -- 172.21.15.31:0/1782905242 wait complete. 2024-09-21T03:01:37.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.637+0000 7f6fa5142740 20 client.17556 trim_cache size 0 max 16384 2024-09-21T03:01:37.644 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:01:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6f840b6680 msgr2=0x7f6f840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6f840b6680 0x7f6f840b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f88031440 tx=0x7f6f88009650 comp rx=0 tx=0).stop 2024-09-21T03:01:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 msgr2=0x7f6f9819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 0x7f6f9819cc70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000cce0 tx=0x7f6f8000a620 comp rx=0 tx=0).stop 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 shutdown_connections 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6f840b6680 0x7f6f840b8b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f98109e40 0x7f6f981a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f981094f0 0x7f6f9819cc70 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 --2- 172.21.15.31:0/180911567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f98103bb0 0x7f6f9819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 >> 172.21.15.31:0/180911567 conn(0x7f6f980ff600 msgr2=0x7f6f98108bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 shutdown_connections 2024-09-21T03:01:37.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:37.638+0000 7f6f9ebce640 1 -- 172.21.15.31:0/180911567 wait complete. 2024-09-21T03:01:38.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:01:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:38.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:38 smithi178 ceph-mon[34759]: pgmap v591: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:01:38.993 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:38 smithi031 ceph-mon[31209]: pgmap v591: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 -- 172.21.15.31:0/56096274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8109e00 msgr2=0x7f78b8110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 --2- 172.21.15.31:0/56096274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8109e00 0x7f78b8110630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009ca0 tx=0x7f78a802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 -- 172.21.15.31:0/56096274 shutdown_connections 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 --2- 172.21.15.31:0/56096274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8109e00 0x7f78b8110630 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 --2- 172.21.15.31:0/56096274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b81094b0 0x7f78b81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 --2- 172.21.15.31:0/56096274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8103bc0 0x7f78b8103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 -- 172.21.15.31:0/56096274 >> 172.21.15.31:0/56096274 conn(0x7f78b807b910 msgr2=0x7f78b807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 -- 172.21.15.31:0/56096274 shutdown_connections 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.087+0000 7f78bcca6640 1 -- 172.21.15.31:0/56096274 wait complete. 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 Processor -- start 2024-09-21T03:01:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 -- start start 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 0x7f78b819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b81094b0 0x7f78b819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8109e00 0x7f78b819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8113ac0 con 0x7f78b8103bc0 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8113940 con 0x7f78b81094b0 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78bcca6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8113c40 con 0x7f78b8109e00 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 0x7f78b819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 0x7f78b819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44724/0 (socket says 172.21.15.31:44724) 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.088+0000 7f78b659c640 1 -- 172.21.15.31:0/2983419516 learned_addr learned my addr 172.21.15.31:0/2983419516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8109e00 0x7f78b819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b81094b0 0x7f78b819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b659c640 1 -- 172.21.15.31:0/2983419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8109e00 msgr2=0x7f78b819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b659c640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8109e00 0x7f78b819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b659c640 1 -- 172.21.15.31:0/2983419516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b81094b0 msgr2=0x7f78b819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b659c640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b81094b0 0x7f78b819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b659c640 1 -- 172.21.15.31:0/2983419516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b81a03a0 con 0x7f78b8103bc0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8109e00 0x7f78b819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78b659c640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 0x7f78b819a430 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f78b000e9f0 tx=0x7f78b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78af7fe640 1 -- 172.21.15.31:0/2983419516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b000cd80 con 0x7f78b8103bc0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78af7fe640 1 -- 172.21.15.31:0/2983419516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b00108e0 con 0x7f78b8103bc0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78af7fe640 1 -- 172.21.15.31:0/2983419516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0018980 con 0x7f78b8103bc0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b81a0690 con 0x7f78b8103bc0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.089+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81a6df0 con 0x7f78b8103bc0 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 Processor -- start 2024-09-21T03:01:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 -- start start 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a058635ea0 con 0x55a0582f7390 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0586252d0 con 0x55a0582ef1b0 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.092+0000 7f78c321a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0586290e0 con 0x55a058318070 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.31:50130/0 (socket says 172.21.15.31:50130) 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 -- 172.21.15.31:0/748821260 learned_addr learned my addr 172.21.15.31:0/748821260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78af7fe640 1 -- 172.21.15.31:0/2983419516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f78b0010430 con 0x7f78b8103bc0 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 -- 172.21.15.31:0/748821260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 msgr2=0x55a0586bf800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:01:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586bf800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 -- 172.21.15.31:0/748821260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 msgr2=0x55a0586bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.093+0000 7f78b6d9d640 1 -- 172.21.15.31:0/748821260 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a058383650 con 0x55a058318070 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.094+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bf2c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009470 tx=0x7f78a80094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f415210d5e1c3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.094+0000 7f78acff9640 1 -- 172.21.15.31:0/748821260 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8031cf0 con 0x55a058318070 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.094+0000 7f78acff9640 1 -- 172.21.15.31:0/748821260 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a8036040 con 0x55a058318070 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.094+0000 7f78acff9640 1 -- 172.21.15.31:0/748821260 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a80042a0 con 0x55a058318070 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.094+0000 7f78af7fe640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78a40b6700 0x7f78a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.094+0000 7f78b659c640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78a40b6700 0x7f78a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78af7fe640 1 -- 172.21.15.31:0/2983419516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f78b00e7b30 con 0x7f78b8103bc0 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 -- 172.21.15.31:0/748821260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 msgr2=0x55a0586bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78b659c640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78a40b6700 0x7f78a40b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55a0586bdf40 tx=0x7f78b0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bf2c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009470 tx=0x7f78a80094a0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 -- 172.21.15.31:0/748821260 shutdown_connections 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 --2- 172.21.15.31:0/748821260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bf2c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 -- 172.21.15.31:0/748821260 >> 172.21.15.31:0/748821260 conn(0x55a05830c460 msgr2=0x55a0586bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:39.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 -- 172.21.15.31:0/748821260 shutdown_connections 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.095+0000 7f78c321a740 1 -- 172.21.15.31:0/748821260 wait complete. 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 -- start start 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 Processor -- start 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 0x55a0586ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0582f7390 0x55a0586cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a058318070 0x55a0586db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a058486700 con 0x55a0582ef1b0 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a0585c0160 con 0x55a058318070 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0586d1e50 con 0x55a0582f7390 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78c321a740 10 client.0 _ll_register_callbacks cb 0x55a0585d99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 0x55a0586ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 0x55a0586ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44748/0 (socket says 172.21.15.31:44748) 2024-09-21T03:01:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.096+0000 7f78b6d9d640 1 -- 172.21.15.31:0/1874189574 learned_addr learned my addr 172.21.15.31:0/1874189574 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0582f7390 0x55a0586cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b6d9d640 1 -- 172.21.15.31:0/1874189574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0582f7390 msgr2=0x55a0586cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0582f7390 0x55a0586cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b6d9d640 1 -- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a058318070 msgr2=0x55a0586db170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a058318070 0x55a0586db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b6d9d640 1 -- 172.21.15.31:0/1874189574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0585edc90 con 0x55a0582ef1b0 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0582f7390 0x55a0586cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 0x55a0586ca950 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009620 tx=0x7f78a8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17ece04b217a4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f789dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f789dffb640 1 -- 172.21.15.31:0/1874189574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8031d00 con 0x55a0582ef1b0 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a8036040 con 0x55a0582ef1b0 2024-09-21T03:01:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f789dffb640 1 -- 172.21.15.31:0/1874189574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f78a8031450 con 0x55a0582ef1b0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f789dffb640 1 -- 172.21.15.31:0/1874189574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78a8031650 con 0x55a0582ef1b0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78c321a740 10 client.17568 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.097+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0586ddb30 con 0x55a0582ef1b0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f789dffb640 1 -- 172.21.15.31:0/1874189574 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a8036040 con 0x55a0582ef1b0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f789dffb640 1 -- 172.21.15.31:0/1874189574 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f78a8058030 con 0x55a0582ef1b0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78c321a740 10 client.17568 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78c321a740 10 client.17568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78c321a740 20 client.17568 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78c321a740 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582d69f0 0x55a0586e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78c321a740 4 client.17568 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a058637df0 con 0x55a0582d69f0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.098+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582d69f0 0x55a0586e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.099+0000 7f789dffb640 10 client.17568 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.099+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582d69f0 0x55a0586e0e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb273d77add00072 server_cookie=486ea32b8d7bbdf8 in_seq=0 out_seq=0 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.099+0000 7f789dffb640 1 -- 172.21.15.31:0/1874189574 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a058637df0 con 0x55a0582d69f0 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.099+0000 7f789dffb640 10 client.17568 handle_command_reply: tid=0 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.099+0000 7f78c321a740 1 client.17568 shutdown 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 msgr2=0x55a0586ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 0x55a0586ca950 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009620 tx=0x7f78a8003970 comp rx=0 tx=0).stop 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 shutdown_connections 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582d69f0 0x55a0586e0e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0582f7390 0x55a0586cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a058318070 0x55a0586db170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 --2- 172.21.15.31:0/1874189574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582ef1b0 0x55a0586ca950 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 >> 172.21.15.31:0/1874189574 conn(0x55a05830c460 msgr2=0x55a0586c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f789dffb640 0 client.17568 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:01:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 shutdown_connections 2024-09-21T03:01:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 1 -- 172.21.15.31:0/1874189574 wait complete. 2024-09-21T03:01:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.100+0000 7f78c321a740 20 client.17568 trim_cache size 0 max 16384 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 Processor -- start 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 -- start start 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81ad600 con 0x55a0582f7390 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81af140 con 0x55a0582ef1b0 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78c321a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81adcd0 con 0x55a058318070 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78b5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.113+0000 7f78b5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44762/0 (socket says 172.21.15.31:44762) 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 -- 172.21.15.31:0/1512967662 learned_addr learned my addr 172.21.15.31:0/1512967662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 -- 172.21.15.31:0/1512967662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 msgr2=0x55a0586bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 -- 172.21.15.31:0/1512967662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 msgr2=0x55a0586c0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586c0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 -- 172.21.15.31:0/1512967662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0586c15b0 con 0x55a0582f7390 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586c0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586cc580 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fd0 tx=0x7f78a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dbb120a7278d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f789ffff640 1 -- 172.21.15.31:0/1512967662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004250 con 0x55a0582f7390 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f789ffff640 1 -- 172.21.15.31:0/1512967662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a00043f0 con 0x55a0582f7390 2024-09-21T03:01:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.114+0000 7f789ffff640 1 -- 172.21.15.31:0/1512967662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00119a0 con 0x55a0582f7390 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 -- 172.21.15.31:0/1512967662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 msgr2=0x55a0586cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586cc580 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fd0 tx=0x7f78a0005f90 comp rx=0 tx=0).stop 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 -- 172.21.15.31:0/1512967662 shutdown_connections 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586cc580 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 --2- 172.21.15.31:0/1512967662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586c0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 -- 172.21.15.31:0/1512967662 >> 172.21.15.31:0/1512967662 conn(0x55a05830c460 msgr2=0x55a0586cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 -- 172.21.15.31:0/1512967662 shutdown_connections 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 -- 172.21.15.31:0/1512967662 wait complete. 2024-09-21T03:01:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 -- start start 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.115+0000 7f78c321a740 1 Processor -- start 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586e1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81aea70 con 0x55a0582f7390 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81ae3a0 con 0x55a0582ef1b0 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81ab5e0 con 0x55a058318070 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78c321a740 10 client.0 _ll_register_callbacks cb 0x7f78a0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586e1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44770/0 (socket says 172.21.15.31:44770) 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 -- 172.21.15.31:0/818374175 learned_addr learned my addr 172.21.15.31:0/818374175 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 -- 172.21.15.31:0/818374175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 msgr2=0x55a0586e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 -- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 msgr2=0x55a0586e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b5d9b640 1 -- 172.21.15.31:0/818374175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0586e2470 con 0x55a0582f7390 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.116+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78b5d9b640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586e1970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f78a0011480 tx=0x7f78a0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb0e7bcbbbb3157 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f789dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f789dffb640 1 -- 172.21.15.31:0/818374175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0003800 con 0x55a0582f7390 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f789dffb640 1 -- 172.21.15.31:0/818374175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7f78a00039a0 con 0x55a0582f7390 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a00043f0 con 0x55a0582f7390 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f789dffb640 1 -- 172.21.15.31:0/818374175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78a003b050 con 0x55a0582f7390 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78c321a740 10 client.17574 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:39.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a05874b340 con 0x55a0582f7390 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f789dffb640 1 -- 172.21.15.31:0/818374175 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a00043f0 con 0x55a0582f7390 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f789dffb640 1 -- 172.21.15.31:0/818374175 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f78a0025410 con 0x55a0582f7390 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78c321a740 10 client.17574 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78c321a740 10 client.17574 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.117+0000 7f78c321a740 20 client.17574 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f78c321a740 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582abda0 0x55a05874d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f78c321a740 4 client.17574 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0586338a0 con 0x55a0582abda0 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582abda0 0x55a05874d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f78b6d9d640 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582abda0 0x55a05874d9e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87a227952f2306bd server_cookie=27ac4defa3b0c1b9 in_seq=0 out_seq=0 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f789dffb640 10 client.17574 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f789dffb640 1 -- 172.21.15.31:0/818374175 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0586338a0 con 0x55a0582abda0 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f789dffb640 10 client.17574 handle_command_reply: tid=0 2024-09-21T03:01:39.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.118+0000 7f78c321a740 1 client.17574 shutdown 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 msgr2=0x55a0586e1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586e1970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f78a0011480 tx=0x7f78a0009ed0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 shutdown_connections 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55a0582abda0 0x55a05874d9e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a058318070 0x55a0586e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a0582f7390 0x55a0586e1970 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 --2- 172.21.15.31:0/818374175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a0582ef1b0 0x55a0586e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f789dffb640 0 client.17574 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 >> 172.21.15.31:0/818374175 conn(0x55a05830c460 msgr2=0x55a0586c09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 shutdown_connections 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 1 -- 172.21.15.31:0/818374175 wait complete. 2024-09-21T03:01:39.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78c321a740 20 client.17574 trim_cache size 0 max 16384 2024-09-21T03:01:39.126 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:01:39.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78a40b6700 msgr2=0x7f78a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78a40b6700 0x7f78a40b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55a0586bdf40 tx=0x7f78b0000bc0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 msgr2=0x7f78b819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 0x7f78b819a430 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f78b000e9f0 tx=0x7f78b000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 shutdown_connections 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78a40b6700 0x7f78a40b8c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b8109e00 0x7f78b819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78b81094b0 0x7f78b819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.119+0000 7f78bcca6640 1 --2- 172.21.15.31:0/2983419516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bc0 0x7f78b819a430 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.120+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 >> 172.21.15.31:0/2983419516 conn(0x7f78b807b910 msgr2=0x7f78b81074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.120+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 shutdown_connections 2024-09-21T03:01:39.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:39.120+0000 7f78bcca6640 1 -- 172.21.15.31:0/2983419516 wait complete. 2024-09-21T03:01:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:38 smithi165 ceph-mon[34818]: pgmap v591: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:01:39.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:01:40.046 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 -- 172.21.15.31:0/2344462543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c1094d0 msgr2=0x7fe09c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/2344462543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c1094d0 0x7fe09c1098e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009900 tx=0x7fe08c02a410 comp rx=0 tx=0).stop 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 -- 172.21.15.31:0/2344462543 shutdown_connections 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/2344462543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe09c109e20 0x7fe09c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/2344462543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c1094d0 0x7fe09c1098e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/2344462543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe09c103b90 0x7fe09c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 -- 172.21.15.31:0/2344462543 >> 172.21.15.31:0/2344462543 conn(0x7fe09c0ff600 msgr2=0x7fe09c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 -- 172.21.15.31:0/2344462543 shutdown_connections 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.564+0000 7fe0a40bc640 1 -- 172.21.15.31:0/2344462543 wait complete. 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 Processor -- start 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 -- start start 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe09c103b90 0x7fe09c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe09c1094d0 0x7fe09c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 0x7fe09c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113ae0 con 0x7fe09c109e20 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113960 con 0x7fe09c1094d0 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.565+0000 7fe0a40bc640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113c60 con 0x7fe09c103b90 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 0x7fe09c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 0x7fe09c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44784/0 (socket says 172.21.15.31:44784) 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 -- 172.21.15.31:0/87133307 learned_addr learned my addr 172.21.15.31:0/87133307 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe09c103b90 0x7fe09c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a1657640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe09c1094d0 0x7fe09c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 -- 172.21.15.31:0/87133307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe09c103b90 msgr2=0x7fe09c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe09c103b90 0x7fe09c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 -- 172.21.15.31:0/87133307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe09c1094d0 msgr2=0x7fe09c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe09c1094d0 0x7fe09c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 -- 172.21.15.31:0/87133307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c1a0400 con 0x7fe09c109e20 2024-09-21T03:01:40.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a2659640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 0x7fe09c19fcd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe094009fd0 tx=0x7fe09400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe092ffd640 1 -- 172.21.15.31:0/87133307 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094017070 con 0x7fe09c109e20 2024-09-21T03:01:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe092ffd640 1 -- 172.21.15.31:0/87133307 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe094003e20 con 0x7fe09c109e20 2024-09-21T03:01:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c1a06f0 con 0x7fe09c109e20 2024-09-21T03:01:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe092ffd640 1 -- 172.21.15.31:0/87133307 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094010460 con 0x7fe09c109e20 2024-09-21T03:01:40.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.566+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a6e50 con 0x7fe09c109e20 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.569+0000 7fe0aa630740 1 Processor -- start 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.570+0000 7fe092ffd640 1 -- 172.21.15.31:0/87133307 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fe094003970 con 0x7fe09c109e20 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.570+0000 7fe0aa630740 1 -- start start 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.570+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d9e070 0x55baa81452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.570+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8145800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d7d390 0x55baa8145d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0aa630740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55baa80bbea0 con 0x55baa7d751b0 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0aa630740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55baa80ab2d0 con 0x55baa7d7d390 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0aa630740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55baa80af0e0 con 0x55baa7d9e070 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44794/0 (socket says 172.21.15.31:44794) 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 -- 172.21.15.31:0/2255132758 learned_addr learned my addr 172.21.15.31:0/2255132758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1657640 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d9e070 0x55baa81452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1657640 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d7d390 0x55baa8145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe092ffd640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe0880b6680 0x7fe0880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 -- 172.21.15.31:0/2255132758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d9e070 msgr2=0x55baa81452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d9e070 0x55baa81452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 -- 172.21.15.31:0/2255132758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d7d390 msgr2=0x55baa8145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d7d390 0x55baa8145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a1e58640 1 -- 172.21.15.31:0/2255132758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa7e09650 con 0x55baa7d751b0 2024-09-21T03:01:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe0a2659640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe0880b6680 0x7fe0880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.571+0000 7fe092ffd640 1 -- 172.21.15.31:0/87133307 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fe0940e7370 con 0x7fe09c109e20 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.572+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8145800 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe09800bd50 tx=0x7fe0980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330f7c892f34d246 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.572+0000 7fe087fff640 1 -- 172.21.15.31:0/2255132758 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098005020 con 0x55baa7d751b0 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.572+0000 7fe087fff640 1 -- 172.21.15.31:0/2255132758 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0980047a0 con 0x55baa7d751b0 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.572+0000 7fe0a2659640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe0880b6680 0x7fe0880b8b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe09400dfd0 tx=0x7fe094058920 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.572+0000 7fe087fff640 1 -- 172.21.15.31:0/2255132758 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098011680 con 0x55baa7d751b0 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 -- 172.21.15.31:0/2255132758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 msgr2=0x55baa8145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8145800 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe09800bd50 tx=0x7fe0980107a0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 -- 172.21.15.31:0/2255132758 shutdown_connections 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d7d390 0x55baa8145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8145800 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 --2- 172.21.15.31:0/2255132758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d9e070 0x55baa81452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 -- 172.21.15.31:0/2255132758 >> 172.21.15.31:0/2255132758 conn(0x55baa7d92460 msgr2=0x55baa81431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 -- 172.21.15.31:0/2255132758 shutdown_connections 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 -- 172.21.15.31:0/2255132758 wait complete. 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.573+0000 7fe0aa630740 1 -- start start 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 Processor -- start 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8150980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55baa7f0c700 con 0x55baa7d751b0 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55baa8046160 con 0x55baa7d9e070 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55baa8153aa0 con 0x55baa7d7d390 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0aa630740 10 client.0 _ll_register_callbacks cb 0x55baa805f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0a1657640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0a1657640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44800/0 (socket says 172.21.15.31:44800) 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0a1657640 1 -- 172.21.15.31:0/1682582008 learned_addr learned my addr 172.21.15.31:0/1682582008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.574+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 -- 172.21.15.31:0/1682582008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 msgr2=0x55baa8150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa8150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 -- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 msgr2=0x55baa815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 -- 172.21.15.31:0/1682582008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa8073c90 con 0x55baa7d751b0 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0a1657640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8150980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe08c032fd0 tx=0x7fe08c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2154d6320c56f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0857fa640 1 -- 172.21.15.31:0/1682582008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c035070 con 0x55baa7d751b0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0857fa640 1 -- 172.21.15.31:0/1682582008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fe08c004300 con 0x55baa7d751b0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0980047a0 con 0x55baa7d751b0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0857fa640 1 -- 172.21.15.31:0/1682582008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe08c051050 con 0x55baa7d751b0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0aa630740 10 client.17586 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.575+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55baa815f7b0 con 0x55baa7d751b0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0857fa640 1 -- 172.21.15.31:0/1682582008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0980047a0 con 0x55baa7d751b0 2024-09-21T03:01:40.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0857fa640 1 -- 172.21.15.31:0/1682582008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe08c004b20 con 0x55baa7d751b0 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0aa630740 10 client.17586 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0aa630740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0aa630740 20 client.17586 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0aa630740 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d5c9f0 0x55baa8162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0aa630740 4 client.17586 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa80bddf0 con 0x55baa7d5c9f0 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.576+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d5c9f0 0x55baa8162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.577+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d5c9f0 0x55baa8162c10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6290ef0b28396787 server_cookie=dbaf408baeb3be4d in_seq=0 out_seq=0 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.577+0000 7fe0857fa640 10 client.17586 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0857fa640 1 -- 172.21.15.31:0/1682582008 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55baa80bddf0 con 0x55baa7d5c9f0 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0857fa640 10 client.17586 handle_command_reply: tid=0 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 client.17586 shutdown 2024-09-21T03:01:40.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 msgr2=0x55baa8150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8150980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe08c032fd0 tx=0x7fe08c009260 comp rx=0 tx=0).stop 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 shutdown_connections 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d5c9f0 0x55baa8162c10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa8150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0857fa640 0 client.17586 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 --2- 172.21.15.31:0/1682582008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d751b0 0x55baa8150980 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 >> 172.21.15.31:0/1682582008 conn(0x55baa7d92460 msgr2=0x55baa8148450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 shutdown_connections 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 1 -- 172.21.15.31:0/1682582008 wait complete. 2024-09-21T03:01:40.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.578+0000 7fe0aa630740 20 client.17586 trim_cache size 0 max 16384 2024-09-21T03:01:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 Processor -- start 2024-09-21T03:01:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 -- start start 2024-09-21T03:01:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d751b0 0x55baa8146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 0x55baa81525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1af840 con 0x55baa7d7d390 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1b1380 con 0x55baa7d9e070 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0aa630740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1aff10 con 0x55baa7d751b0 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0a1e58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 0x55baa81525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0a1e58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 0x55baa81525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44806/0 (socket says 172.21.15.31:44806) 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0a1e58640 1 -- 172.21.15.31:0/3786433155 learned_addr learned my addr 172.21.15.31:0/3786433155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.592+0000 7fe0a1657640 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d751b0 0x55baa8146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1657640 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1e58640 1 -- 172.21.15.31:0/3786433155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d751b0 msgr2=0x55baa8146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d751b0 0x55baa8146cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1e58640 1 -- 172.21.15.31:0/3786433155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 msgr2=0x55baa815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1e58640 1 -- 172.21.15.31:0/3786433155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa814ee80 con 0x55baa7d7d390 2024-09-21T03:01:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1657640 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 0x55baa81525b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b630 tx=0x7fe09800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4731e87d9e66bc67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0877fe640 1 -- 172.21.15.31:0/3786433155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098004220 con 0x55baa7d7d390 2024-09-21T03:01:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0877fe640 1 -- 172.21.15.31:0/3786433155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0980043c0 con 0x55baa7d7d390 2024-09-21T03:01:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.593+0000 7fe0877fe640 1 -- 172.21.15.31:0/3786433155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098004d60 con 0x55baa7d7d390 2024-09-21T03:01:40.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 -- 172.21.15.31:0/3786433155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 msgr2=0x55baa81525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 0x55baa81525b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b630 tx=0x7fe09800b410 comp rx=0 tx=0).stop 2024-09-21T03:01:40.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 -- 172.21.15.31:0/3786433155 shutdown_connections 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d9e070 0x55baa815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d7d390 0x55baa81525b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 --2- 172.21.15.31:0/3786433155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d751b0 0x55baa8146cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 -- 172.21.15.31:0/3786433155 >> 172.21.15.31:0/3786433155 conn(0x55baa7d92460 msgr2=0x55baa8151ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 -- 172.21.15.31:0/3786433155 shutdown_connections 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 -- 172.21.15.31:0/3786433155 wait complete. 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.594+0000 7fe0aa630740 1 -- start start 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 Processor -- start 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d751b0 0x55baa8157140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa81576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 0x55baa81d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1b0cb0 con 0x55baa7d9e070 2024-09-21T03:01:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1b05e0 con 0x55baa7d751b0 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1ad820 con 0x55baa7d7d390 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0aa630740 10 client.0 _ll_register_callbacks cb 0x7fe08c0314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0a1657640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 0x55baa81d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0a1657640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 0x55baa81d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44808/0 (socket says 172.21.15.31:44808) 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0a1657640 1 -- 172.21.15.31:0/4217756740 learned_addr learned my addr 172.21.15.31:0/4217756740 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0a1657640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d751b0 0x55baa8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.595+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa81576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0a1657640 1 -- 172.21.15.31:0/4217756740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 msgr2=0x55baa81576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0a1657640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa81576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0a1657640 1 -- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d751b0 msgr2=0x55baa8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0a1657640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d751b0 0x55baa8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0a1657640 1 -- 172.21.15.31:0/4217756740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa81d0f30 con 0x55baa7d9e070 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0a1657640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 0x55baa81d09f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009850 tx=0x7fe08c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd36cc36fc5aa04c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0857fa640 1 -- 172.21.15.31:0/4217756740 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c035040 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0857fa640 1 -- 172.21.15.31:0/4217756740 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fe08c003cc0 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0980043c0 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0857fa640 1 -- 172.21.15.31:0/4217756740 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe08c004100 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0aa630740 10 client.17592 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55baa81d1ff0 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.596+0000 7fe0857fa640 1 -- 172.21.15.31:0/4217756740 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0980043c0 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0857fa640 1 -- 172.21.15.31:0/4217756740 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe08c005000 con 0x55baa7d9e070 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0aa630740 10 client.17592 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0aa630740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0aa630740 20 client.17592 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0aa630740 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d31da0 0x55baa81d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0aa630740 4 client.17592 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:01:40.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55baa80b98a0 con 0x55baa7d31da0 2024-09-21T03:01:40.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.597+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d31da0 0x55baa81d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0a1e58640 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d31da0 0x55baa81d46f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538df17c499bdfcb server_cookie=e494d4863bc9bd56 in_seq=0 out_seq=0 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0857fa640 10 client.17592 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0857fa640 1 -- 172.21.15.31:0/4217756740 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55baa80b98a0 con 0x55baa7d31da0 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0857fa640 10 client.17592 handle_command_reply: tid=0 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0aa630740 1 client.17592 shutdown 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 msgr2=0x55baa81d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.598+0000 7fe0aa630740 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 0x55baa81d09f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009850 tx=0x7fe08c009260 comp rx=0 tx=0).stop 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 shutdown_connections 2024-09-21T03:01:40.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55baa7d31da0 0x55baa81d46f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0857fa640 0 client.17592 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55baa7d7d390 0x55baa81576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55baa7d9e070 0x55baa81d09f0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 --2- 172.21.15.31:0/4217756740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55baa7d751b0 0x55baa8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 >> 172.21.15.31:0/4217756740 conn(0x55baa7d92460 msgr2=0x55baa8146a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 shutdown_connections 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 1 -- 172.21.15.31:0/4217756740 wait complete. 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.599+0000 7fe0aa630740 20 client.17592 trim_cache size 0 max 16384 2024-09-21T03:01:40.604 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe0880b6680 msgr2=0x7fe0880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe0880b6680 0x7fe0880b8b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe09400dfd0 tx=0x7fe094058920 comp rx=0 tx=0).stop 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 msgr2=0x7fe09c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 0x7fe09c19fcd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe094009fd0 tx=0x7fe09400b260 comp rx=0 tx=0).stop 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 shutdown_connections 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fe0880b6680 0x7fe0880b8b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c109e20 0x7fe09c19fcd0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe09c1094d0 0x7fe09c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 --2- 172.21.15.31:0/87133307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe09c103b90 0x7fe09c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 >> 172.21.15.31:0/87133307 conn(0x7fe09c0ff600 msgr2=0x7fe09c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 shutdown_connections 2024-09-21T03:01:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:40.600+0000 7fe0a40bc640 1 -- 172.21.15.31:0/87133307 wait complete. 2024-09-21T03:01:40.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:40 smithi178 ceph-mon[34759]: pgmap v592: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:40 smithi031 ceph-mon[31209]: pgmap v592: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:40 smithi165 ceph-mon[34818]: pgmap v592: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:42.272 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:01:42.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:01:42.546 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:42.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:42 smithi031 ceph-mon[31209]: pgmap v593: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:42.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:42 smithi178 ceph-mon[34759]: pgmap v593: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 -- 172.21.15.31:0/1142214024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 msgr2=0x7fc584103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 --2- 172.21.15.31:0/1142214024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc584103f70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc574009900 tx=0x7fc57402a410 comp rx=0 tx=0).stop 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 -- 172.21.15.31:0/1142214024 shutdown_connections 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 --2- 172.21.15.31:0/1142214024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584109df0 0x7fc584110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 --2- 172.21.15.31:0/1142214024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5841094a0 0x7fc5841098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 --2- 172.21.15.31:0/1142214024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc584103f70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 -- 172.21.15.31:0/1142214024 >> 172.21.15.31:0/1142214024 conn(0x7fc58407b850 msgr2=0x7fc58407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 -- 172.21.15.31:0/1142214024 shutdown_connections 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.066+0000 7fc58acc1640 1 -- 172.21.15.31:0/1142214024 wait complete. 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 Processor -- start 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 -- start start 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc58419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5841094a0 0x7fc58419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584109df0 0x7fc58419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc584113ab0 con 0x7fc584103b60 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc584113930 con 0x7fc5841094a0 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.067+0000 7fc58acc1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc584113c30 con 0x7fc584109df0 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc58419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc58419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44820/0 (socket says 172.21.15.31:44820) 2024-09-21T03:01:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 -- 172.21.15.31:0/2237100097 learned_addr learned my addr 172.21.15.31:0/2237100097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc58925e640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584109df0 0x7fc58419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc583fff640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5841094a0 0x7fc58419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 -- 172.21.15.31:0/2237100097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584109df0 msgr2=0x7fc58419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584109df0 0x7fc58419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 -- 172.21.15.31:0/2237100097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5841094a0 msgr2=0x7fc58419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5841094a0 0x7fc58419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 -- 172.21.15.31:0/2237100097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5841a03f0 con 0x7fc584103b60 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc588a5d640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc58419a480 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc574005940 tx=0x7fc574004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc581ffb640 1 -- 172.21.15.31:0/2237100097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57403f030 con 0x7fc584103b60 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc581ffb640 1 -- 172.21.15.31:0/2237100097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc574003aa0 con 0x7fc584103b60 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841a0680 con 0x7fc584103b60 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.068+0000 7fc581ffb640 1 -- 172.21.15.31:0/2237100097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574035070 con 0x7fc584103b60 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.069+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a6e40 con 0x7fc584103b60 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.071+0000 7fc591235740 1 Processor -- start 2024-09-21T03:01:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.071+0000 7fc591235740 1 -- start start 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.071+0000 7fc581ffb640 1 -- 172.21.15.31:0/2237100097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fc574042030 con 0x7fc584103b60 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc591235740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0276430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc591235740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f0276970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc591235740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f0276eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc591235740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5652f01ece60 con 0x5652efeae670 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc591235740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5652f01dc290 con 0x5652efea6490 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc591235740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5652f01e00a0 con 0x5652efecf350 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f0276eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f0276eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44832/0 (socket says 172.21.15.31:44832) 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 -- 172.21.15.31:0/3653886156 learned_addr learned my addr 172.21.15.31:0/3653886156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc58925e640 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0276430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f0276970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 -- 172.21.15.31:0/3653886156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 msgr2=0x5652f0276430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0276430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 -- 172.21.15.31:0/3653886156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 msgr2=0x5652f0276970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f0276970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc583fff640 1 -- 172.21.15.31:0/3653886156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652eff3a930 con 0x5652efeae670 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc58925e640 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0276430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.072+0000 7fc581ffb640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc56c0b6680 0x7fc56c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.073+0000 7fc583fff640 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f0276eb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc578009bb0 tx=0x7fc5780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af891e606830eaa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.073+0000 7fc58925e640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc56c0b6680 0x7fc56c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.073+0000 7fc581ffb640 1 -- 172.21.15.31:0/2237100097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fc574107e30 con 0x7fc584103b60 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.073+0000 7fc56affd640 1 -- 172.21.15.31:0/3653886156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578012450 con 0x5652efeae670 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.073+0000 7fc56affd640 1 -- 172.21.15.31:0/3653886156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc578012aa0 con 0x5652efeae670 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.073+0000 7fc56affd640 1 -- 172.21.15.31:0/3653886156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57800f910 con 0x5652efeae670 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 -- 172.21.15.31:0/3653886156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 msgr2=0x5652f0276eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc58925e640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc56c0b6680 0x7fc56c0b8b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc57c005fd0 tx=0x7fc57c009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f0276eb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc578009bb0 tx=0x7fc5780117b0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 -- 172.21.15.31:0/3653886156 shutdown_connections 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f0276eb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f0276970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 --2- 172.21.15.31:0/3653886156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0276430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 -- 172.21.15.31:0/3653886156 >> 172.21.15.31:0/3653886156 conn(0x5652efec3740 msgr2=0x5652f0274320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:43.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 -- 172.21.15.31:0/3653886156 shutdown_connections 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 -- 172.21.15.31:0/3653886156 wait complete. 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 -- start start 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.074+0000 7fc591235740 1 Processor -- start 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0286410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efeae670 0x5652f0286950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f02922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5652f003d890 con 0x5652efea6490 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5652f0176d40 con 0x5652efeae670 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5652f0288f70 con 0x5652efecf350 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc591235740 10 client.0 _ll_register_callbacks cb 0x5652f01dac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0286410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0286410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44838/0 (socket says 172.21.15.31:44838) 2024-09-21T03:01:43.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 -- 172.21.15.31:0/3558682775 learned_addr learned my addr 172.21.15.31:0/3558682775 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc58925e640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f02922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc583fff640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efeae670 0x5652f0286950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 -- 172.21.15.31:0/3558682775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 msgr2=0x5652f02922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f02922d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 -- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efeae670 msgr2=0x5652f0286950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efeae670 0x5652f0286950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc588a5d640 1 -- 172.21.15.31:0/3558682775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f0292890 con 0x5652efea6490 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.075+0000 7fc58925e640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f02922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0286410 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc5740cdc10 tx=0x7fc574003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7319ba06efa31955 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc563fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc563fff640 1 -- 172.21.15.31:0/3558682775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574049070 con 0x5652efea6490 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc563fff640 1 -- 172.21.15.31:0/3558682775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fc5740f10b0 con 0x5652efea6490 2024-09-21T03:01:43.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc578012aa0 con 0x5652efea6490 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc563fff640 1 -- 172.21.15.31:0/3558682775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5740ca2d0 con 0x5652efea6490 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 10 client.17604 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652f02951c0 con 0x5652efea6490 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc563fff640 1 -- 172.21.15.31:0/3558682775 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc578012aa0 con 0x5652efea6490 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc563fff640 1 -- 172.21.15.31:0/3558682775 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc5740d7210 con 0x5652efea6490 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 10 client.17604 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 20 client.17604 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe8dcd0 0x5652f0298780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 4 client.17604 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.076+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f01eedb0 con 0x5652efe8dcd0 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.077+0000 7fc583fff640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe8dcd0 0x5652f0298780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.077+0000 7fc583fff640 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe8dcd0 0x5652f0298780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a20830d30772d73 server_cookie=ed660515541b3e5d in_seq=0 out_seq=0 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.077+0000 7fc563fff640 10 client.17604 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc563fff640 1 -- 172.21.15.31:0/3558682775 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652f01eedb0 con 0x5652efe8dcd0 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc563fff640 10 client.17604 handle_command_reply: tid=0 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 client.17604 shutdown 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 msgr2=0x5652f0286410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0286410 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc5740cdc10 tx=0x7fc574003e60 comp rx=0 tx=0).stop 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 shutdown_connections 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe8dcd0 0x5652f0298780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f02922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efeae670 0x5652f0286950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc563fff640 0 client.17604 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 --2- 172.21.15.31:0/3558682775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0286410 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 >> 172.21.15.31:0/3558682775 conn(0x5652efec3740 msgr2=0x5652f0272850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 shutdown_connections 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 1 -- 172.21.15.31:0/3558682775 wait complete. 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.078+0000 7fc591235740 20 client.17604 trim_cache size 0 max 16384 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 Processor -- start 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 -- start start 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0284a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efeae670 0x5652f0284fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efecf350 0x5652f0272850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ad4b0 con 0x5652efea6490 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841aeff0 con 0x5652efecf350 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc591235740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841adb80 con 0x5652efeae670 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc588a5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0284a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc588a5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0284a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44842/0 (socket says 172.21.15.31:44842) 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc588a5d640 1 -- 172.21.15.31:0/462386361 learned_addr learned my addr 172.21.15.31:0/462386361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc583fff640 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efeae670 0x5652f0284fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc58925e640 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efecf350 0x5652f0272850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc583fff640 1 -- 172.21.15.31:0/462386361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efecf350 msgr2=0x5652f0272850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc583fff640 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efecf350 0x5652f0272850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc583fff640 1 -- 172.21.15.31:0/462386361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 msgr2=0x5652f0284a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc583fff640 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0284a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.097+0000 7fc583fff640 1 -- 172.21.15.31:0/462386361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f0286db0 con 0x5652efeae670 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.098+0000 7fc588a5d640 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0284a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.098+0000 7fc583fff640 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efeae670 0x5652f0284fd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5652f0288bc0 tx=0x7fc57800f550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11a875444ee1ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.098+0000 7fc56a7fc640 1 -- 172.21.15.31:0/462386361 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578030070 con 0x5652efeae670 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.098+0000 7fc56a7fc640 1 -- 172.21.15.31:0/462386361 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5780095a0 con 0x5652efeae670 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.098+0000 7fc56a7fc640 1 -- 172.21.15.31:0/462386361 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578005600 con 0x5652efeae670 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 -- 172.21.15.31:0/462386361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efeae670 msgr2=0x5652f0284fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efeae670 0x5652f0284fd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5652f0288bc0 tx=0x7fc57800f550 comp rx=0 tx=0).stop 2024-09-21T03:01:43.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 -- 172.21.15.31:0/462386361 shutdown_connections 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efeae670 0x5652f0284fd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efecf350 0x5652f0272850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 --2- 172.21.15.31:0/462386361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efea6490 0x5652f0284a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 -- 172.21.15.31:0/462386361 >> 172.21.15.31:0/462386361 conn(0x5652efec3740 msgr2=0x5652f0281350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 -- 172.21.15.31:0/462386361 shutdown_connections 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 -- 172.21.15.31:0/462386361 wait complete. 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.099+0000 7fc591235740 1 -- start start 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 Processor -- start 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f028e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f028ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0289300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ae920 con 0x5652efeae670 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ae250 con 0x5652efea6490 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841ab490 con 0x5652efecf350 2024-09-21T03:01:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc591235740 10 client.0 _ll_register_callbacks cb 0x7fc574005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc583fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f028ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc583fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f028ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44850/0 (socket says 172.21.15.31:44850) 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc583fff640 1 -- 172.21.15.31:0/3754578835 learned_addr learned my addr 172.21.15.31:0/3754578835 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f028e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.100+0000 7fc58925e640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0289300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc583fff640 1 -- 172.21.15.31:0/3754578835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 msgr2=0x5652f0289300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc583fff640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0289300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc583fff640 1 -- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 msgr2=0x5652f028e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc583fff640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f028e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc583fff640 1 -- 172.21.15.31:0/3754578835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f0289a90 con 0x5652efeae670 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f028e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc583fff640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f028ebb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc57800f520 tx=0x7fc578005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac79a083984e6e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc563fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:43.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc563fff640 1 -- 172.21.15.31:0/3754578835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578030070 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc563fff640 1 -- 172.21.15.31:0/3754578835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fc5780124d0 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652f01ea860 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc563fff640 1 -- 172.21.15.31:0/3754578835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5780126d0 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc591235740 10 client.17610 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652f0303ca0 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.101+0000 7fc563fff640 1 -- 172.21.15.31:0/3754578835 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5652f01ea860 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc563fff640 1 -- 172.21.15.31:0/3754578835 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc57802b040 con 0x5652efeae670 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc591235740 10 client.17610 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc591235740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc591235740 20 client.17610 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc591235740 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe63080 0x5652f0306150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc591235740 4 client.17610 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:01:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5652f01e7c90 con 0x5652efe63080 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe63080 0x5652f0306150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc563fff640 10 client.17610 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.102+0000 7fc588a5d640 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe63080 0x5652f0306150 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3a8eecb6126ea server_cookie=c096a421418baa93 in_seq=0 out_seq=0 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc563fff640 1 -- 172.21.15.31:0/3754578835 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5652f01e7c90 con 0x5652efe63080 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc563fff640 10 client.17610 handle_command_reply: tid=0 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 client.17610 shutdown 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 msgr2=0x5652f028ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f028ebb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc57800f520 tx=0x7fc578005cd0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 shutdown_connections 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5652efe63080 0x5652f0306150 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc563fff640 0 client.17610 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5652efecf350 0x5652f0289300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652efeae670 0x5652f028ebb0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 --2- 172.21.15.31:0/3754578835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652efea6490 0x5652f028e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.104+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 >> 172.21.15.31:0/3754578835 conn(0x5652efec3740 msgr2=0x5652f0284a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 shutdown_connections 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc591235740 1 -- 172.21.15.31:0/3754578835 wait complete. 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc591235740 20 client.17610 trim_cache size 0 max 16384 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "6c1e1d72-fabe-4268-a17c-b4153b12ad02", 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T03:01:43.117 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:01:43.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc56c0b6680 msgr2=0x7fc56c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc56c0b6680 0x7fc56c0b8b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc57c005fd0 tx=0x7fc57c009600 comp rx=0 tx=0).stop 2024-09-21T03:01:43.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 msgr2=0x7fc58419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:43.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc58419a480 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc574005940 tx=0x7fc574004270 comp rx=0 tx=0).stop 2024-09-21T03:01:43.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 shutdown_connections 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fc56c0b6680 0x7fc56c0b8b90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc584109df0 0x7fc58419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5841094a0 0x7fc58419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 --2- 172.21.15.31:0/2237100097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc584103b60 0x7fc58419a480 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.105+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 >> 172.21.15.31:0/2237100097 conn(0x7fc58407b850 msgr2=0x7fc584107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.106+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 shutdown_connections 2024-09-21T03:01:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:43.106+0000 7fc58acc1640 1 -- 172.21.15.31:0/2237100097 wait complete. 2024-09-21T03:01:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:42 smithi165 ceph-mon[34818]: pgmap v593: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:01:43.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:01:43.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:43 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T03:01:43.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:43 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:43.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:43 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T03:01:43.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:43 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:43 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T03:01:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:43 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:43 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T03:01:44.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:43 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:43 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T03:01:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:43 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:43 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T03:01:44.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:43 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:44.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 -- 172.21.15.31:0/3447934572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c108d70 msgr2=0x7fb11c10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 --2- 172.21.15.31:0/3447934572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c108d70 0x7fb11c10f5a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb114009ca0 tx=0x7fb11402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 -- 172.21.15.31:0/3447934572 shutdown_connections 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 --2- 172.21.15.31:0/3447934572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c108d70 0x7fb11c10f5a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 --2- 172.21.15.31:0/3447934572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11c108420 0x7fb11c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 --2- 172.21.15.31:0/3447934572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11c102ae0 0x7fb11c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 -- 172.21.15.31:0/3447934572 >> 172.21.15.31:0/3447934572 conn(0x7fb11c079770 msgr2=0x7fb11c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 -- 172.21.15.31:0/3447934572 shutdown_connections 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.684+0000 7fb1225aa640 1 -- 172.21.15.31:0/3447934572 wait complete. 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 Processor -- start 2024-09-21T03:01:44.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 -- start start 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 0x7fb11c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11c108420 0x7fb11c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11c108d70 0x7fb11c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c112a80 con 0x7fb11c102ae0 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c112900 con 0x7fb11c108d70 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.685+0000 7fb1225aa640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c112c00 con 0x7fb11c108420 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 0x7fb11c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 0x7fb11c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44870/0 (socket says 172.21.15.31:44870) 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 -- 172.21.15.31:0/223442146 learned_addr learned my addr 172.21.15.31:0/223442146 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11c108420 0x7fb11c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb120b47640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11c108d70 0x7fb11c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 -- 172.21.15.31:0/223442146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11c108420 msgr2=0x7fb11c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11c108420 0x7fb11c19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 -- 172.21.15.31:0/223442146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11c108d70 msgr2=0x7fb11c19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11c108d70 0x7fb11c19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 -- 172.21.15.31:0/223442146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c1a65e0 con 0x7fb11c102ae0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb11bfff640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 0x7fb11c19b350 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00d950 tx=0x7fb10c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb1197fa640 1 -- 172.21.15.31:0/223442146 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c017030 con 0x7fb11c102ae0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb1197fa640 1 -- 172.21.15.31:0/223442146 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb10c004020 con 0x7fb11c102ae0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.686+0000 7fb1197fa640 1 -- 172.21.15.31:0/223442146 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0047a0 con 0x7fb11c102ae0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.687+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c1a68d0 con 0x7fb11c102ae0 2024-09-21T03:01:44.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.687+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c1a6e60 con 0x7fb11c102ae0 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.689+0000 7fb128b1e740 1 Processor -- start 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.689+0000 7fb128b1e740 1 -- start start 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.689+0000 7fb1197fa640 1 -- 172.21.15.31:0/223442146 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb10c009a30 con 0x7fb11c102ae0 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 0x56362eb28650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e7581b0 0x56362eb28b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb128b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56362ea9ee50 con 0x56362e781070 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb128b1e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56362ea8e280 con 0x56362e7581b0 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb128b1e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56362ea92090 con 0x56362e760390 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 0x56362eb28650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 0x56362eb28650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44874/0 (socket says 172.21.15.31:44874) 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 -- 172.21.15.31:0/390610500 learned_addr learned my addr 172.21.15.31:0/390610500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb11bfff640 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e7581b0 0x56362eb28b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 -- 172.21.15.31:0/390610500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 msgr2=0x56362eb290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb290d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 -- 172.21.15.31:0/390610500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e7581b0 msgr2=0x56362eb28b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e7581b0 0x56362eb28b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb120b47640 1 -- 172.21.15.31:0/390610500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362e7ec650 con 0x56362e781070 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.690+0000 7fb11bfff640 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e7581b0 0x56362eb28b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb120b47640 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 0x56362eb28650 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb114005e30 tx=0x7fb114009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994db602149c0b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb1027fc640 1 -- 172.21.15.31:0/390610500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114031cf0 con 0x56362e781070 2024-09-21T03:01:44.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb1197fa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb1040b6730 0x7fb1040b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb1027fc640 1 -- 172.21.15.31:0/390610500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb114036040 con 0x56362e781070 2024-09-21T03:01:44.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb1027fc640 1 -- 172.21.15.31:0/390610500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1140042a0 con 0x56362e781070 2024-09-21T03:01:44.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb11bfff640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb1040b6730 0x7fb1040b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.691+0000 7fb1197fa640 1 -- 172.21.15.31:0/223442146 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fb10c0e70e0 con 0x7fb11c102ae0 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 -- 172.21.15.31:0/390610500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 msgr2=0x56362eb28650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 0x56362eb28650 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb114005e30 tx=0x7fb114009800 comp rx=0 tx=0).stop 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb11bfff640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb1040b6730 0x7fb1040b8c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56362eb272d0 tx=0x7fb10c094040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 -- 172.21.15.31:0/390610500 shutdown_connections 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e7581b0 0x56362eb28b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 --2- 172.21.15.31:0/390610500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e781070 0x56362eb28650 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 -- 172.21.15.31:0/390610500 >> 172.21.15.31:0/390610500 conn(0x56362e775460 msgr2=0x56362eb26540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 -- 172.21.15.31:0/390610500 shutdown_connections 2024-09-21T03:01:44.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 -- 172.21.15.31:0/390610500 wait complete. 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.692+0000 7fb128b1e740 1 -- start start 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 Processor -- start 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb33d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb34270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56362e8ef700 con 0x56362e7581b0 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56362ea29110 con 0x56362e781070 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56362eb3b1e0 con 0x56362e760390 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb128b1e740 10 client.0 _ll_register_callbacks cb 0x56362ea42990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb120b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb33d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb120b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb33d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44876/0 (socket says 172.21.15.31:44876) 2024-09-21T03:01:44.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb120b47640 1 -- 172.21.15.31:0/368574968 learned_addr learned my addr 172.21.15.31:0/368574968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.693+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb34270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb120b47640 1 -- 172.21.15.31:0/368574968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 msgr2=0x56362eb34270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb120b47640 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb34270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb120b47640 1 -- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 msgr2=0x56362eb44500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb120b47640 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb120b47640 1 -- 172.21.15.31:0/368574968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362ea56c40 con 0x56362e7581b0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb120b47640 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb33d30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb11402a900 tx=0x7fb1140039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316918c890107970 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb0fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/368574968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114031d00 con 0x56362e7581b0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/368574968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fb114031450 con 0x56362e7581b0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb114036040 con 0x56362e7581b0 2024-09-21T03:01:44.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/368574968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb11405d050 con 0x56362e7581b0 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb128b1e740 10 client.17622 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56362eb46ee0 con 0x56362e7581b0 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.694+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/368574968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb114036040 con 0x56362e7581b0 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/368574968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb114058030 con 0x56362e7581b0 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb128b1e740 10 client.17622 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb128b1e740 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb128b1e740 20 client.17622 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb128b1e740 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e73f9f0 0x56362eb4a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb128b1e740 4 client.17622 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56362eaa0da0 con 0x56362e73f9f0 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e73f9f0 0x56362eb4a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e73f9f0 0x56362eb4a240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe2e8a5f4a7fa2f server_cookie=9eaa02be5c8eb4dc in_seq=0 out_seq=0 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.695+0000 7fb0fb7fe640 10 client.17622 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/368574968 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56362eaa0da0 con 0x56362e73f9f0 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb0fb7fe640 10 client.17622 handle_command_reply: tid=0 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 client.17622 shutdown 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 msgr2=0x56362eb33d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb33d30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb11402a900 tx=0x7fb1140039f0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 shutdown_connections 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e73f9f0 0x56362eb4a240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb34270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.696+0000 7fb128b1e740 1 --2- 172.21.15.31:0/368574968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb33d30 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.697+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 >> 172.21.15.31:0/368574968 conn(0x56362e775460 msgr2=0x56362eb2b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.697+0000 7fb0fb7fe640 0 client.17622 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:44.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.697+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 shutdown_connections 2024-09-21T03:01:44.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.697+0000 7fb128b1e740 1 -- 172.21.15.31:0/368574968 wait complete. 2024-09-21T03:01:44.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.697+0000 7fb128b1e740 20 client.17622 trim_cache size 0 max 16384 2024-09-21T03:01:44.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 Processor -- start 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 -- start start 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e7581b0 0x56362eb2a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 0x56362eb35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1ad5d0 con 0x56362e760390 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1af110 con 0x56362e781070 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.714+0000 7fb128b1e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1adca0 con 0x56362e7581b0 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 0x56362eb35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 0x56362eb35c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44878/0 (socket says 172.21.15.31:44878) 2024-09-21T03:01:44.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 -- 172.21.15.31:0/2013165886 learned_addr learned my addr 172.21.15.31:0/2013165886 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb120b47640 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e7581b0 0x56362eb2a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb120b47640 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 -- 172.21.15.31:0/2013165886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e7581b0 msgr2=0x56362eb2a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e7581b0 0x56362eb2a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 -- 172.21.15.31:0/2013165886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 msgr2=0x56362eb44470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 -- 172.21.15.31:0/2013165886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362eb34ff0 con 0x56362e760390 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb120b47640 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 0x56362eb35c10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb11c06b9c0 tx=0x7fb11000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df34c85887ecc3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb101ffb640 1 -- 172.21.15.31:0/2013165886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110003cd0 con 0x56362e760390 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb101ffb640 1 -- 172.21.15.31:0/2013165886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb110003e70 con 0x56362e760390 2024-09-21T03:01:44.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.715+0000 7fb101ffb640 1 -- 172.21.15.31:0/2013165886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11001d470 con 0x56362e760390 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 -- 172.21.15.31:0/2013165886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 msgr2=0x56362eb35c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 0x56362eb35c10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb11c06b9c0 tx=0x7fb11000bca0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 -- 172.21.15.31:0/2013165886 shutdown_connections 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e760390 0x56362eb35c10 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 --2- 172.21.15.31:0/2013165886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e7581b0 0x56362eb2a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 -- 172.21.15.31:0/2013165886 >> 172.21.15.31:0/2013165886 conn(0x56362e775460 msgr2=0x56362eb2d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 -- 172.21.15.31:0/2013165886 shutdown_connections 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 -- 172.21.15.31:0/2013165886 wait complete. 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.716+0000 7fb128b1e740 1 -- start start 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 Processor -- start 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb4db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb4b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1aea40 con 0x56362e7581b0 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1ae370 con 0x56362e781070 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c1ab5b0 con 0x56362e760390 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb128b1e740 10 client.0 _ll_register_callbacks cb 0x7fb114009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb120b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb4db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb120b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb4db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44892/0 (socket says 172.21.15.31:44892) 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb120b47640 1 -- 172.21.15.31:0/1409315289 learned_addr learned my addr 172.21.15.31:0/1409315289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.717+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb4b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb120b47640 1 -- 172.21.15.31:0/1409315289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 msgr2=0x56362eb4b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb120b47640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb4b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb120b47640 1 -- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 msgr2=0x56362eb4b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb120b47640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb120b47640 1 -- 172.21.15.31:0/1409315289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362eb4bec0 con 0x56362e7581b0 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb4b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb120b47640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb4db90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb11402ade0 tx=0x7fb11402f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c197375c5825f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb0fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/1409315289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114065070 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/1409315289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8528+0+0 (secure 0 0 0) 0x7fb114031720 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb110003e70 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/1409315289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb114031920 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb128b1e740 10 client.17628 fetch_fsmap learned FSMap version 27 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56362ebb49c0 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.718+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/1409315289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb110003e70 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/1409315289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb114068030 con 0x56362e7581b0 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb128b1e740 10 client.17628 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb128b1e740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb128b1e740 20 client.17628 populate_metadata read hostname 'smithi031' 2024-09-21T03:01:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb128b1e740 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e714da0 0x56362ebb7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb128b1e740 4 client.17628 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56362ea9c850 con 0x56362e714da0 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e714da0 0x56362ebb7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb0fb7fe640 10 client.17628 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.719+0000 7fb11b7fe640 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e714da0 0x56362ebb7140 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd9170b038da52c server_cookie=a887ddd0bdca89d4 in_seq=0 out_seq=0 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb0fb7fe640 1 -- 172.21.15.31:0/1409315289 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x56362ea9c850 con 0x56362e714da0 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb0fb7fe640 10 client.17628 handle_command_reply: tid=0 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 client.17628 shutdown 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 msgr2=0x56362eb4db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb4db90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb11402ade0 tx=0x7fb11402f430 comp rx=0 tx=0).stop 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 shutdown_connections 2024-09-21T03:01:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x56362e714da0 0x56362ebb7140 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56362e760390 0x56362eb4b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56362e781070 0x56362eb4b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 --2- 172.21.15.31:0/1409315289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56362e7581b0 0x56362eb4db90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 >> 172.21.15.31:0/1409315289 conn(0x56362e775460 msgr2=0x56362eb35950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb0fb7fe640 0 client.17628 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 shutdown_connections 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 1 -- 172.21.15.31:0/1409315289 wait complete. 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.720+0000 7fb128b1e740 20 client.17628 trim_cache size 0 max 16384 2024-09-21T03:01:44.727 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "6c1e1d72-fabe-4268-a17c-b4153b12ad02": { 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "6c1e1d72-fabe-4268-a17c-b4153b12ad02", 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "6c1e1d72-fabe-4268-a17c-b4153b12ad02": { 2024-09-21T03:01:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:01:43.102904+0000", 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:58.094612+0000", 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:01:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:01:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:01:44.739 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:01:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb1040b6730 msgr2=0x7fb1040b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb1040b6730 0x7fb1040b8c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56362eb272d0 tx=0x7fb10c094040 comp rx=0 tx=0).stop 2024-09-21T03:01:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 msgr2=0x7fb11c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 0x7fb11c19b350 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00d950 tx=0x7fb10c00de10 comp rx=0 tx=0).stop 2024-09-21T03:01:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 shutdown_connections 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb1040b6730 0x7fb1040b8c40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11c108420 0x7fb11c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb11c108d70 0x7fb11c19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 --2- 172.21.15.31:0/223442146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb11c102ae0 0x7fb11c19b350 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 >> 172.21.15.31:0/223442146 conn(0x7fb11c079770 msgr2=0x7fb11c106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 shutdown_connections 2024-09-21T03:01:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:01:44.721+0000 7fb1225aa640 1 -- 172.21.15.31:0/223442146 wait complete. 2024-09-21T03:01:45.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:44 smithi031 ceph-mon[31209]: pgmap v594: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:01:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:44 smithi165 ceph-mon[34818]: pgmap v594: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:01:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:44 smithi178 ceph-mon[34759]: pgmap v594: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:01:45.621 INFO:tasks.cephfs.filesystem:scrub status for tag:6c1e1d72-fabe-4268-a17c-b4153b12ad02 - {'path': '/', 'tag': '6c1e1d72-fabe-4268-a17c-b4153b12ad02', 'options': 'recursive,force'} 2024-09-21T03:01:46.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:45 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T03:01:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:45 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T03:01:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:45 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T03:01:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:46 smithi031 ceph-mon[31209]: pgmap v595: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:46 smithi165 ceph-mon[34818]: pgmap v595: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:46 smithi178 ceph-mon[34759]: pgmap v595: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:01:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:48 smithi031 ceph-mon[31209]: pgmap v596: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:01:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:48 smithi165 ceph-mon[34818]: pgmap v596: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:01:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:48 smithi178 ceph-mon[34759]: pgmap v596: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:01:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:50 smithi031 ceph-mon[31209]: pgmap v597: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:01:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:50 smithi165 ceph-mon[34818]: pgmap v597: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:01:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:50 smithi178 ceph-mon[34759]: pgmap v597: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:01:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:52 smithi031 ceph-mon[31209]: pgmap v598: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T03:01:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:52 smithi165 ceph-mon[34818]: pgmap v598: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T03:01:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:52 smithi178 ceph-mon[34759]: pgmap v598: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T03:01:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:54 smithi031 ceph-mon[31209]: pgmap v599: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:01:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:54 smithi165 ceph-mon[34818]: pgmap v599: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:01:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:54 smithi178 ceph-mon[34759]: pgmap v599: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T03:01:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:56 smithi031 ceph-mon[31209]: pgmap v600: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T03:01:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:56 smithi165 ceph-mon[34818]: pgmap v600: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T03:01:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:56 smithi178 ceph-mon[34759]: pgmap v600: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T03:01:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:01:58 smithi031 ceph-mon[31209]: pgmap v601: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:01:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:01:58 smithi165 ceph-mon[34818]: pgmap v601: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:01:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:01:58 smithi178 ceph-mon[34759]: pgmap v601: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:02:01.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:00 smithi031 ceph-mon[31209]: pgmap v602: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:00 smithi165 ceph-mon[34818]: pgmap v602: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:00 smithi178 ceph-mon[34759]: pgmap v602: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:03.030 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:02 smithi178 ceph-mon[34759]: pgmap v603: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:02:03.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:02 smithi031 ceph-mon[31209]: pgmap v603: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:02:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:02 smithi165 ceph-mon[34818]: pgmap v603: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:02:05.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:04 smithi031 ceph-mon[31209]: pgmap v604: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:02:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:04 smithi165 ceph-mon[34818]: pgmap v604: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:02:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:04 smithi178 ceph-mon[34759]: pgmap v604: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:02:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:06 smithi031 ceph-mon[31209]: pgmap v605: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:02:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:06 smithi165 ceph-mon[34818]: pgmap v605: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:02:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:06 smithi178 ceph-mon[34759]: pgmap v605: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:02:09.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:08 smithi031 ceph-mon[31209]: pgmap v606: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:02:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:08 smithi165 ceph-mon[34818]: pgmap v606: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:02:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:08 smithi178 ceph-mon[34759]: pgmap v606: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T03:02:11.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:10 smithi031 ceph-mon[31209]: pgmap v607: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:02:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:10 smithi165 ceph-mon[34818]: pgmap v607: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:02:11.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:10 smithi178 ceph-mon[34759]: pgmap v607: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:02:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:11 smithi031 ceph-mon[31209]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:02:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:11 smithi165 ceph-mon[34818]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:02:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:11 smithi178 ceph-mon[34759]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:02:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:12 smithi031 ceph-mon[31209]: pgmap v609: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:02:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:12 smithi165 ceph-mon[34818]: pgmap v609: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:02:13.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:12 smithi178 ceph-mon[34759]: pgmap v609: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T03:02:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:14 smithi031 ceph-mon[31209]: pgmap v610: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:02:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:14 smithi165 ceph-mon[34818]: pgmap v610: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:02:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:14 smithi178 ceph-mon[34759]: pgmap v610: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-21T03:02:15.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:02:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:16.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:02:16.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:02:16.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:02:16.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:02:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:02:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:02:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:02:16.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:02:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:02:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:02:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:02:16.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:02:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 -- 172.21.15.31:0/2090076304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed88110180 msgr2=0x7fed881125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 --2- 172.21.15.31:0/2090076304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed88110180 0x7fed881125a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fed80009ca0 tx=0x7fed8002a950 comp rx=0 tx=0).stop 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 -- 172.21.15.31:0/2090076304 shutdown_connections 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 --2- 172.21.15.31:0/2090076304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed88110180 0x7fed881125a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 --2- 172.21.15.31:0/2090076304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed88102f20 0x7fed8810fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 --2- 172.21.15.31:0/2090076304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed881025d0 0x7fed881029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 -- 172.21.15.31:0/2090076304 >> 172.21.15.31:0/2090076304 conn(0x7fed88079730 msgr2=0x7fed88079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 -- 172.21.15.31:0/2090076304 shutdown_connections 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.369+0000 7fed8fe02640 1 -- 172.21.15.31:0/2090076304 wait complete. 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 Processor -- start 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 -- start start 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 0x7fed8819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed88102f20 0x7fed8819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed88110180 0x7fed881a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed88115c70 con 0x7fed881025d0 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed88115af0 con 0x7fed88110180 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8fe02640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed88115df0 con 0x7fed88102f20 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8db9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 0x7fed8819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8db9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 0x7fed8819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60686/0 (socket says 172.21.15.31:60686) 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8db9e640 1 -- 172.21.15.31:0/1780925902 learned_addr learned my addr 172.21.15.31:0/1780925902 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.370+0000 7fed8d39d640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed88102f20 0x7fed8819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8e39f640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed88110180 0x7fed881a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8db9e640 1 -- 172.21.15.31:0/1780925902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed88102f20 msgr2=0x7fed8819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8db9e640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed88102f20 0x7fed8819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8db9e640 1 -- 172.21.15.31:0/1780925902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed88110180 msgr2=0x7fed881a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8db9e640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed88110180 0x7fed881a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8db9e640 1 -- 172.21.15.31:0/1780925902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed881a2630 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8db9e640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 0x7fed8819c6c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fed8400e9f0 tx=0x7fed8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed7effd640 1 -- 172.21.15.31:0/1780925902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8400cd80 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed881a2920 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed7effd640 1 -- 172.21.15.31:0/1780925902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed840108e0 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed7effd640 1 -- 172.21.15.31:0/1780925902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84018980 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.371+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed881a9080 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 Processor -- start 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 -- start start 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed7effd640 1 -- 172.21.15.31:0/1780925902 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fed84010430 con 0x7fed881025d0 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505497650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050c71b0 0x560505497b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054980d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56050540de50 con 0x5605050f0070 2024-09-21T03:02:16.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605053fd280 con 0x5605050cf390 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed96376740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560505401090 con 0x5605050c71b0 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505497650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505497650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60700/0 (socket says 172.21.15.31:60700) 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.374+0000 7fed8e39f640 1 -- 172.21.15.31:0/1743845698 learned_addr learned my addr 172.21.15.31:0/1743845698 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8db9e640 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050c71b0 0x560505497b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8d39d640 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8e39f640 1 -- 172.21.15.31:0/1743845698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050c71b0 msgr2=0x560505497b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8e39f640 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050c71b0 0x560505497b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8e39f640 1 -- 172.21.15.31:0/1743845698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 msgr2=0x5605054980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8e39f640 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8e39f640 1 -- 172.21.15.31:0/1743845698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050515b650 con 0x5605050f0070 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed7effd640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed740b6680 0x7fed740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8d39d640 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8db9e640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed740b6680 0x7fed740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed8e39f640 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505497650 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed8002fdf0 tx=0x7fed8002fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a24feb5bd0a8825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed7effd640 1 -- 172.21.15.31:0/1780925902 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7fed840e7b30 con 0x7fed881025d0 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed73fff640 1 -- 172.21.15.31:0/1743845698 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80034040 con 0x5605050f0070 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed73fff640 1 -- 172.21.15.31:0/1743845698 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed80003e20 con 0x5605050f0070 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.375+0000 7fed73fff640 1 -- 172.21.15.31:0/1743845698 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed800044a0 con 0x5605050f0070 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 -- 172.21.15.31:0/1743845698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 msgr2=0x560505497650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed8db9e640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed740b6680 0x7fed740b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed840e5300 tx=0x7fed840eb680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505497650 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed8002fdf0 tx=0x7fed8002fe20 comp rx=0 tx=0).stop 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 -- 172.21.15.31:0/1743845698 shutdown_connections 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050c71b0 0x560505497b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 --2- 172.21.15.31:0/1743845698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505497650 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.376+0000 7fed96376740 1 -- 172.21.15.31:0/1743845698 >> 172.21.15.31:0/1743845698 conn(0x5605050e4460 msgr2=0x560505495540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 -- 172.21.15.31:0/1743845698 shutdown_connections 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 -- 172.21.15.31:0/1743845698 wait complete. 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 -- start start 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 Processor -- start 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 0x5605054a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.377+0000 7fed96376740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050f0070 0x5605054a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed96376740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56050525e700 con 0x5605050c71b0 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed96376740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560505398110 con 0x5605050cf390 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed96376740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5605054a5dd0 con 0x5605050f0070 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 0x5605054a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed96376740 10 client.0 _ll_register_callbacks cb 0x5605053b1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 0x5605054a2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60708/0 (socket says 172.21.15.31:60708) 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 -- 172.21.15.31:0/2542736747 learned_addr learned my addr 172.21.15.31:0/2542736747 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050f0070 0x5605054a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8d39d640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 -- 172.21.15.31:0/2542736747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050f0070 msgr2=0x5605054a37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050f0070 0x5605054a37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 -- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 msgr2=0x5605054a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 -- 172.21.15.31:0/2542736747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605053c5c40 con 0x5605050c71b0 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8d39d640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed8e39f640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 0x5605054a2cf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fed8002a990 tx=0x7fed8002b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69f2265189d5ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed717fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:16.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed717fa640 1 -- 172.21.15.31:0/2542736747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80034070 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed717fa640 1 -- 172.21.15.31:0/2542736747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7fed80003b80 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.378+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed80003e20 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed717fa640 1 -- 172.21.15.31:0/2542736747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed8002f660 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 10 client.17640 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5605054b1aa0 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed717fa640 1 -- 172.21.15.31:0/2542736747 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed80003e20 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed717fa640 1 -- 172.21.15.31:0/2542736747 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed8004c030 con 0x5605050c71b0 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 10 client.17640 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 20 client.17640 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5605050ae9f0 0x5605054b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 4 client.17640 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:16.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050540fda0 con 0x5605050ae9f0 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.379+0000 7fed8d39d640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5605050ae9f0 0x5605054b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.380+0000 7fed8d39d640 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5605050ae9f0 0x5605054b4e00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53e47cce2c1c16c server_cookie=7607b0df8dbd406b in_seq=0 out_seq=0 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.380+0000 7fed717fa640 10 client.17640 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.380+0000 7fed717fa640 1 -- 172.21.15.31:0/2542736747 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56050540fda0 con 0x5605050ae9f0 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.380+0000 7fed717fa640 10 client.17640 handle_command_reply: tid=0 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.380+0000 7fed96376740 1 client.17640 shutdown 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 msgr2=0x5605054a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 0x5605054a2cf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fed8002a990 tx=0x7fed8002b050 comp rx=0 tx=0).stop 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 shutdown_connections 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5605050ae9f0 0x5605054b4e00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050f0070 0x5605054a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050cf390 0x5605054a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 --2- 172.21.15.31:0/2542736747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050c71b0 0x5605054a2cf0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed717fa640 0 client.17640 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 >> 172.21.15.31:0/2542736747 conn(0x5605050e4460 msgr2=0x56050549a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 shutdown_connections 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 1 -- 172.21.15.31:0/2542736747 wait complete. 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.381+0000 7fed96376740 20 client.17640 trim_cache size 0 max 16384 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.398+0000 7fed96376740 1 Processor -- start 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T03:02:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "6c1e1d72-fabe-4268-a17c-b4153b12ad02": { 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:01:43.102904+0000", 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:58.094612+0000", 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:02:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:02:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:02:16.429 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 -- start start 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x560505499040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505494b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed881b19a0 con 0x5605050f0070 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed881b34e0 con 0x5605050c71b0 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed96376740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed881b2070 con 0x5605050cf390 2024-09-21T03:02:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505494b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505494b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60716/0 (socket says 172.21.15.31:60716) 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 -- 172.21.15.31:0/3666333810 learned_addr learned my addr 172.21.15.31:0/3666333810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8d39d640 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x560505499040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 -- 172.21.15.31:0/3666333810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 msgr2=0x5605054a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 -- 172.21.15.31:0/3666333810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 msgr2=0x560505499040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x560505499040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.399+0000 7fed8e39f640 1 -- 172.21.15.31:0/3666333810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605054a3fb0 con 0x5605050f0070 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.400+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505494b70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fed80002230 tx=0x7fed80009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8b1692acc594ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.400+0000 7fed737fe640 1 -- 172.21.15.31:0/3666333810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80034040 con 0x5605050f0070 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.400+0000 7fed737fe640 1 -- 172.21.15.31:0/3666333810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed80030ad0 con 0x5605050f0070 2024-09-21T03:02:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.400+0000 7fed737fe640 1 -- 172.21.15.31:0/3666333810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80058c10 con 0x5605050f0070 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.400+0000 7fed96376740 1 -- 172.21.15.31:0/3666333810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 msgr2=0x560505494b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.400+0000 7fed96376740 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505494b70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fed80002230 tx=0x7fed80009360 comp rx=0 tx=0).stop 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 -- 172.21.15.31:0/3666333810 shutdown_connections 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x560505494b70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 --2- 172.21.15.31:0/3666333810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x560505499040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 -- 172.21.15.31:0/3666333810 >> 172.21.15.31:0/3666333810 conn(0x5605050e4460 msgr2=0x5605054a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 -- 172.21.15.31:0/3666333810 shutdown_connections 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 -- 172.21.15.31:0/3666333810 wait complete. 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 -- start start 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.401+0000 7fed96376740 1 Processor -- start 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x5605054b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x5605054b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed881b2e10 con 0x5605050f0070 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed881b2740 con 0x5605050c71b0 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fed881af9a0 con 0x5605050cf390 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed96376740 10 client.0 _ll_register_callbacks cb 0x7fed840eb550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x5605054b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x5605054b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60730/0 (socket says 172.21.15.31:60730) 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 -- 172.21.15.31:0/3835902310 learned_addr learned my addr 172.21.15.31:0/3835902310 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x5605054b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8d39d640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 -- 172.21.15.31:0/3835902310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 msgr2=0x5605054b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 -- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 msgr2=0x5605054b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x5605054b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 -- 172.21.15.31:0/3835902310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050551e580 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed8e39f640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x5605054b6850 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fed8002ae70 tx=0x7fed80009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac67740ec656af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed717fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.402+0000 7fed717fa640 1 -- 172.21.15.31:0/3835902310 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8006a020 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed717fa640 1 -- 172.21.15.31:0/3835902310 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7fed800589d0 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed80030ad0 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed717fa640 1 -- 172.21.15.31:0/3835902310 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed80049040 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 10 client.17646 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56050551f770 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed717fa640 1 -- 172.21.15.31:0/3835902310 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed80030ad0 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed717fa640 1 -- 172.21.15.31:0/3835902310 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fed80061070 con 0x5605050f0070 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 10 client.17646 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 20 client.17646 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560505083da0 0x560505521c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 4 client.17646 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56050540b850 con 0x560505083da0 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.403+0000 7fed8d39d640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560505083da0 0x560505521c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed8d39d640 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560505083da0 0x560505521c50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499fe86e9c2b0d8f server_cookie=c219ea975e8c6c3c in_seq=0 out_seq=0 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed717fa640 10 client.17646 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed717fa640 1 -- 172.21.15.31:0/3835902310 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x56050540b850 con 0x560505083da0 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed717fa640 10 client.17646 handle_command_reply: tid=0 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 client.17646 shutdown 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 msgr2=0x5605054b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x5605054b6850 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fed8002ae70 tx=0x7fed80009b20 comp rx=0 tx=0).stop 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 shutdown_connections 2024-09-21T03:02:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x560505083da0 0x560505521c50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5605050cf390 0x5605054b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5605050f0070 0x5605054b6850 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 --2- 172.21.15.31:0/3835902310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605050c71b0 0x5605054b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed717fa640 0 client.17646 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 >> 172.21.15.31:0/3835902310 conn(0x5605050e4460 msgr2=0x560505498df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 shutdown_connections 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.404+0000 7fed96376740 1 -- 172.21.15.31:0/3835902310 wait complete. 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed96376740 20 client.17646 trim_cache size 0 max 16384 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed740b6680 msgr2=0x7fed740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed740b6680 0x7fed740b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed840e5300 tx=0x7fed840eb680 comp rx=0 tx=0).stop 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 msgr2=0x7fed8819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 0x7fed8819c6c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fed8400e9f0 tx=0x7fed8400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 shutdown_connections 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fed740b6680 0x7fed740b8b90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed88102f20 0x7fed8819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed88110180 0x7fed881a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 --2- 172.21.15.31:0/1780925902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed881025d0 0x7fed8819c6c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 >> 172.21.15.31:0/1780925902 conn(0x7fed88079730 msgr2=0x7fed88109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 shutdown_connections 2024-09-21T03:02:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:16.405+0000 7fed8fe02640 1 -- 172.21.15.31:0/1780925902 wait complete. 2024-09-21T03:02:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:16 smithi031 ceph-mon[31209]: pgmap v611: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:17.189 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:02:17.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:02:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:16 smithi165 ceph-mon[34818]: pgmap v611: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:16 smithi178 ceph-mon[34759]: pgmap v611: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:17.452 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:17.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.976+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/743127767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18108d20 msgr2=0x7f7c1810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.976+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/743127767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18108d20 0x7f7c1810f550 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009ca0 tx=0x7f7c1002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/743127767 shutdown_connections 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/743127767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c18108d20 0x7f7c1810f550 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/743127767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c181083d0 0x7f7c181087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/743127767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18102ae0 0x7f7c18102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/743127767 >> 172.21.15.31:0/743127767 conn(0x7f7c18079760 msgr2=0x7f7c18079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/743127767 shutdown_connections 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/743127767 wait complete. 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 Processor -- start 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.977+0000 7f7c1f5c9640 1 -- start start 2024-09-21T03:02:17.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1f5c9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c18102ae0 0x7f7c180ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1f5c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 0x7f7c180ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1f5c9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18108d20 0x7f7c181000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1f5c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18112990 con 0x7f7c181083d0 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1f5c9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18112810 con 0x7f7c18102ae0 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1f5c9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18112b10 con 0x7f7c18108d20 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 0x7f7c180ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1d365640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c18102ae0 0x7f7c180ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1d365640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c18102ae0 0x7f7c180ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:45286/0 (socket says 172.21.15.31:45286) 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1d365640 1 -- 172.21.15.31:0/2277417715 learned_addr learned my addr 172.21.15.31:0/2277417715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 0x7f7c180ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60738/0 (socket says 172.21.15.31:60738) 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1db66640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18108d20 0x7f7c181000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 -- 172.21.15.31:0/2277417715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18108d20 msgr2=0x7f7c181000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:17.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18108d20 0x7f7c181000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 -- 172.21.15.31:0/2277417715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c18102ae0 msgr2=0x7f7c180ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c18102ae0 0x7f7c180ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 -- 172.21.15.31:0/2277417715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c181a8600 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.978+0000 7f7c1cb64640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 0x7f7c180ffbb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c940 tx=0x7f7c0800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.979+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08007bb0 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.979+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c08007d50 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.979+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181a88f0 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.979+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08018730 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.979+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c181a8e30 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.980+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c18103d10 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.982+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f7c080188d0 con 0x7f7c181083d0 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.982+0000 7f7c0e7fc640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7c040b6680 0x7f7c040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:17.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.982+0000 7f7c1d365640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7c040b6680 0x7f7c040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:17.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.982+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7f7c080e79e0 con 0x7f7c181083d0 2024-09-21T03:02:17.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.983+0000 7f7c1d365640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7c040b6680 0x7f7c040b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c181010c0 tx=0x7f7c1400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:17.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:17.986+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c08014030 con 0x7f7c181083d0 2024-09-21T03:02:18.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.216+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c18102ef0 con 0x7f7c181083d0 2024-09-21T03:02:18.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.216+0000 7f7c0e7fc640 1 -- 172.21.15.31:0/2277417715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f7c18102ef0 con 0x7f7c181083d0 2024-09-21T03:02:18.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:02:18.219 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:02:18.219 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:02:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7c040b6680 msgr2=0x7f7c040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7c040b6680 0x7f7c040b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7c181010c0 tx=0x7f7c1400b410 comp rx=0 tx=0).stop 2024-09-21T03:02:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 msgr2=0x7f7c180ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 0x7f7c180ffbb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c940 tx=0x7f7c0800ce00 comp rx=0 tx=0).stop 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 shutdown_connections 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f7c040b6680 0x7f7c040b8b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18108d20 0x7f7c181000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c181083d0 0x7f7c180ffbb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 --2- 172.21.15.31:0/2277417715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c18102ae0 0x7f7c180ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 >> 172.21.15.31:0/2277417715 conn(0x7f7c18079760 msgr2=0x7f7c18101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 shutdown_connections 2024-09-21T03:02:18.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:18.219+0000 7f7c1f5c9640 1 -- 172.21.15.31:0/2277417715 wait complete. 2024-09-21T03:02:18.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:02:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:18 smithi031 ceph-mon[31209]: pgmap v612: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:19.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:18 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2277417715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:02:19.122 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:18 smithi165 ceph-mon[34818]: pgmap v612: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:19.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:18 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2277417715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:02:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:18 smithi178 ceph-mon[34759]: pgmap v612: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:19.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:18 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2277417715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:02:19.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1858045383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4981083a0 msgr2=0x7fa4981087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1858045383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4981083a0 0x7fa4981087b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02a410 comp rx=0 tx=0).stop 2024-09-21T03:02:19.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1858045383 shutdown_connections 2024-09-21T03:02:19.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1858045383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa498108cf0 0x7fa49810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1858045383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4981083a0 0x7fa4981087b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1858045383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa498102a60 0x7fa498102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1858045383 >> 172.21.15.31:0/1858045383 conn(0x7fa498079730 msgr2=0x7fa498079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:19.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1858045383 shutdown_connections 2024-09-21T03:02:19.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.623+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1858045383 wait complete. 2024-09-21T03:02:19.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.624+0000 7fa49dfd5640 1 Processor -- start 2024-09-21T03:02:19.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.624+0000 7fa49dfd5640 1 -- start start 2024-09-21T03:02:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa49dfd5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa498102a60 0x7fa49819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa49dfd5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4981083a0 0x7fa49819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa49dfd5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 0x7fa49819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa49dfd5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981129b0 con 0x7fa498108cf0 2024-09-21T03:02:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa49dfd5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa498112830 con 0x7fa4981083a0 2024-09-21T03:02:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa49dfd5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa498112b30 con 0x7fa498102a60 2024-09-21T03:02:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 0x7fa49819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 0x7fa49819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38510/0 (socket says 172.21.15.31:38510) 2024-09-21T03:02:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 -- 172.21.15.31:0/1775767718 learned_addr learned my addr 172.21.15.31:0/1775767718 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:19.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa4977fe640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa498102a60 0x7fa49819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa496ffd640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4981083a0 0x7fa49819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 -- 172.21.15.31:0/1775767718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa498102a60 msgr2=0x7fa49819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa498102a60 0x7fa49819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 -- 172.21.15.31:0/1775767718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4981083a0 msgr2=0x7fa49819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4981083a0 0x7fa49819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.625+0000 7fa497fff640 1 -- 172.21.15.31:0/1775767718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4981a6540 con 0x7fa498108cf0 2024-09-21T03:02:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.626+0000 7fa497fff640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 0x7fa49819bd40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa49000eaa0 tx=0x7fa49000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.626+0000 7fa494ff9640 1 -- 172.21.15.31:0/1775767718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490004d20 con 0x7fa498108cf0 2024-09-21T03:02:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.626+0000 7fa494ff9640 1 -- 172.21.15.31:0/1775767718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa490005020 con 0x7fa498108cf0 2024-09-21T03:02:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.626+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981a6830 con 0x7fa498108cf0 2024-09-21T03:02:19.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.626+0000 7fa494ff9640 1 -- 172.21.15.31:0/1775767718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4900116b0 con 0x7fa498108cf0 2024-09-21T03:02:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.626+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981a6d90 con 0x7fa498108cf0 2024-09-21T03:02:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.628+0000 7fa4a4549740 1 Processor -- start 2024-09-21T03:02:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa494ff9640 1 -- 172.21.15.31:0/1775767718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa4900051d0 con 0x7fa498108cf0 2024-09-21T03:02:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 -- start start 2024-09-21T03:02:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf06d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629bec9d1b0 0x5629bf06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629beca5390 0x5629bf06e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5629befe3ea0 con 0x5629becc6070 2024-09-21T03:02:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5629befd32d0 con 0x5629bec9d1b0 2024-09-21T03:02:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4a4549740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629befd70e0 con 0x5629beca5390 2024-09-21T03:02:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf06d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf06d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38520/0 (socket says 172.21.15.31:38520) 2024-09-21T03:02:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa496ffd640 1 -- 172.21.15.31:0/212238771 learned_addr learned my addr 172.21.15.31:0/212238771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa496ffd640 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629beca5390 0x5629bf06e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.629+0000 7fa4977fe640 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629bec9d1b0 0x5629bf06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa496ffd640 1 -- 172.21.15.31:0/212238771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629beca5390 msgr2=0x5629bf06e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa496ffd640 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629beca5390 0x5629bf06e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa496ffd640 1 -- 172.21.15.31:0/212238771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629bec9d1b0 msgr2=0x5629bf06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa496ffd640 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629bec9d1b0 0x5629bf06dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa494ff9640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4800b6700 0x7fa4800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa496ffd640 1 -- 172.21.15.31:0/212238771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bed31650 con 0x5629becc6070 2024-09-21T03:02:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa497fff640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4800b6700 0x7fa4800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa496ffd640 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf06d6a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa49806aaa0 tx=0x7fa48c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14ef41487bbbe14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa494ff9640 1 -- 172.21.15.31:0/1775767718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7fa4900e77d0 con 0x7fa498108cf0 2024-09-21T03:02:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa47dffb640 1 -- 172.21.15.31:0/212238771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c044020 con 0x5629becc6070 2024-09-21T03:02:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa47dffb640 1 -- 172.21.15.31:0/212238771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48c004750 con 0x5629becc6070 2024-09-21T03:02:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.630+0000 7fa47dffb640 1 -- 172.21.15.31:0/212238771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c035040 con 0x5629becc6070 2024-09-21T03:02:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 -- 172.21.15.31:0/212238771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 msgr2=0x5629bf06d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa497fff640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4800b6700 0x7fa4800b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa4900e4fa0 tx=0x7fa49009e040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf06d6a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa49806aaa0 tx=0x7fa48c000bc0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 -- 172.21.15.31:0/212238771 shutdown_connections 2024-09-21T03:02:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629beca5390 0x5629bf06e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629bec9d1b0 0x5629bf06dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 --2- 172.21.15.31:0/212238771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf06d6a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 -- 172.21.15.31:0/212238771 >> 172.21.15.31:0/212238771 conn(0x5629becba460 msgr2=0x5629bf06b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 -- 172.21.15.31:0/212238771 shutdown_connections 2024-09-21T03:02:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.631+0000 7fa4a4549740 1 -- 172.21.15.31:0/212238771 wait complete. 2024-09-21T03:02:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 -- start start 2024-09-21T03:02:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 Processor -- start 2024-09-21T03:02:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf078db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf0792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf085110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5629bee34700 con 0x5629becc6070 2024-09-21T03:02:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5629bef6e160 con 0x5629beca5390 2024-09-21T03:02:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5629bf07be20 con 0x5629bec9d1b0 2024-09-21T03:02:19.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa4a4549740 10 client.0 _ll_register_callbacks cb 0x5629bef879e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf085110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf085110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38522/0 (socket says 172.21.15.31:38522) 2024-09-21T03:02:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.632+0000 7fa496ffd640 1 -- 172.21.15.31:0/3076318073 learned_addr learned my addr 172.21.15.31:0/3076318073 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf078db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf0792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 -- 172.21.15.31:0/3076318073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 msgr2=0x5629bf078db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf078db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 -- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 msgr2=0x5629bf0792f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf0792f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 -- 172.21.15.31:0/3076318073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629bef9bc90 con 0x5629becc6070 2024-09-21T03:02:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf085110 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa48c031440 tx=0x7fa48c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7078ecdc679cad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:19.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa476ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa476ffd640 1 -- 172.21.15.31:0/3076318073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c035070 con 0x5629becc6070 2024-09-21T03:02:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa476ffd640 1 -- 172.21.15.31:0/3076318073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7fa48c004b00 con 0x5629becc6070 2024-09-21T03:02:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48c004750 con 0x5629becc6070 2024-09-21T03:02:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa476ffd640 1 -- 172.21.15.31:0/3076318073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa48c044070 con 0x5629becc6070 2024-09-21T03:02:19.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa4a4549740 10 client.17664 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.633+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629bf087b30 con 0x5629becc6070 2024-09-21T03:02:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa476ffd640 1 -- 172.21.15.31:0/3076318073 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48c004750 con 0x5629becc6070 2024-09-21T03:02:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa476ffd640 1 -- 172.21.15.31:0/3076318073 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa48c031530 con 0x5629becc6070 2024-09-21T03:02:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4a4549740 10 client.17664 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4a4549740 10 client.17664 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:02:19.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4a4549740 20 client.17664 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:19.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec849f0 0x5629bf08ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4a4549740 4 client.17664 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:19.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629befe5df0 con 0x5629bec849f0 2024-09-21T03:02:19.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.634+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec849f0 0x5629bf08ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.635+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec849f0 0x5629bf08ae90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd39debcefb87ca2 server_cookie=43dd4dc4dcc55482 in_seq=0 out_seq=0 2024-09-21T03:02:19.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.635+0000 7fa476ffd640 10 client.17664 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:02:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.635+0000 7fa476ffd640 1 -- 172.21.15.31:0/3076318073 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5629befe5df0 con 0x5629bec849f0 2024-09-21T03:02:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.635+0000 7fa476ffd640 10 client.17664 handle_command_reply: tid=0 2024-09-21T03:02:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.635+0000 7fa4a4549740 1 client.17664 shutdown 2024-09-21T03:02:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 msgr2=0x5629bf085110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf085110 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa48c031440 tx=0x7fa48c040040 comp rx=0 tx=0).stop 2024-09-21T03:02:19.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 shutdown_connections 2024-09-21T03:02:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf085110 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf0792f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec849f0 0x5629bf08ae90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3076318073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf078db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 >> 172.21.15.31:0/3076318073 conn(0x5629becba460 msgr2=0x5629bf070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa476ffd640 0 client.17664 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:02:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 shutdown_connections 2024-09-21T03:02:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 1 -- 172.21.15.31:0/3076318073 wait complete. 2024-09-21T03:02:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.636+0000 7fa4a4549740 20 client.17664 trim_cache size 0 max 16384 2024-09-21T03:02:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 Processor -- start 2024-09-21T03:02:19.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 -- start start 2024-09-21T03:02:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 0x5629bf085080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf07a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629becc6070 0x5629bf075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981af700 con 0x5629bec9d1b0 2024-09-21T03:02:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b1240 con 0x5629beca5390 2024-09-21T03:02:19.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.649+0000 7fa4a4549740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981afdd0 con 0x5629becc6070 2024-09-21T03:02:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 0x5629bf085080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 0x5629bf085080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38538/0 (socket says 172.21.15.31:38538) 2024-09-21T03:02:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 -- 172.21.15.31:0/359736853 learned_addr learned my addr 172.21.15.31:0/359736853 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629becc6070 0x5629bf075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa4977fe640 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf07a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 -- 172.21.15.31:0/359736853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629becc6070 msgr2=0x5629bf075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629becc6070 0x5629bf075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 -- 172.21.15.31:0/359736853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 msgr2=0x5629bf07a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf07a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 -- 172.21.15.31:0/359736853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bf07a070 con 0x5629bec9d1b0 2024-09-21T03:02:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa496ffd640 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 0x5629bf085080 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0318a0 tx=0x7fa48c032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71141080f4408486 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa47d7fa640 1 -- 172.21.15.31:0/359736853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c035070 con 0x5629bec9d1b0 2024-09-21T03:02:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa47d7fa640 1 -- 172.21.15.31:0/359736853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48c031d10 con 0x5629bec9d1b0 2024-09-21T03:02:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.650+0000 7fa47d7fa640 1 -- 172.21.15.31:0/359736853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0046c0 con 0x5629bec9d1b0 2024-09-21T03:02:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 -- 172.21.15.31:0/359736853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 msgr2=0x5629bf085080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 0x5629bf085080 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0318a0 tx=0x7fa48c032fc0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 -- 172.21.15.31:0/359736853 shutdown_connections 2024-09-21T03:02:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629becc6070 0x5629bf075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf07a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 --2- 172.21.15.31:0/359736853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629bec9d1b0 0x5629bf085080 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 -- 172.21.15.31:0/359736853 >> 172.21.15.31:0/359736853 conn(0x5629becba460 msgr2=0x5629bf07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 -- 172.21.15.31:0/359736853 shutdown_connections 2024-09-21T03:02:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 -- 172.21.15.31:0/359736853 wait complete. 2024-09-21T03:02:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.651+0000 7fa4a4549740 1 -- start start 2024-09-21T03:02:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 Processor -- start 2024-09-21T03:02:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf08bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf08c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf08c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b0b70 con 0x5629becc6070 2024-09-21T03:02:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b04a0 con 0x5629beca5390 2024-09-21T03:02:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981ad700 con 0x5629bec9d1b0 2024-09-21T03:02:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4a4549740 10 client.0 _ll_register_callbacks cb 0x7fa48c004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf08c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf08c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38542/0 (socket says 172.21.15.31:38542) 2024-09-21T03:02:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa496ffd640 1 -- 172.21.15.31:0/3458688408 learned_addr learned my addr 172.21.15.31:0/3458688408 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf08c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.652+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf08bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa496ffd640 1 -- 172.21.15.31:0/3458688408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 msgr2=0x5629bf08bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf08bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa496ffd640 1 -- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 msgr2=0x5629bf08c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf08c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa496ffd640 1 -- 172.21.15.31:0/3458688408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629bf0f4660 con 0x5629becc6070 2024-09-21T03:02:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf08c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa496ffd640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf08c990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0057c0 tx=0x7fa48c031540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adeb47d62bc80572 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa476ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa476ffd640 1 -- 172.21.15.31:0/3458688408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c035040 con 0x5629becc6070 2024-09-21T03:02:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa476ffd640 1 -- 172.21.15.31:0/3458688408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7fa48c004ad0 con 0x5629becc6070 2024-09-21T03:02:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48c031d10 con 0x5629becc6070 2024-09-21T03:02:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa476ffd640 1 -- 172.21.15.31:0/3458688408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa48c004cd0 con 0x5629becc6070 2024-09-21T03:02:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa4a4549740 10 client.17670 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.653+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629bf0f58e0 con 0x5629becc6070 2024-09-21T03:02:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa476ffd640 1 -- 172.21.15.31:0/3458688408 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48c031d10 con 0x5629becc6070 2024-09-21T03:02:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa476ffd640 1 -- 172.21.15.31:0/3458688408 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa48c0624e0 con 0x5629becc6070 2024-09-21T03:02:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4a4549740 10 client.17670 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4a4549740 10 client.17670 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:02:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4a4549740 20 client.17670 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec59da0 0x5629bf0f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4a4549740 4 client.17670 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:02:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629befe18a0 con 0x5629bec59da0 2024-09-21T03:02:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec59da0 0x5629bf0f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa4977fe640 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec59da0 0x5629bf0f7f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f780f61a8940aac server_cookie=6a20d19e78c2686c in_seq=0 out_seq=0 2024-09-21T03:02:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.654+0000 7fa476ffd640 10 client.17670 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:02:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa476ffd640 1 -- 172.21.15.31:0/3458688408 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629befe18a0 con 0x5629bec59da0 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa476ffd640 10 client.17670 handle_command_reply: tid=0 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 client.17670 shutdown 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 msgr2=0x5629bf08c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf08c990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0057c0 tx=0x7fa48c031540 comp rx=0 tx=0).stop 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 shutdown_connections 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5629becc6070 0x5629bf08c990 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5629beca5390 0x5629bf08c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5629bec59da0 0x5629bf0f7f80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 --2- 172.21.15.31:0/3458688408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5629bec9d1b0 0x5629bf08bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 >> 172.21.15.31:0/3458688408 conn(0x5629becba460 msgr2=0x5629bf07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa476ffd640 0 client.17670 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:02:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 shutdown_connections 2024-09-21T03:02:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 1 -- 172.21.15.31:0/3458688408 wait complete. 2024-09-21T03:02:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.655+0000 7fa4a4549740 20 client.17670 trim_cache size 0 max 16384 2024-09-21T03:02:19.686 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4800b6700 msgr2=0x7fa4800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4800b6700 0x7fa4800b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa4900e4fa0 tx=0x7fa49009e040 comp rx=0 tx=0).stop 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 msgr2=0x7fa49819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 0x7fa49819bd40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa49000eaa0 tx=0x7fa49000eea0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 shutdown_connections 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa4800b6700 0x7fa4800b8c10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa498108cf0 0x7fa49819bd40 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4981083a0 0x7fa49819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 --2- 172.21.15.31:0/1775767718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa498102a60 0x7fa49819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 >> 172.21.15.31:0/1775767718 conn(0x7fa498079730 msgr2=0x7fa498104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 shutdown_connections 2024-09-21T03:02:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:19.656+0000 7fa49dfd5640 1 -- 172.21.15.31:0/1775767718 wait complete. 2024-09-21T03:02:20.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:02:20.610 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:21.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:20 smithi031 ceph-mon[31209]: pgmap v613: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 -- 172.21.15.31:0/4095933576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 msgr2=0x7f204010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 --2- 172.21.15.31:0/4095933576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f204010f520 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2030009900 tx=0x7f203002a410 comp rx=0 tx=0).stop 2024-09-21T03:02:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 -- 172.21.15.31:0/4095933576 shutdown_connections 2024-09-21T03:02:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 --2- 172.21.15.31:0/4095933576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20401083a0 0x7f20401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 --2- 172.21.15.31:0/4095933576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f204010f520 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 --2- 172.21.15.31:0/4095933576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2040102a60 0x7f2040102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 -- 172.21.15.31:0/4095933576 >> 172.21.15.31:0/4095933576 conn(0x7f2040079730 msgr2=0x7f2040079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 -- 172.21.15.31:0/4095933576 shutdown_connections 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.115+0000 7f20454c1640 1 -- 172.21.15.31:0/4095933576 wait complete. 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.116+0000 7f20454c1640 1 Processor -- start 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.116+0000 7f20454c1640 1 -- start start 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f20454c1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f20454c1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f20454c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f20454c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112960 con 0x7f2040108cf0 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f20454c1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20401127e0 con 0x7f20401083a0 2024-09-21T03:02:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f20454c1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112ae0 con 0x7f2040102a60 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38548/0 (socket says 172.21.15.31:38548) 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 -- 172.21.15.31:0/2825381435 learned_addr learned my addr 172.21.15.31:0/2825381435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203effd640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 -- 172.21.15.31:0/2825381435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2040102a60 msgr2=0x7f20400ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 -- 172.21.15.31:0/2825381435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20401083a0 msgr2=0x7f20400ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 -- 172.21.15.31:0/2825381435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401a8740 con 0x7f2040108cf0 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.117+0000 7f203f7fe640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2030004770 tx=0x7f2030004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.118+0000 7f2037fff640 1 -- 172.21.15.31:0/2825381435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203003f030 con 0x7f2040108cf0 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.118+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401a89d0 con 0x7f2040108cf0 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.118+0000 7f2037fff640 1 -- 172.21.15.31:0/2825381435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2030003aa0 con 0x7f2040108cf0 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.118+0000 7f2037fff640 1 -- 172.21.15.31:0/2825381435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030035070 con 0x7f2040108cf0 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.118+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401a8f00 con 0x7f2040108cf0 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.121+0000 7f204ba35740 1 Processor -- start 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f2037fff640 1 -- 172.21.15.31:0/2825381435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2030042030 con 0x7f2040108cf0 2024-09-21T03:02:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 -- start start 2024-09-21T03:02:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a45800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a45d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a6349bbea0 con 0x55a6346751b0 2024-09-21T03:02:21.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a6349ab2d0 con 0x55a63467d390 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.122+0000 7f204ba35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6349af0e0 con 0x55a63469e070 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a45800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a45800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38560/0 (socket says 172.21.15.31:38560) 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203effd640 1 -- 172.21.15.31:0/2928522008 learned_addr learned my addr 172.21.15.31:0/2928522008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a45d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f2037fff640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f202c0b6680 0x7f202c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 -- 172.21.15.31:0/2928522008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 msgr2=0x55a634a45d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a45d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 -- 172.21.15.31:0/2928522008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 msgr2=0x55a634a45800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a45800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203e7fc640 1 -- 172.21.15.31:0/2928522008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a634709650 con 0x55a63469e070 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203effd640 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a45800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.123+0000 7f203f7fe640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f202c0b6680 0x7f202c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.124+0000 7f2037fff640 1 -- 172.21.15.31:0/2825381435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7f2030107e90 con 0x7f2040108cf0 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.124+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a452c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2028005fd0 tx=0x7f202800b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f2c27407ae4b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.124+0000 7f203f7fe640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f202c0b6680 0x7f202c0b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2030105660 tx=0x7f20300b5040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.124+0000 7f20357fa640 1 -- 172.21.15.31:0/2928522008 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028005020 con 0x55a63469e070 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.124+0000 7f20357fa640 1 -- 172.21.15.31:0/2928522008 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20280047a0 con 0x55a63469e070 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.124+0000 7f20357fa640 1 -- 172.21.15.31:0/2928522008 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202800f630 con 0x55a63469e070 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.125+0000 7f204ba35740 1 -- 172.21.15.31:0/2928522008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 msgr2=0x55a634a452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.125+0000 7f204ba35740 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a452c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2028005fd0 tx=0x7f202800b6e0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 -- 172.21.15.31:0/2928522008 shutdown_connections 2024-09-21T03:02:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a45d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a45800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 --2- 172.21.15.31:0/2928522008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a452c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 -- 172.21.15.31:0/2928522008 >> 172.21.15.31:0/2928522008 conn(0x55a634692460 msgr2=0x55a634a431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 -- 172.21.15.31:0/2928522008 shutdown_connections 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 -- 172.21.15.31:0/2928522008 wait complete. 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 -- start start 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.126+0000 7f204ba35740 1 Processor -- start 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63467d390 0x55a634a50ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63469e070 0x55a634a5cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a63480c700 con 0x55a6346751b0 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a634946160 con 0x55a63469e070 2024-09-21T03:02:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a634a53aa0 con 0x55a63467d390 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f204ba35740 10 client.0 _ll_register_callbacks cb 0x55a63495f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203effd640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63467d390 0x55a634a50ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38568/0 (socket says 172.21.15.31:38568) 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 -- 172.21.15.31:0/2310704963 learned_addr learned my addr 172.21.15.31:0/2310704963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63469e070 0x55a634a5cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 -- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63467d390 msgr2=0x55a634a50ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63467d390 0x55a634a50ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 -- 172.21.15.31:0/2310704963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63469e070 msgr2=0x55a634a5cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63469e070 0x55a634a5cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.127+0000 7f203e7fc640 1 -- 172.21.15.31:0/2310704963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a634973c90 con 0x55a6346751b0 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f203e7fc640 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a509a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f202800beb0 tx=0x7f202800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64de6298f4e8caf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f20267fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f20267fc640 1 -- 172.21.15.31:0/2310704963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028026070 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f20267fc640 1 -- 172.21.15.31:0/2310704963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7f2028005410 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20280047a0 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f20267fc640 1 -- 172.21.15.31:0/2310704963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2028032050 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f204ba35740 10 client.17682 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.128+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a634a5f7a0 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f20267fc640 1 -- 172.21.15.31:0/2310704963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20280047a0 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f20267fc640 1 -- 172.21.15.31:0/2310704963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f202800f3f0 con 0x55a6346751b0 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f204ba35740 10 client.17682 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f204ba35740 10 client.17682 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f204ba35740 20 client.17682 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f204ba35740 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a63465c9f0 0x55a634a62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f204ba35740 4 client.17682 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6349bddf0 con 0x55a63465c9f0 2024-09-21T03:02:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.129+0000 7f203effd640 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a63465c9f0 0x55a634a62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.130+0000 7f203effd640 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a63465c9f0 0x55a634a62b00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3eefc286e20b507 server_cookie=975972e385a8070d in_seq=0 out_seq=0 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.130+0000 7f20267fc640 10 client.17682 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.130+0000 7f20267fc640 1 -- 172.21.15.31:0/2310704963 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6349bddf0 con 0x55a63465c9f0 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.130+0000 7f20267fc640 10 client.17682 handle_command_reply: tid=0 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.130+0000 7f204ba35740 1 client.17682 shutdown 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 msgr2=0x55a634a509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a509a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f202800beb0 tx=0x7f202800be40 comp rx=0 tx=0).stop 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 shutdown_connections 2024-09-21T03:02:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a63465c9f0 0x55a634a62b00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63467d390 0x55a634a50ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63469e070 0x55a634a5cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 --2- 172.21.15.31:0/2310704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a509a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 >> 172.21.15.31:0/2310704963 conn(0x55a634692460 msgr2=0x55a634a48450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f20267fc640 0 client.17682 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 shutdown_connections 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 1 -- 172.21.15.31:0/2310704963 wait complete. 2024-09-21T03:02:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.131+0000 7f204ba35740 20 client.17682 trim_cache size 0 max 16384 2024-09-21T03:02:21.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.159+0000 7f204ba35740 1 Processor -- start 2024-09-21T03:02:21.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.159+0000 7f204ba35740 1 -- start start 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f204ba35740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a46cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f204ba35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f204ba35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f204ba35740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b19a0 con 0x55a6346751b0 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f204ba35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b34e0 con 0x55a63467d390 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f204ba35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b2070 con 0x55a63469e070 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a46cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a46cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38570/0 (socket says 172.21.15.31:38570) 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.160+0000 7f203e7fc640 1 -- 172.21.15.31:0/3717718327 learned_addr learned my addr 172.21.15.31:0/3717718327 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:21.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203effd640 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203e7fc640 1 -- 172.21.15.31:0/3717718327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 msgr2=0x55a634a42760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203e7fc640 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203e7fc640 1 -- 172.21.15.31:0/3717718327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 msgr2=0x55a634a525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203e7fc640 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203e7fc640 1 -- 172.21.15.31:0/3717718327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63465cc20 con 0x55a6346751b0 2024-09-21T03:02:21.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203effd640 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:21.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f203e7fc640 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a46cb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2028005880 tx=0x7f2028012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da6026185388be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f2034ff9640 1 -- 172.21.15.31:0/3717718327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028026070 con 0x55a6346751b0 2024-09-21T03:02:21.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f2034ff9640 1 -- 172.21.15.31:0/3717718327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2028016050 con 0x55a6346751b0 2024-09-21T03:02:21.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.161+0000 7f2034ff9640 1 -- 172.21.15.31:0/3717718327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202800f650 con 0x55a6346751b0 2024-09-21T03:02:21.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 -- 172.21.15.31:0/3717718327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 msgr2=0x55a634a46cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a46cb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2028005880 tx=0x7f2028012590 comp rx=0 tx=0).stop 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 -- 172.21.15.31:0/3717718327 shutdown_connections 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a42760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a63467d390 0x55a634a525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 --2- 172.21.15.31:0/3717718327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a6346751b0 0x55a634a46cb0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 -- 172.21.15.31:0/3717718327 >> 172.21.15.31:0/3717718327 conn(0x55a634692460 msgr2=0x55a634a51ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 -- 172.21.15.31:0/3717718327 shutdown_connections 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.162+0000 7f204ba35740 1 -- 172.21.15.31:0/3717718327 wait complete. 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 -- start start 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 Processor -- start 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6346751b0 0x55a634a54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 0x55a634a54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a58b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b2e10 con 0x55a63467d390 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b2740 con 0x55a6346751b0 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20401af9a0 con 0x55a63469e070 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f204ba35740 10 client.0 _ll_register_callbacks cb 0x7f2038009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f203effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 0x55a634a54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f203effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 0x55a634a54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38586/0 (socket says 172.21.15.31:38586) 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f203effd640 1 -- 172.21.15.31:0/554850337 learned_addr learned my addr 172.21.15.31:0/554850337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f203e7fc640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6346751b0 0x55a634a54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f203e7fc640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a58b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.163+0000 7f203effd640 1 -- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 msgr2=0x55a634a58b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f203effd640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a58b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f203effd640 1 -- 172.21.15.31:0/554850337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6346751b0 msgr2=0x55a634a54670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f203effd640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6346751b0 0x55a634a54670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f203effd640 1 -- 172.21.15.31:0/554850337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a634a59190 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f203e7fc640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6346751b0 0x55a634a54670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f203effd640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 0x55a634a54c10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2038004a80 tx=0x7f2038012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853ff3ebbe5058cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f20267fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f20267fc640 1 -- 172.21.15.31:0/554850337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038011830 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f20267fc640 1 -- 172.21.15.31:0/554850337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8572+0+0 (secure 0 0 0) 0x7f20380119d0 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2028016050 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f20267fc640 1 -- 172.21.15.31:0/554850337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2038011bd0 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f204ba35740 10 client.17688 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a634accd70 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.164+0000 7f20267fc640 1 -- 172.21.15.31:0/554850337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2028016050 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f20267fc640 1 -- 172.21.15.31:0/554850337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f203800f040 con 0x55a63467d390 2024-09-21T03:02:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f204ba35740 10 client.17688 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f204ba35740 10 client.17688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:02:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f204ba35740 20 client.17688 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f204ba35740 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a634631da0 0x55a634acf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f204ba35740 4 client.17688 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:02:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6349b98a0 con 0x55a634631da0 2024-09-21T03:02:21.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f203e7fc640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a634631da0 0x55a634acf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.165+0000 7f20267fc640 10 client.17688 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f203e7fc640 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a634631da0 0x55a634acf5b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5acca428f72c632 server_cookie=5982de3d578f18b1 in_seq=0 out_seq=0 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f20267fc640 1 -- 172.21.15.31:0/554850337 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6349b98a0 con 0x55a634631da0 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f20267fc640 10 client.17688 handle_command_reply: tid=0 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 client.17688 shutdown 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 msgr2=0x55a634a54c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 0x55a634a54c10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2038004a80 tx=0x7f2038012a40 comp rx=0 tx=0).stop 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 shutdown_connections 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x55a634631da0 0x55a634acf5b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a63469e070 0x55a634a58b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a63467d390 0x55a634a54c10 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 --2- 172.21.15.31:0/554850337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6346751b0 0x55a634a54670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 >> 172.21.15.31:0/554850337 conn(0x55a634692460 msgr2=0x55a634a46a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f20267fc640 0 client.17688 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:02:21.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 shutdown_connections 2024-09-21T03:02:21.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 1 -- 172.21.15.31:0/554850337 wait complete. 2024-09-21T03:02:21.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.166+0000 7f204ba35740 20 client.17688 trim_cache size 0 max 16384 2024-09-21T03:02:21.173 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f202c0b6680 msgr2=0x7f202c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f202c0b6680 0x7f202c0b8b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2030105660 tx=0x7f20300b5040 comp rx=0 tx=0).stop 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 msgr2=0x7f20401000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2030004770 tx=0x7f2030004270 comp rx=0 tx=0).stop 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 shutdown_connections 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f202c0b6680 0x7f202c0b8b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 --2- 172.21.15.31:0/2825381435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 >> 172.21.15.31:0/2825381435 conn(0x7f2040079730 msgr2=0x7f204006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 shutdown_connections 2024-09-21T03:02:21.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:21.167+0000 7f20454c1640 1 -- 172.21.15.31:0/2825381435 wait complete. 2024-09-21T03:02:21.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:20 smithi165 ceph-mon[34818]: pgmap v613: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:21.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:20 smithi178 ceph-mon[34759]: pgmap v613: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T03:02:21.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:02:22.085 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 -- 172.21.15.31:0/1690526120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb338108d10 msgr2=0x7fb33810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 --2- 172.21.15.31:0/1690526120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb338108d10 0x7fb33810f540 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb330009ca0 tx=0x7fb33002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 -- 172.21.15.31:0/1690526120 shutdown_connections 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 --2- 172.21.15.31:0/1690526120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb338108d10 0x7fb33810f540 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 --2- 172.21.15.31:0/1690526120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3381083c0 0x7fb3381087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 --2- 172.21.15.31:0/1690526120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb338102a80 0x7fb338102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 -- 172.21.15.31:0/1690526120 >> 172.21.15.31:0/1690526120 conn(0x7fb338079740 msgr2=0x7fb338079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:22.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 -- 172.21.15.31:0/1690526120 shutdown_connections 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.582+0000 7fb3402f1640 1 -- 172.21.15.31:0/1690526120 wait complete. 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 Processor -- start 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 -- start start 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb338102a80 0x7fb33819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 0x7fb33819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb338108d10 0x7fb33819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb338113aa0 con 0x7fb3381083c0 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb338113920 con 0x7fb338108d10 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb3402f1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb338113c20 con 0x7fb338102a80 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb33d88c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 0x7fb33819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb33d88c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 0x7fb33819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38612/0 (socket says 172.21.15.31:38612) 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb33d88c640 1 -- 172.21.15.31:0/6968362 learned_addr learned my addr 172.21.15.31:0/6968362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:22.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb33e08d640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb338102a80 0x7fb33819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.583+0000 7fb33e88e640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb338108d10 0x7fb33819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33d88c640 1 -- 172.21.15.31:0/6968362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb338102a80 msgr2=0x7fb33819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33d88c640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb338102a80 0x7fb33819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33d88c640 1 -- 172.21.15.31:0/6968362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb338108d10 msgr2=0x7fb33819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33d88c640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb338108d10 0x7fb33819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33d88c640 1 -- 172.21.15.31:0/6968362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3381a0420 con 0x7fb3381083c0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33e88e640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb338108d10 0x7fb33819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb33d88c640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 0x7fb33819aa50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb32800d770 tx=0x7fb32800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb32f7fe640 1 -- 172.21.15.31:0/6968362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328017070 con 0x7fb3381083c0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3381a0710 con 0x7fb3381083c0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb32f7fe640 1 -- 172.21.15.31:0/6968362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb328009c30 con 0x7fb3381083c0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb32f7fe640 1 -- 172.21.15.31:0/6968362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32800f620 con 0x7fb3381083c0 2024-09-21T03:02:22.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.584+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381a6e70 con 0x7fb3381083c0 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.587+0000 7fb346865740 1 Processor -- start 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 -- start start 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb32f7fe640 1 -- 172.21.15.31:0/6968362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fb32801a070 con 0x7fb3381083c0 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ada6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670adb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562670a50ea0 con 0x56267070a1b0 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562670a402d0 con 0x562670733070 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb346865740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562670a440e0 con 0x562670712390 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38624/0 (socket says 172.21.15.31:38624) 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 -- 172.21.15.31:0/3690126604 learned_addr learned my addr 172.21.15.31:0/3690126604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:22.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33d88c640 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670adb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e88e640 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ada6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 -- 172.21.15.31:0/3690126604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 msgr2=0x562670adb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670adb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 -- 172.21.15.31:0/3690126604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 msgr2=0x562670ada6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ada6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.588+0000 7fb33e08d640 1 -- 172.21.15.31:0/3690126604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267079e650 con 0x56267070a1b0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb33e08d640 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adabe0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b760 tx=0x7fb33400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb27ee1e089fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb32cd9a640 1 -- 172.21.15.31:0/3690126604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334003dd0 con 0x56267070a1b0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb32cd9a640 1 -- 172.21.15.31:0/3690126604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb334015070 con 0x56267070a1b0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb32cd9a640 1 -- 172.21.15.31:0/3690126604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334004da0 con 0x56267070a1b0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb32f7fe640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb3240b6680 0x7fb3240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb33e88e640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb3240b6680 0x7fb3240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.589+0000 7fb32f7fe640 1 -- 172.21.15.31:0/6968362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fb3280e7200 con 0x7fb3381083c0 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 -- 172.21.15.31:0/3690126604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 msgr2=0x562670adabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb33e88e640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb3240b6680 0x7fb3240b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3300094e0 tx=0x7fb3300093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adabe0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b760 tx=0x7fb33400bc20 comp rx=0 tx=0).stop 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 -- 172.21.15.31:0/3690126604 shutdown_connections 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670adb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adabe0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 --2- 172.21.15.31:0/3690126604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ada6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 -- 172.21.15.31:0/3690126604 >> 172.21.15.31:0/3690126604 conn(0x562670727460 msgr2=0x562670ad8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 -- 172.21.15.31:0/3690126604 shutdown_connections 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.590+0000 7fb346865740 1 -- 172.21.15.31:0/3690126604 wait complete. 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 -- start start 2024-09-21T03:02:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 Processor -- start 2024-09-21T03:02:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56267070a1b0 0x562670ae5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 0x562670ae6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5626708a1700 con 0x562670733070 2024-09-21T03:02:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5626709db160 con 0x56267070a1b0 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562670ae8e80 con 0x562670712390 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb346865740 10 client.0 _ll_register_callbacks cb 0x5626709f49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb33e88e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 0x562670ae6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb33e88e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 0x562670ae6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38630/0 (socket says 172.21.15.31:38630) 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb33e88e640 1 -- 172.21.15.31:0/3148637598 learned_addr learned my addr 172.21.15.31:0/3148637598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb33e08d640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56267070a1b0 0x562670ae5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.591+0000 7fb33d88c640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb33e88e640 1 -- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 msgr2=0x562670ae6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb33e88e640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb33e88e640 1 -- 172.21.15.31:0/3148637598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56267070a1b0 msgr2=0x562670ae5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb33e88e640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56267070a1b0 0x562670ae5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb33e88e640 1 -- 172.21.15.31:0/3148637598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562670a08c90 con 0x562670733070 2024-09-21T03:02:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb33e88e640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 0x562670ae6800 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb3300316d0 tx=0x7fb3300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07827557edee16b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb321ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb321ffb640 1 -- 172.21.15.31:0/3148637598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33004f070 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb334015070 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb321ffb640 1 -- 172.21.15.31:0/3148637598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fb330004240 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb321ffb640 1 -- 172.21.15.31:0/3148637598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb330004440 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb346865740 10 client.17700 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.592+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562670af4b50 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb321ffb640 1 -- 172.21.15.31:0/3148637598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb334015070 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb321ffb640 1 -- 172.21.15.31:0/3148637598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb330044040 con 0x562670733070 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb346865740 10 client.17700 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb346865740 10 client.17700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb346865740 20 client.17700 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb346865740 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706f19f0 0x562670af8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb346865740 4 client.17700 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562670a52df0 con 0x5626706f19f0 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.593+0000 7fb33e08d640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706f19f0 0x562670af8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.594+0000 7fb321ffb640 10 client.17700 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.594+0000 7fb33e08d640 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706f19f0 0x562670af8110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abc6b72a572e6f2d server_cookie=afbbfa01fd480c07 in_seq=0 out_seq=0 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.594+0000 7fb321ffb640 1 -- 172.21.15.31:0/3148637598 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562670a52df0 con 0x5626706f19f0 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb321ffb640 10 client.17700 handle_command_reply: tid=0 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 client.17700 shutdown 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 msgr2=0x562670ae6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 0x562670ae6800 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb3300316d0 tx=0x7fb3300095c0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 shutdown_connections 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706f19f0 0x562670af8110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb321ffb640 0 client.17700 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:02:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562670733070 0x562670ae6800 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 --2- 172.21.15.31:0/3148637598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56267070a1b0 0x562670ae5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 >> 172.21.15.31:0/3148637598 conn(0x562670727460 msgr2=0x562670add830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 shutdown_connections 2024-09-21T03:02:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 1 -- 172.21.15.31:0/3148637598 wait complete. 2024-09-21T03:02:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.595+0000 7fb346865740 20 client.17700 trim_cache size 0 max 16384 2024-09-21T03:02:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.608+0000 7fb346865740 1 Processor -- start 2024-09-21T03:02:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.608+0000 7fb346865740 1 -- start start 2024-09-21T03:02:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.608+0000 7fb346865740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.608+0000 7fb346865740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb346865740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ae3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb346865740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381af7b0 con 0x56267070a1b0 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb346865740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381b12f0 con 0x562670733070 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb346865740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381afe80 con 0x562670712390 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38638/0 (socket says 172.21.15.31:38638) 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 -- 172.21.15.31:0/335291534 learned_addr learned my addr 172.21.15.31:0/335291534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 -- 172.21.15.31:0/335291534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 msgr2=0x562670ae2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 -- 172.21.15.31:0/335291534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 msgr2=0x562670ae3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ae3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 -- 172.21.15.31:0/335291534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562670ae7000 con 0x56267070a1b0 2024-09-21T03:02:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb33e08d640 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adbde0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb334005fe0 tx=0x7fb3340049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f0604e70c5bf4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:22.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb323fff640 1 -- 172.21.15.31:0/335291534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334003dd0 con 0x56267070a1b0 2024-09-21T03:02:22.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb323fff640 1 -- 172.21.15.31:0/335291534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb334015040 con 0x56267070a1b0 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.609+0000 7fb323fff640 1 -- 172.21.15.31:0/335291534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334019680 con 0x56267070a1b0 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 -- 172.21.15.31:0/335291534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 msgr2=0x562670adbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adbde0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb334005fe0 tx=0x7fb3340049e0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 -- 172.21.15.31:0/335291534 shutdown_connections 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670712390 0x562670ae2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670733070 0x562670ae3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 --2- 172.21.15.31:0/335291534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670adbde0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 -- 172.21.15.31:0/335291534 >> 172.21.15.31:0/335291534 conn(0x562670727460 msgr2=0x562670adb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:22.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 -- 172.21.15.31:0/335291534 shutdown_connections 2024-09-21T03:02:22.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.610+0000 7fb346865740 1 -- 172.21.15.31:0/335291534 wait complete. 2024-09-21T03:02:22.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 -- start start 2024-09-21T03:02:22.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 Processor -- start 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670aea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670712390 0x562670aeaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670733070 0x562670b61b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381b0c20 con 0x56267070a1b0 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381b0550 con 0x562670712390 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381ad790 con 0x562670733070 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb346865740 10 client.0 _ll_register_callbacks cb 0x7fb330009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb33e08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670aea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb33e08d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670aea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38650/0 (socket says 172.21.15.31:38650) 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.611+0000 7fb33e08d640 1 -- 172.21.15.31:0/2814322365 learned_addr learned my addr 172.21.15.31:0/2814322365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e88e640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670733070 0x562670b61b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33d88c640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670712390 0x562670aeaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e08d640 1 -- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670733070 msgr2=0x562670b61b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e08d640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670733070 0x562670b61b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e08d640 1 -- 172.21.15.31:0/2814322365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670712390 msgr2=0x562670aeaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e08d640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670712390 0x562670aeaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e08d640 1 -- 172.21.15.31:0/2814322365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562670b62050 con 0x56267070a1b0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33d88c640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670712390 0x562670aeaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb33e08d640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670aea510 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb33400bc60 tx=0x7fb334012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca8e85857bb3161 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb321ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb321ffb640 1 -- 172.21.15.31:0/2814322365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334032040 con 0x56267070a1b0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb321ffb640 1 -- 172.21.15.31:0/2814322365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fb334011440 con 0x56267070a1b0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb334015040 con 0x56267070a1b0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb321ffb640 1 -- 172.21.15.31:0/2814322365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb334044050 con 0x56267070a1b0 2024-09-21T03:02:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb346865740 10 client.17706 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.612+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562670b631d0 con 0x56267070a1b0 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb321ffb640 1 -- 172.21.15.31:0/2814322365 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb334015040 con 0x56267070a1b0 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb321ffb640 1 -- 172.21.15.31:0/2814322365 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb334035070 con 0x56267070a1b0 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb346865740 10 client.17706 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb346865740 10 client.17706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb346865740 20 client.17706 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb346865740 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706c6da0 0x562670b65780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb346865740 4 client.17706 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562670a4e8a0 con 0x5626706c6da0 2024-09-21T03:02:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.613+0000 7fb33d88c640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706c6da0 0x562670b65780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb33d88c640 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706c6da0 0x562670b65780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a6497e39acef73a server_cookie=6df55fc358f0720c in_seq=0 out_seq=0 2024-09-21T03:02:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb321ffb640 10 client.17706 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:02:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb321ffb640 1 -- 172.21.15.31:0/2814322365 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562670a4e8a0 con 0x5626706c6da0 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb321ffb640 10 client.17706 handle_command_reply: tid=0 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 client.17706 shutdown 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 msgr2=0x562670aea510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670aea510 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb33400bc60 tx=0x7fb334012ea0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 shutdown_connections 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x5626706c6da0 0x562670b65780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562670733070 0x562670b61b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562670712390 0x562670aeaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 --2- 172.21.15.31:0/2814322365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56267070a1b0 0x562670aea510 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb321ffb640 0 client.17706 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 >> 172.21.15.31:0/2814322365 conn(0x562670727460 msgr2=0x562670ae71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 shutdown_connections 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 1 -- 172.21.15.31:0/2814322365 wait complete. 2024-09-21T03:02:22.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.614+0000 7fb346865740 20 client.17706 trim_cache size 0 max 16384 2024-09-21T03:02:22.623 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:02:22.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb3240b6680 msgr2=0x7fb3240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb3240b6680 0x7fb3240b8b90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3300094e0 tx=0x7fb3300093f0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 msgr2=0x7fb33819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 0x7fb33819aa50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb32800d770 tx=0x7fb32800dc30 comp rx=0 tx=0).stop 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 shutdown_connections 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fb3240b6680 0x7fb3240b8b90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb338108d10 0x7fb33819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3381083c0 0x7fb33819aa50 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 --2- 172.21.15.31:0/6968362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb338102a80 0x7fb33819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 >> 172.21.15.31:0/6968362 conn(0x7fb338079740 msgr2=0x7fb338104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 shutdown_connections 2024-09-21T03:02:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:22.615+0000 7fb3402f1640 1 -- 172.21.15.31:0/6968362 wait complete. 2024-09-21T03:02:22.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:22 smithi031 ceph-mon[31209]: pgmap v614: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 193 B/s wr, 10 op/s 2024-09-21T03:02:22.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:22 smithi031 ceph-mon[31209]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:02:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:22 smithi178 ceph-mon[34759]: pgmap v614: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 193 B/s wr, 10 op/s 2024-09-21T03:02:22.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:22 smithi178 ceph-mon[34759]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:02:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:22 smithi165 ceph-mon[34818]: pgmap v614: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 193 B/s wr, 10 op/s 2024-09-21T03:02:23.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:22 smithi165 ceph-mon[34818]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:02:23.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:02:23.559 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 -- 172.21.15.31:0/1810967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd390102f30 msgr2=0x7fd39010fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 --2- 172.21.15.31:0/1810967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd390102f30 0x7fd39010fc50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd380009900 tx=0x7fd38002a480 comp rx=0 tx=0).stop 2024-09-21T03:02:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 -- 172.21.15.31:0/1810967363 shutdown_connections 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 --2- 172.21.15.31:0/1810967363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 0x7fd3901125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 --2- 172.21.15.31:0/1810967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd390102f30 0x7fd39010fc50 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 --2- 172.21.15.31:0/1810967363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3901025e0 0x7fd3901029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 -- 172.21.15.31:0/1810967363 >> 172.21.15.31:0/1810967363 conn(0x7fd390079710 msgr2=0x7fd390079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 -- 172.21.15.31:0/1810967363 shutdown_connections 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.086+0000 7fd397fc4640 1 -- 172.21.15.31:0/1810967363 wait complete. 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 Processor -- start 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 -- start start 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 0x7fd39019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd390102f30 0x7fd39019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 0x7fd3901a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd390115c80 con 0x7fd3901025e0 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd390115b00 con 0x7fd390102f30 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd397fc4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd390115e00 con 0x7fd390110190 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 0x7fd39019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 0x7fd39019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38666/0 (socket says 172.21.15.31:38666) 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 -- 172.21.15.31:0/581265333 learned_addr learned my addr 172.21.15.31:0/581265333 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd396561640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 0x7fd3901a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 -- 172.21.15.31:0/581265333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 msgr2=0x7fd3901a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 0x7fd3901a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 -- 172.21.15.31:0/581265333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd390102f30 msgr2=0x7fd39019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd390102f30 0x7fd39019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd395d60640 1 -- 172.21.15.31:0/581265333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3901a25c0 con 0x7fd3901025e0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.087+0000 7fd396561640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 0x7fd3901a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.088+0000 7fd395d60640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 0x7fd39019c650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00d950 tx=0x7fd38c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.088+0000 7fd386ffd640 1 -- 172.21.15.31:0/581265333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c017030 con 0x7fd3901025e0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.088+0000 7fd386ffd640 1 -- 172.21.15.31:0/581265333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38c004020 con 0x7fd3901025e0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.088+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3901a28b0 con 0x7fd3901025e0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.088+0000 7fd386ffd640 1 -- 172.21.15.31:0/581265333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c0047a0 con 0x7fd3901025e0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.088+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3901a9010 con 0x7fd3901025e0 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.091+0000 7fd39e538740 1 Processor -- start 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.091+0000 7fd39e538740 1 -- start start 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39e538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 0x55bfe0f0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39e538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39e538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39e538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0e83ea0 con 0x55bfe0b66070 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39e538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0e732d0 con 0x55bfe0b45390 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39e538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0e770e0 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd386ffd640 1 -- 172.21.15.31:0/581265333 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd38c009a30 con 0x7fd3901025e0 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 0x55bfe0f0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 0x55bfe0f0d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38676/0 (socket says 172.21.15.31:38676) 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 -- 172.21.15.31:0/164424018 learned_addr learned my addr 172.21.15.31:0/164424018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd39555f640 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd395d60640 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 -- 172.21.15.31:0/164424018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b3d1b0 msgr2=0x55bfe0f0dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 -- 172.21.15.31:0/164424018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 msgr2=0x55bfe0f0e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f0e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.092+0000 7fd396561640 1 -- 172.21.15.31:0/164424018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe0bd1650 con 0x55bfe0b66070 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd396561640 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 0x55bfe0f0d6a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd388008d50 tx=0x7fd38800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6b1d1ac6dd02ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd37bfff640 1 -- 172.21.15.31:0/164424018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800f830 con 0x55bfe0b66070 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd37bfff640 1 -- 172.21.15.31:0/164424018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38800de20 con 0x55bfe0b66070 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd37bfff640 1 -- 172.21.15.31:0/164424018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388018d70 con 0x55bfe0b66070 2024-09-21T03:02:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd386ffd640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd37c0b6700 0x7fd37c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd395d60640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd37c0b6700 0x7fd37c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.093+0000 7fd386ffd640 1 -- 172.21.15.31:0/581265333 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fd38c0e7270 con 0x7fd3901025e0 2024-09-21T03:02:24.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 -- 172.21.15.31:0/164424018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 msgr2=0x55bfe0f0d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 0x55bfe0f0d6a0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd388008d50 tx=0x7fd38800d710 comp rx=0 tx=0).stop 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd395d60640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd37c0b6700 0x7fd37c0b8c10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55bfe0f0c320 tx=0x7fd38c0e74f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 -- 172.21.15.31:0/164424018 shutdown_connections 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f0e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 --2- 172.21.15.31:0/164424018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b66070 0x55bfe0f0d6a0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 -- 172.21.15.31:0/164424018 >> 172.21.15.31:0/164424018 conn(0x55bfe0b5a460 msgr2=0x55bfe0f0b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 -- 172.21.15.31:0/164424018 shutdown_connections 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 -- 172.21.15.31:0/164424018 wait complete. 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 -- start start 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.094+0000 7fd39e538740 1 Processor -- start 2024-09-21T03:02:24.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f18dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0cd4700 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0e0e160 con 0x55bfe0b45390 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0f1be80 con 0x55bfe0b66070 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39e538740 10 client.0 _ll_register_callbacks cb 0x55bfe0e279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f18dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f18dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38688/0 (socket says 172.21.15.31:38688) 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd396561640 1 -- 172.21.15.31:0/2737231528 learned_addr learned my addr 172.21.15.31:0/2737231528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd396561640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.095+0000 7fd39555f640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd396561640 1 -- 172.21.15.31:0/2737231528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 msgr2=0x55bfe0f251a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd396561640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f251a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd396561640 1 -- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 msgr2=0x55bfe0f19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd396561640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f19300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd396561640 1 -- 172.21.15.31:0/2737231528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe0e3bc90 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd39555f640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd396561640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f18dc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd388018d10 tx=0x7fd388018970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea2c55f06100ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd3797fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd3797fa640 1 -- 172.21.15.31:0/2737231528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38802d060 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38800de20 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd3797fa640 1 -- 172.21.15.31:0/2737231528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fd38800f840 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd3797fa640 1 -- 172.21.15.31:0/2737231528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd38800fa40 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd39e538740 10 client.17718 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfe0f27bc0 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.096+0000 7fd3797fa640 1 -- 172.21.15.31:0/2737231528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38800de20 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd3797fa640 1 -- 172.21.15.31:0/2737231528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd388028bc0 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39e538740 10 client.17718 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39e538740 10 client.17718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39e538740 20 client.17718 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39e538740 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0b249f0 0x55bfe0f2af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39e538740 4 client.17718 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe0e85df0 con 0x55bfe0b249f0 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.097+0000 7fd39555f640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0b249f0 0x55bfe0f2af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.098+0000 7fd39555f640 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0b249f0 0x55bfe0f2af80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68d887389bcab39f server_cookie=6a514f37d70c437d in_seq=0 out_seq=0 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.098+0000 7fd3797fa640 10 client.17718 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:02:24.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd3797fa640 1 -- 172.21.15.31:0/2737231528 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bfe0e85df0 con 0x55bfe0b249f0 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd3797fa640 10 client.17718 handle_command_reply: tid=0 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 client.17718 shutdown 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 msgr2=0x55bfe0f18dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f18dc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd388018d10 tx=0x7fd388018970 comp rx=0 tx=0).stop 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 shutdown_connections 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0b249f0 0x55bfe0f2af80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f251a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd3797fa640 0 client.17718 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 --2- 172.21.15.31:0/2737231528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f18dc0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 >> 172.21.15.31:0/2737231528 conn(0x55bfe0b5a460 msgr2=0x55bfe0f10880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 shutdown_connections 2024-09-21T03:02:24.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 1 -- 172.21.15.31:0/2737231528 wait complete. 2024-09-21T03:02:24.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.099+0000 7fd39e538740 20 client.17718 trim_cache size 0 max 16384 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 Processor -- start 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 -- start start 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b45390 0x55bfe0f25110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b66070 0x55bfe0f15fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b19a0 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b34e0 con 0x55bfe0b66070 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.112+0000 7fd39e538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b2070 con 0x55bfe0b45390 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38696/0 (socket says 172.21.15.31:38696) 2024-09-21T03:02:24.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 -- 172.21.15.31:0/3437434267 learned_addr learned my addr 172.21.15.31:0/3437434267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 -- 172.21.15.31:0/3437434267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b45390 msgr2=0x55bfe0f25110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b45390 0x55bfe0f25110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 -- 172.21.15.31:0/3437434267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b66070 msgr2=0x55bfe0f15fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b66070 0x55bfe0f15fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 -- 172.21.15.31:0/3437434267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe0f0be80 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd396561640 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0b5e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd388000f80 tx=0x7fd388018a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d80a1d1ba114b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd37b7fe640 1 -- 172.21.15.31:0/3437434267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38802d070 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd37b7fe640 1 -- 172.21.15.31:0/3437434267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38800dc10 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.113+0000 7fd37b7fe640 1 -- 172.21.15.31:0/3437434267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800fbe0 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 -- 172.21.15.31:0/3437434267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 msgr2=0x55bfe0f0b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0b5e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd388000f80 tx=0x7fd388018a00 comp rx=0 tx=0).stop 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 -- 172.21.15.31:0/3437434267 shutdown_connections 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b45390 0x55bfe0f25110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b66070 0x55bfe0f15fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 --2- 172.21.15.31:0/3437434267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f0b5e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 -- 172.21.15.31:0/3437434267 >> 172.21.15.31:0/3437434267 conn(0x55bfe0b5a460 msgr2=0x55bfe0f0aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 -- 172.21.15.31:0/3437434267 shutdown_connections 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 -- 172.21.15.31:0/3437434267 wait complete. 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 -- start start 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.114+0000 7fd39e538740 1 Processor -- start 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f2e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f2bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f2c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b2e10 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b2740 con 0x55bfe0b45390 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901af980 con 0x55bfe0b66070 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39e538740 10 client.0 _ll_register_callbacks cb 0x7fd388008bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f2e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f2e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38710/0 (socket says 172.21.15.31:38710) 2024-09-21T03:02:24.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 -- 172.21.15.31:0/1088136500 learned_addr learned my addr 172.21.15.31:0/1088136500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39555f640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f2bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 -- 172.21.15.31:0/1088136500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 msgr2=0x55bfe0f2c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f2c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 -- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 msgr2=0x55bfe0f2bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f2bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd396561640 1 -- 172.21.15.31:0/1088136500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe0f2cb00 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.115+0000 7fd39555f640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f2bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd396561640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f2e870 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd3880083c0 tx=0x7fd388008520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215b7f4976d14a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd3797fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd3797fa640 1 -- 172.21.15.31:0/1088136500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388042070 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd3797fa640 1 -- 172.21.15.31:0/1088136500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fd388018650 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38800dc10 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd3797fa640 1 -- 172.21.15.31:0/1088136500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd38802d050 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd39e538740 10 client.17724 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bfe0f95710 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd3797fa640 1 -- 172.21.15.31:0/1088136500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38800dc10 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.116+0000 7fd3797fa640 1 -- 172.21.15.31:0/1088136500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd38803a070 con 0x55bfe0b3d1b0 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39e538740 10 client.17724 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39e538740 10 client.17724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39e538740 20 client.17724 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39e538740 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0af9da0 0x55bfe0f97d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39e538740 4 client.17724 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:02:24.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfe0e818a0 con 0x55bfe0af9da0 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39555f640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0af9da0 0x55bfe0f97d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd39555f640 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0af9da0 0x55bfe0f97d50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34cc1d94362443e2 server_cookie=beb239dfd3252b8c in_seq=0 out_seq=0 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.117+0000 7fd3797fa640 10 client.17724 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd3797fa640 1 -- 172.21.15.31:0/1088136500 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfe0e818a0 con 0x55bfe0af9da0 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd3797fa640 10 client.17724 handle_command_reply: tid=0 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 client.17724 shutdown 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 msgr2=0x55bfe0f2e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f2e870 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd3880083c0 tx=0x7fd388008520 comp rx=0 tx=0).stop 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 shutdown_connections 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x55bfe0af9da0 0x55bfe0f97d50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bfe0b66070 0x55bfe0f2c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfe0b45390 0x55bfe0f2bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 --2- 172.21.15.31:0/1088136500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfe0b3d1b0 0x55bfe0f2e870 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd3797fa640 0 client.17724 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:02:24.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 >> 172.21.15.31:0/1088136500 conn(0x55bfe0b5a460 msgr2=0x55bfe0f0e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 shutdown_connections 2024-09-21T03:02:24.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 1 -- 172.21.15.31:0/1088136500 wait complete. 2024-09-21T03:02:24.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.118+0000 7fd39e538740 20 client.17724 trim_cache size 0 max 16384 2024-09-21T03:02:24.125 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd37c0b6700 msgr2=0x7fd37c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd37c0b6700 0x7fd37c0b8c10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55bfe0f0c320 tx=0x7fd38c0e74f0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 msgr2=0x7fd39019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 0x7fd39019c650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00d950 tx=0x7fd38c00de10 comp rx=0 tx=0).stop 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 shutdown_connections 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd37c0b6700 0x7fd37c0b8c10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd390110190 0x7fd3901a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd390102f30 0x7fd39019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 --2- 172.21.15.31:0/581265333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3901025e0 0x7fd39019c650 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 >> 172.21.15.31:0/581265333 conn(0x7fd390079710 msgr2=0x7fd390109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 shutdown_connections 2024-09-21T03:02:24.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:24.119+0000 7fd397fc4640 1 -- 172.21.15.31:0/581265333 wait complete. 2024-09-21T03:02:24.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:02:24.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:24 smithi178 ceph-mon[34759]: pgmap v616: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:02:25.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:24 smithi031 ceph-mon[31209]: pgmap v616: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:02:25.103 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:25.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:24 smithi165 ceph-mon[34818]: pgmap v616: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:02:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/3140614765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4102a60 msgr2=0x7fcaf4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/3140614765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4102a60 0x7fcaf4102e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009900 tx=0x7fcae402a410 comp rx=0 tx=0).stop 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/3140614765 shutdown_connections 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/3140614765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf41083a0 0x7fcaf41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/3140614765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf4108cf0 0x7fcaf410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/3140614765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4102a60 0x7fcaf4102e70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/3140614765 >> 172.21.15.31:0/3140614765 conn(0x7fcaf4079730 msgr2=0x7fcaf4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/3140614765 shutdown_connections 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.605+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/3140614765 wait complete. 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 Processor -- start 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 -- start start 2024-09-21T03:02:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf4102a60 0x7fcaf419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf41083a0 0x7fcaf419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 0x7fcaf419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41129b0 con 0x7fcaf4108cf0 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4112830 con 0x7fcaf41083a0 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf9fb5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4112b30 con 0x7fcaf4102a60 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 0x7fcaf419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 0x7fcaf419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38724/0 (socket says 172.21.15.31:38724) 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf3fff640 1 -- 172.21.15.31:0/4052501191 learned_addr learned my addr 172.21.15.31:0/4052501191 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf41083a0 0x7fcaf419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.606+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf4102a60 0x7fcaf419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf3fff640 1 -- 172.21.15.31:0/4052501191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf4102a60 msgr2=0x7fcaf419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf3fff640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf4102a60 0x7fcaf419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf3fff640 1 -- 172.21.15.31:0/4052501191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf41083a0 msgr2=0x7fcaf419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf3fff640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf41083a0 0x7fcaf419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf3fff640 1 -- 172.21.15.31:0/4052501191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf41a03e0 con 0x7fcaf4108cf0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf41083a0 0x7fcaf419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf3fff640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 0x7fcaf419fcb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00cad0 tx=0x7fcaec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf0ff9640 1 -- 172.21.15.31:0/4052501191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00ae20 con 0x7fcaf4108cf0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf41a06d0 con 0x7fcaf4108cf0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf0ff9640 1 -- 172.21.15.31:0/4052501191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec005020 con 0x7fcaf4108cf0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf0ff9640 1 -- 172.21.15.31:0/4052501191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00f6b0 con 0x7fcaf4108cf0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.607+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41a6e30 con 0x7fcaf4108cf0 2024-09-21T03:02:25.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.610+0000 7fcb00529740 1 Processor -- start 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.610+0000 7fcaf0ff9640 1 -- 172.21.15.31:0/4052501191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcaec004b20 con 0x7fcaf4108cf0 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.610+0000 7fcb00529740 1 -- start start 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcb00529740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcb00529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e16800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcb00529740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e16d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcb00529740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5604f3d8cea0 con 0x5604f3a6f070 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcb00529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604f3d7c2d0 con 0x5604f3a461b0 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcb00529740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604f3d800e0 con 0x5604f3a4e390 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e16800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38728/0 (socket says 172.21.15.31:38728) 2024-09-21T03:02:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/3533428137 learned_addr learned my addr 172.21.15.31:0/3533428137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e16d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf0ff9640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6680 0x7fcadc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/3533428137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 msgr2=0x5604f3e16d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e16d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/3533428137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 msgr2=0x5604f3e16800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e16800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/3533428137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f3ada650 con 0x5604f3a6f070 2024-09-21T03:02:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e16800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.611+0000 7fcaf3fff640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6680 0x7fcadc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.612+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e162c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcae800cfa0 tx=0x7fcae800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b757fbb905649b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.612+0000 7fcaf0ff9640 1 -- 172.21.15.31:0/4052501191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fcaec0e7810 con 0x7fcaf4108cf0 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.612+0000 7fcad9ffb640 1 -- 172.21.15.31:0/3533428137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8012110 con 0x5604f3a6f070 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.612+0000 7fcad9ffb640 1 -- 172.21.15.31:0/3533428137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae80054d0 con 0x5604f3a6f070 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.612+0000 7fcad9ffb640 1 -- 172.21.15.31:0/3533428137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e620 con 0x5604f3a6f070 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 -- 172.21.15.31:0/3533428137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 msgr2=0x5604f3e162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e162c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcae800cfa0 tx=0x7fcae800c8b0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcaf3fff640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6680 0x7fcadc0b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ad5f0 tx=0x7fcaec0a9580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 -- 172.21.15.31:0/3533428137 shutdown_connections 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e16d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e16800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 --2- 172.21.15.31:0/3533428137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e162c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 -- 172.21.15.31:0/3533428137 >> 172.21.15.31:0/3533428137 conn(0x5604f3a63460 msgr2=0x5604f3e141b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 -- 172.21.15.31:0/3533428137 shutdown_connections 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 -- 172.21.15.31:0/3533428137 wait complete. 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.613+0000 7fcb00529740 1 -- start start 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 Processor -- start 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e219c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e21f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5604f3bdd700 con 0x5604f3a6f070 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604f3d17160 con 0x5604f3a461b0 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604f3e24aa0 con 0x5604f3a4e390 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcb00529740 10 client.0 _ll_register_callbacks cb 0x5604f3d309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e2dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38734/0 (socket says 172.21.15.31:38734) 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/1859803370 learned_addr learned my addr 172.21.15.31:0/1859803370 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e21f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/1859803370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 msgr2=0x5604f3e21f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e21f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 msgr2=0x5604f3e219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e219c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.614+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/1859803370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f3d44c90 con 0x5604f3a6f070 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e2dd90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c3e0 tx=0x7fcae8004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a5add4d3d89774 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcad2ffd640 1 -- 172.21.15.31:0/1859803370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80046c0 con 0x5604f3a6f070 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae80054d0 con 0x5604f3a6f070 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcad2ffd640 1 -- 172.21.15.31:0/1859803370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fcae8012110 con 0x5604f3a6f070 2024-09-21T03:02:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcad2ffd640 1 -- 172.21.15.31:0/1859803370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcae8004020 con 0x5604f3a6f070 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcb00529740 10 client.17736 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604f3e30810 con 0x5604f3a6f070 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.615+0000 7fcad2ffd640 1 -- 172.21.15.31:0/1859803370 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae80054d0 con 0x5604f3a6f070 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcad2ffd640 1 -- 172.21.15.31:0/1859803370 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcae80042a0 con 0x5604f3a6f070 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcb00529740 10 client.17736 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcb00529740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcb00529740 20 client.17736 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcb00529740 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a2d9f0 0x5604f3e33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcb00529740 4 client.17736 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f3d8edf0 con 0x5604f3a2d9f0 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a2d9f0 0x5604f3e33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcad2ffd640 10 client.17736 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.616+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a2d9f0 0x5604f3e33b70 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1850e9076ea28d43 server_cookie=79c6a557da099898 in_seq=0 out_seq=0 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcad2ffd640 1 -- 172.21.15.31:0/1859803370 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604f3d8edf0 con 0x5604f3a2d9f0 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcad2ffd640 10 client.17736 handle_command_reply: tid=0 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 client.17736 shutdown 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 msgr2=0x5604f3e2dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e2dd90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c3e0 tx=0x7fcae8004bb0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 shutdown_connections 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a2d9f0 0x5604f3e33b70 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a4e390 0x5604f3e21f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcad2ffd640 0 client.17736 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e2dd90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 --2- 172.21.15.31:0/1859803370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a461b0 0x5604f3e219c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.617+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 >> 172.21.15.31:0/1859803370 conn(0x5604f3a63460 msgr2=0x5604f3e194a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.618+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 shutdown_connections 2024-09-21T03:02:25.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.618+0000 7fcb00529740 1 -- 172.21.15.31:0/1859803370 wait complete. 2024-09-21T03:02:25.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.618+0000 7fcb00529740 20 client.17736 trim_cache size 0 max 16384 2024-09-21T03:02:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 Processor -- start 2024-09-21T03:02:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 -- start start 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a461b0 0x5604f3e238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41af7f0 con 0x5604f3a6f070 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41b1330 con 0x5604f3a4e390 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcb00529740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41afec0 con 0x5604f3a461b0 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e174d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38748/0 (socket says 172.21.15.31:38748) 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.631+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/2307743190 learned_addr learned my addr 172.21.15.31:0/2307743190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:25.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a461b0 0x5604f3e238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/2307743190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a461b0 msgr2=0x5604f3e238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a461b0 0x5604f3e238a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/2307743190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 msgr2=0x5604f3e2dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e2dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/2307743190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f3e18a10 con 0x5604f3a6f070 2024-09-21T03:02:25.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:25.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e174d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcae8004ff0 tx=0x7fcae8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3da52477539c8f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:25.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcad97fa640 1 -- 172.21.15.31:0/2307743190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae800ed70 con 0x5604f3a6f070 2024-09-21T03:02:25.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcad97fa640 1 -- 172.21.15.31:0/2307743190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8023030 con 0x5604f3a6f070 2024-09-21T03:02:25.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.632+0000 7fcad97fa640 1 -- 172.21.15.31:0/2307743190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8003ca0 con 0x5604f3a6f070 2024-09-21T03:02:25.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 -- 172.21.15.31:0/2307743190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 msgr2=0x5604f3e174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e174d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcae8004ff0 tx=0x7fcae8021040 comp rx=0 tx=0).stop 2024-09-21T03:02:25.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 -- 172.21.15.31:0/2307743190 shutdown_connections 2024-09-21T03:02:25.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a6f070 0x5604f3e174d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e2dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 --2- 172.21.15.31:0/2307743190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a461b0 0x5604f3e238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 -- 172.21.15.31:0/2307743190 >> 172.21.15.31:0/2307743190 conn(0x5604f3a63460 msgr2=0x5604f3e16dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 -- 172.21.15.31:0/2307743190 shutdown_connections 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.633+0000 7fcb00529740 1 -- 172.21.15.31:0/2307743190 wait complete. 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 -- start start 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 Processor -- start 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 0x5604f3e2a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e27880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a6f070 0x5604f3e27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41b0c60 con 0x5604f3a461b0 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41b0590 con 0x5604f3a4e390 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41ad7d0 con 0x5604f3a6f070 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcb00529740 10 client.0 _ll_register_callbacks cb 0x7fcae4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 0x5604f3e2a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 0x5604f3e2a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38758/0 (socket says 172.21.15.31:38758) 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/669338953 learned_addr learned my addr 172.21.15.31:0/669338953 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.634+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e27880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a6f070 0x5604f3e27dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/669338953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a6f070 msgr2=0x5604f3e27dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a6f070 0x5604f3e27dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 msgr2=0x5604f3e27880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e27880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 -- 172.21.15.31:0/669338953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f3e0f5d0 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcaf2ffd640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 0x5604f3e2a3b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcae8003780 tx=0x7fcae8005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d38b1479557055 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcad2ffd640 1 -- 172.21.15.31:0/669338953 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8003a10 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae8023030 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcad2ffd640 1 -- 172.21.15.31:0/669338953 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fcae80041c0 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcad2ffd640 1 -- 172.21.15.31:0/669338953 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcae80043c0 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcb00529740 10 client.17742 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.635+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604f3e9df20 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcad2ffd640 1 -- 172.21.15.31:0/669338953 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8023030 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcad2ffd640 1 -- 172.21.15.31:0/669338953 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcae800e900 con 0x5604f3a461b0 2024-09-21T03:02:25.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcb00529740 10 client.17742 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcb00529740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcb00529740 20 client.17742 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcb00529740 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a02da0 0x5604f3ea0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcb00529740 4 client.17742 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604f3d8a8a0 con 0x5604f3a02da0 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a02da0 0x5604f3ea0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:25.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcad2ffd640 10 client.17742 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.636+0000 7fcaf37fe640 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a02da0 0x5604f3ea0540 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203e7b104b0966ff server_cookie=70303229979b77d1 in_seq=0 out_seq=0 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcad2ffd640 1 -- 172.21.15.31:0/669338953 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604f3d8a8a0 con 0x5604f3a02da0 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcad2ffd640 10 client.17742 handle_command_reply: tid=0 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 client.17742 shutdown 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 msgr2=0x5604f3e2a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 0x5604f3e2a3b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcae8003780 tx=0x7fcae8005d00 comp rx=0 tx=0).stop 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 shutdown_connections 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x5604f3a02da0 0x5604f3ea0540 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f3a6f070 0x5604f3e27dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f3a4e390 0x5604f3e27880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 --2- 172.21.15.31:0/669338953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5604f3a461b0 0x5604f3e2a3b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 >> 172.21.15.31:0/669338953 conn(0x5604f3a63460 msgr2=0x5604f3e23600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:25.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcad2ffd640 0 client.17742 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:25.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 shutdown_connections 2024-09-21T03:02:25.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 1 -- 172.21.15.31:0/669338953 wait complete. 2024-09-21T03:02:25.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.637+0000 7fcb00529740 20 client.17742 trim_cache size 0 max 16384 2024-09-21T03:02:25.646 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6680 msgr2=0x7fcadc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6680 0x7fcadc0b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ad5f0 tx=0x7fcaec0a9580 comp rx=0 tx=0).stop 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 msgr2=0x7fcaf419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 0x7fcaf419fcb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00cad0 tx=0x7fcaec00cf90 comp rx=0 tx=0).stop 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 shutdown_connections 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcadc0b6680 0x7fcadc0b8b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcaf4108cf0 0x7fcaf419fcb0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaf41083a0 0x7fcaf419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 --2- 172.21.15.31:0/4052501191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf4102a60 0x7fcaf419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 >> 172.21.15.31:0/4052501191 conn(0x7fcaf4079730 msgr2=0x7fcaf4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 shutdown_connections 2024-09-21T03:02:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:25.638+0000 7fcaf9fb5640 1 -- 172.21.15.31:0/4052501191 wait complete. 2024-09-21T03:02:26.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:26 smithi178 ceph-mon[34759]: pgmap v617: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:02:27.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:26 smithi031 ceph-mon[31209]: pgmap v617: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:02:27.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:26 smithi165 ceph-mon[34818]: pgmap v617: 97 pgs: 97 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:02:27.315 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:02:27.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:02:27.587 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 -- 172.21.15.31:0/2832720598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8102770 msgr2=0x7efdc81105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 --2- 172.21.15.31:0/2832720598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8102770 0x7efdc81105a0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009930 tx=0x7efdc002a420 comp rx=0 tx=0).stop 2024-09-21T03:02:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 -- 172.21.15.31:0/2832720598 shutdown_connections 2024-09-21T03:02:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 --2- 172.21.15.31:0/2832720598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8102770 0x7efdc81105a0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 --2- 172.21.15.31:0/2832720598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8101e20 0x7efdc8102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 --2- 172.21.15.31:0/2832720598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 0x7efdc8108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 -- 172.21.15.31:0/2832720598 >> 172.21.15.31:0/2832720598 conn(0x7efdc8077cc0 msgr2=0x7efdc8078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 -- 172.21.15.31:0/2832720598 shutdown_connections 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.108+0000 7efdcf316640 1 -- 172.21.15.31:0/2832720598 wait complete. 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 Processor -- start 2024-09-21T03:02:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 -- start start 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 0x7efdc819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8102770 0x7efdc819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 0x7efdc819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8113ab0 con 0x7efdc8101e20 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8113930 con 0x7efdc8102770 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.109+0000 7efdcf316640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8113c30 con 0x7efdc8108730 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 0x7efdc819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 0x7efdc819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38784/0 (socket says 172.21.15.31:38784) 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3878822260 learned_addr learned my addr 172.21.15.31:0/3878822260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8102770 0x7efdc819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3878822260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 msgr2=0x7efdc819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 0x7efdc819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 0x7efdc819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3878822260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8102770 msgr2=0x7efdc819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8102770 0x7efdc819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3878822260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc81a0410 con 0x7efdc8101e20 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8102770 0x7efdc819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 0x7efdc819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 0x7efdc819a4a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7efdc400ea80 tx=0x7efdc400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdbe7fc640 1 -- 172.21.15.31:0/3878822260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc400ce40 con 0x7efdc8101e20 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdbe7fc640 1 -- 172.21.15.31:0/3878822260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4005020 con 0x7efdc8101e20 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc81a0700 con 0x7efdc8101e20 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.110+0000 7efdbe7fc640 1 -- 172.21.15.31:0/3878822260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40116d0 con 0x7efdc8101e20 2024-09-21T03:02:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.111+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc81a6e60 con 0x7efdc8101e20 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.113+0000 7efdd588a740 1 Processor -- start 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.113+0000 7efdbe7fc640 1 -- 172.21.15.31:0/3878822260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7efdc40054d0 con 0x7efdc8101e20 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 -- start start 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612502430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612502970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612502eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b612478e60 con 0x55b612132490 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b612468290 con 0x55b61213a670 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdd588a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b61246c0a0 con 0x55b61215b350 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612502970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612502970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38800/0 (socket says 172.21.15.31:38800) 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3007029587 learned_addr learned my addr 172.21.15.31:0/3007029587 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612502430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.114+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612502eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3007029587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 msgr2=0x55b612502430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612502430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3007029587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 msgr2=0x55b612502eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612502eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd0b2640 1 -- 172.21.15.31:0/3007029587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6121c6930 con 0x55b612132490 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612502430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:28.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612502eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:28.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdbe7fc640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efdb40b6680 0x7efdb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efdb40b6680 0x7efdb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdbe7fc640 1 -- 172.21.15.31:0/3878822260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7efdc40e7a20 con 0x7efdc8101e20 2024-09-21T03:02:28.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612502970 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7efdc40e50b0 tx=0x7efdc40f0520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefce92c0070b1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:28.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdb37fe640 1 -- 172.21.15.31:0/3007029587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc401f070 con 0x55b612132490 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdb37fe640 1 -- 172.21.15.31:0/3007029587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc40e7e50 con 0x55b612132490 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.115+0000 7efdb37fe640 1 -- 172.21.15.31:0/3007029587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40ad640 con 0x55b612132490 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 -- 172.21.15.31:0/3007029587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 msgr2=0x55b612502970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efdb40b6680 0x7efdb40b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efdc0005ab0 tx=0x7efdc0005ee0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612502970 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7efdc40e50b0 tx=0x7efdc40f0520 comp rx=0 tx=0).stop 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 -- 172.21.15.31:0/3007029587 shutdown_connections 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612502eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612502970 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 --2- 172.21.15.31:0/3007029587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612502430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 -- 172.21.15.31:0/3007029587 >> 172.21.15.31:0/3007029587 conn(0x55b61214f740 msgr2=0x55b612500320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 -- 172.21.15.31:0/3007029587 shutdown_connections 2024-09-21T03:02:28.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.116+0000 7efdd588a740 1 -- 172.21.15.31:0/3007029587 wait complete. 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 -- start start 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 Processor -- start 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b61250dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b61250e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612519f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b6122c9890 con 0x55b612132490 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b612402d40 con 0x55b61213a670 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b612510c10 con 0x55b61215b350 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdd588a740 10 client.0 _ll_register_callbacks cb 0x55b612466c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b61250dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.117+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b61250dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38802/0 (socket says 172.21.15.31:38802) 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 -- 172.21.15.31:0/814066979 learned_addr learned my addr 172.21.15.31:0/814066979 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b61250e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612519f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 -- 172.21.15.31:0/814066979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 msgr2=0x55b612519f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612519f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 -- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 msgr2=0x55b61250e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b61250e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 -- 172.21.15.31:0/814066979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61251a4d0 con 0x55b612132490 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b61250dff0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7efdc40ad800 tx=0x7efdc40eb0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a774ff3b4a1084 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdb0ff9640 1 -- 172.21.15.31:0/814066979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc401f070 con 0x55b612132490 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdb0ff9640 1 -- 172.21.15.31:0/814066979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7efdc40f0c30 con 0x55b612132490 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc40e7e50 con 0x55b612132490 2024-09-21T03:02:28.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdb0ff9640 1 -- 172.21.15.31:0/814066979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdc40acfe0 con 0x55b612132490 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdd588a740 10 client.17754 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.118+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b61251ce00 con 0x55b612132490 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdb0ff9640 1 -- 172.21.15.31:0/814066979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc40e7e50 con 0x55b612132490 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdb0ff9640 1 -- 172.21.15.31:0/814066979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efdc400f040 con 0x55b612132490 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdd588a740 10 client.17754 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdd588a740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdd588a740 20 client.17754 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdd588a740 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b612119cd0 0x55b612520160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdd588a740 4 client.17754 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b61247adb0 con 0x55b612119cd0 2024-09-21T03:02:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.119+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b612119cd0 0x55b612520160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.120+0000 7efdb0ff9640 10 client.17754 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.120+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b612119cd0 0x55b612520160 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a8c0db33791d54 server_cookie=2cc93952ea80bc3b in_seq=0 out_seq=0 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdb0ff9640 1 -- 172.21.15.31:0/814066979 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b61247adb0 con 0x55b612119cd0 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdb0ff9640 10 client.17754 handle_command_reply: tid=0 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 client.17754 shutdown 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 msgr2=0x55b61250dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b61250dff0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7efdc40ad800 tx=0x7efdc40eb0a0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 shutdown_connections 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b612119cd0 0x55b612520160 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b612519f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b61250e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 --2- 172.21.15.31:0/814066979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b61250dff0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdb0ff9640 0 client.17754 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 >> 172.21.15.31:0/814066979 conn(0x55b61214f740 msgr2=0x55b6125055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 shutdown_connections 2024-09-21T03:02:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 1 -- 172.21.15.31:0/814066979 wait complete. 2024-09-21T03:02:28.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.121+0000 7efdd588a740 20 client.17754 trim_cache size 0 max 16384 2024-09-21T03:02:28.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 Processor -- start 2024-09-21T03:02:28.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 -- start start 2024-09-21T03:02:28.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b612132490 0x55b612519ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 0x55b612508550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b61250afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81af7a0 con 0x55b61213a670 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81b12e0 con 0x55b612132490 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdd588a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81afe70 con 0x55b61215b350 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdcc8b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 0x55b612508550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdcc8b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 0x55b612508550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38812/0 (socket says 172.21.15.31:38812) 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.139+0000 7efdcc8b1640 1 -- 172.21.15.31:0/1086308703 learned_addr learned my addr 172.21.15.31:0/1086308703 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b612132490 0x55b612519ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b61250afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcc8b1640 1 -- 172.21.15.31:0/1086308703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 msgr2=0x55b61250afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b61250afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcc8b1640 1 -- 172.21.15.31:0/1086308703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b612132490 msgr2=0x55b612519ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b612132490 0x55b612519ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcc8b1640 1 -- 172.21.15.31:0/1086308703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6125107a0 con 0x55b61213a670 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b61250afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 0x55b612508550 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005fd0 tx=0x7efdb80094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1849761b9cac9103 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdb2ffd640 1 -- 172.21.15.31:0/1086308703 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8023600 con 0x55b61213a670 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdb2ffd640 1 -- 172.21.15.31:0/1086308703 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb8023c50 con 0x55b61213a670 2024-09-21T03:02:28.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.140+0000 7efdb2ffd640 1 -- 172.21.15.31:0/1086308703 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8011cd0 con 0x55b61213a670 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 -- 172.21.15.31:0/1086308703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 msgr2=0x55b612508550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 0x55b612508550 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005fd0 tx=0x7efdb80094b0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 -- 172.21.15.31:0/1086308703 shutdown_connections 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b61250afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b61213a670 0x55b612508550 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 --2- 172.21.15.31:0/1086308703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b612132490 0x55b612519ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 -- 172.21.15.31:0/1086308703 >> 172.21.15.31:0/1086308703 conn(0x55b61214f740 msgr2=0x55b61250c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 -- 172.21.15.31:0/1086308703 shutdown_connections 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 -- 172.21.15.31:0/1086308703 wait complete. 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 -- start start 2024-09-21T03:02:28.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.141+0000 7efdd588a740 1 Processor -- start 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612521a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612521fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b6125898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81b0c10 con 0x55b612132490 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81b0540 con 0x55b61213a670 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81ad780 con 0x55b61215b350 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdd588a740 10 client.0 _ll_register_callbacks cb 0x7efdc40ebac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612521a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612521a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38826/0 (socket says 172.21.15.31:38826) 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 -- 172.21.15.31:0/476580555 learned_addr learned my addr 172.21.15.31:0/476580555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd8b3640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b6125898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612521fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 -- 172.21.15.31:0/476580555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 msgr2=0x55b6125898e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b6125898e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 -- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 msgr2=0x55b612521fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612521fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.142+0000 7efdcd0b2640 1 -- 172.21.15.31:0/476580555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b612589f20 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdcd0b2640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612521a10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efdc40ad380 tx=0x7efdc40e5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bd654e9f99dddf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdb0ff9640 1 -- 172.21.15.31:0/476580555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc401f070 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdb0ff9640 1 -- 172.21.15.31:0/476580555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7efdc40acad0 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb8023c50 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdb0ff9640 1 -- 172.21.15.31:0/476580555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efdc4098070 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdd588a740 10 client.17760 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b61258b150 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdb0ff9640 1 -- 172.21.15.31:0/476580555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb8023c50 con 0x55b612132490 2024-09-21T03:02:28.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.143+0000 7efdb0ff9640 1 -- 172.21.15.31:0/476580555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efdc40aeba0 con 0x55b612132490 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdd588a740 10 client.17760 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdd588a740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdd588a740 20 client.17760 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdd588a740 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b6120ef080 0x55b61258d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdd588a740 4 client.17760 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b612476860 con 0x55b6120ef080 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b6120ef080 0x55b61258d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdcc8b1640 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b6120ef080 0x55b61258d7a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31fde0225d15280d server_cookie=c5f93757415edfb5 in_seq=0 out_seq=0 2024-09-21T03:02:28.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.144+0000 7efdb0ff9640 10 client.17760 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:28.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdb0ff9640 1 -- 172.21.15.31:0/476580555 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b612476860 con 0x55b6120ef080 2024-09-21T03:02:28.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdb0ff9640 10 client.17760 handle_command_reply: tid=0 2024-09-21T03:02:28.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 client.17760 shutdown 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 msgr2=0x55b612521a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612521a10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efdc40ad380 tx=0x7efdc40e5700 comp rx=0 tx=0).stop 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 shutdown_connections 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55b6120ef080 0x55b61258d7a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b61215b350 0x55b6125898e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdd588a740 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b61213a670 0x55b612521fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.146+0000 7efdb0ff9640 0 client.17760 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdd588a740 1 --2- 172.21.15.31:0/476580555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b612132490 0x55b612521a10 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 >> 172.21.15.31:0/476580555 conn(0x55b61214f740 msgr2=0x55b6125033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 shutdown_connections 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdd588a740 1 -- 172.21.15.31:0/476580555 wait complete. 2024-09-21T03:02:28.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdd588a740 20 client.17760 trim_cache size 0 max 16384 2024-09-21T03:02:28.152 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:02:28.152 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T03:02:28.152 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "2043732d-b892-4c41-ac8d-903a913c8697", 2024-09-21T03:02:28.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T03:02:28.152 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efdb40b6680 msgr2=0x7efdb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efdb40b6680 0x7efdb40b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efdc0005ab0 tx=0x7efdc0005ee0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 msgr2=0x7efdc819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 0x7efdc819a4a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7efdc400ea80 tx=0x7efdc400ef40 comp rx=0 tx=0).stop 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 shutdown_connections 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7efdb40b6680 0x7efdb40b8b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdc8108730 0x7efdc819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdc8102770 0x7efdc819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 --2- 172.21.15.31:0/3878822260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdc8101e20 0x7efdc819a4a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.147+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 >> 172.21.15.31:0/3878822260 conn(0x7efdc8077cc0 msgr2=0x7efdc8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:28.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.148+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 shutdown_connections 2024-09-21T03:02:28.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:28.148+0000 7efdcf316640 1 -- 172.21.15.31:0/3878822260 wait complete. 2024-09-21T03:02:28.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:02:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:28 smithi178 ceph-mon[34759]: pgmap v618: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:28.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:28 smithi178 ceph-mon[34759]: scrub queued for path: / 2024-09-21T03:02:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:28 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:28 smithi178 ceph-mon[34759]: scrub summary: active paths [/] 2024-09-21T03:02:28.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:28 smithi178 ceph-mon[34759]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:28.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:28 smithi165 ceph-mon[34818]: pgmap v618: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:28.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:28 smithi165 ceph-mon[34818]: scrub queued for path: / 2024-09-21T03:02:28.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:28 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:28.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:28 smithi165 ceph-mon[34818]: scrub summary: active paths [/] 2024-09-21T03:02:28.994 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:28 smithi165 ceph-mon[34818]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:29.054 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:28 smithi031 ceph-mon[31209]: pgmap v618: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:29.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:28 smithi031 ceph-mon[31209]: scrub queued for path: / 2024-09-21T03:02:29.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:28 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:29.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:28 smithi031 ceph-mon[31209]: scrub summary: active paths [/] 2024-09-21T03:02:29.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:28 smithi031 ceph-mon[31209]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 -- 172.21.15.31:0/2684769355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 msgr2=0x7f518410fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 --2- 172.21.15.31:0/2684769355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518410fc60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5178009900 tx=0x7f517802a410 comp rx=0 tx=0).stop 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 -- 172.21.15.31:0/2684769355 shutdown_connections 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 --2- 172.21.15.31:0/2684769355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51841101a0 0x7f51841125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 --2- 172.21.15.31:0/2684769355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518410fc60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 --2- 172.21.15.31:0/2684769355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5184075730 0x7f5184075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 -- 172.21.15.31:0/2684769355 >> 172.21.15.31:0/2684769355 conn(0x7f518407b910 msgr2=0x7f518407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 -- 172.21.15.31:0/2684769355 shutdown_connections 2024-09-21T03:02:29.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.531+0000 7f518b6ec640 1 -- 172.21.15.31:0/2684769355 wait complete. 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.532+0000 7f518b6ec640 1 Processor -- start 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.532+0000 7f518b6ec640 1 -- start start 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5184075730 0x7f518419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51841101a0 0x7f51841a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5184115c90 con 0x7f5184076080 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5184115b10 con 0x7f5184075730 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5184115e10 con 0x7f51841101a0 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49362/0 (socket says 172.21.15.31:49362) 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 -- 172.21.15.31:0/3842395028 learned_addr learned my addr 172.21.15.31:0/3842395028 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:29.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5189488640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5184075730 0x7f518419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5189c89640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51841101a0 0x7f51841a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 -- 172.21.15.31:0/3842395028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51841101a0 msgr2=0x7f51841a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51841101a0 0x7f51841a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 -- 172.21.15.31:0/3842395028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5184075730 msgr2=0x7f518419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5184075730 0x7f518419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 -- 172.21.15.31:0/3842395028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51841a2590 con 0x7f5184076080 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f5188c87640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518419cc80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f517802a930 tx=0x7f5178004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f51827fc640 1 -- 172.21.15.31:0/3842395028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517803f030 con 0x7f5184076080 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f51827fc640 1 -- 172.21.15.31:0/3842395028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5178003aa0 con 0x7f5184076080 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.533+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51841a2820 con 0x7f5184076080 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.534+0000 7f51827fc640 1 -- 172.21.15.31:0/3842395028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178035070 con 0x7f5184076080 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.534+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51841a2d00 con 0x7f5184076080 2024-09-21T03:02:29.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.536+0000 7f5191c60740 1 Processor -- start 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.536+0000 7f5191c60740 1 -- start start 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.536+0000 7f51827fc640 1 -- 172.21.15.31:0/3842395028 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5178042030 con 0x7f5184076080 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.536+0000 7f5191c60740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35d7070 0x55bed397e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5191c60740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35ae1b0 0x55bed397eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5191c60740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed397f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5191c60740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bed38f4e50 con 0x55bed35b6390 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5191c60740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bed38e4280 con 0x55bed35d7070 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5191c60740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bed38e8090 con 0x55bed35ae1b0 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed397f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed397f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49366/0 (socket says 172.21.15.31:49366) 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 -- 172.21.15.31:0/2899158404 learned_addr learned my addr 172.21.15.31:0/2899158404 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5189c89640 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35d7070 0x55bed397e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5189488640 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35ae1b0 0x55bed397eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 -- 172.21.15.31:0/2899158404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35ae1b0 msgr2=0x55bed397eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35ae1b0 0x55bed397eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 -- 172.21.15.31:0/2899158404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35d7070 msgr2=0x55bed397e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35d7070 0x55bed397e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 -- 172.21.15.31:0/2899158404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed3642650 con 0x55bed35b6390 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5189c89640 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35d7070 0x55bed397e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f51827fc640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f516c0b66b0 0x7f516c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.537+0000 7f5188c87640 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed397f0d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f51781050a0 tx=0x7f517810bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55caa1ef7d7511d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.538+0000 7f5189c89640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f516c0b66b0 0x7f516c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.538+0000 7f516b7fe640 1 -- 172.21.15.31:0/2899158404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780c9f60 con 0x55bed35b6390 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.538+0000 7f51827fc640 1 -- 172.21.15.31:0/3842395028 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f5178107a10 con 0x7f5184076080 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.538+0000 7f516b7fe640 1 -- 172.21.15.31:0/2899158404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51780cd420 con 0x55bed35b6390 2024-09-21T03:02:29.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.538+0000 7f516b7fe640 1 -- 172.21.15.31:0/2899158404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780ce420 con 0x55bed35b6390 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 -- 172.21.15.31:0/2899158404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 msgr2=0x55bed397f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5189c89640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f516c0b66b0 0x7f516c0b8bc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5174010620 tx=0x7f5174007520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed397f0d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f51781050a0 tx=0x7f517810bc20 comp rx=0 tx=0).stop 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 -- 172.21.15.31:0/2899158404 shutdown_connections 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35ae1b0 0x55bed397eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed397f0d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 --2- 172.21.15.31:0/2899158404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35d7070 0x55bed397e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 -- 172.21.15.31:0/2899158404 >> 172.21.15.31:0/2899158404 conn(0x55bed35cb460 msgr2=0x55bed397c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 -- 172.21.15.31:0/2899158404 shutdown_connections 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 -- 172.21.15.31:0/2899158404 wait complete. 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 -- start start 2024-09-21T03:02:29.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.539+0000 7f5191c60740 1 Processor -- start 2024-09-21T03:02:29.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 0x55bed3989cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35b6390 0x55bed398a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3995e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bed3745700 con 0x55bed35ae1b0 2024-09-21T03:02:29.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bed387f110 con 0x55bed35b6390 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bed398cbf0 con 0x55bed35d7070 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5191c60740 10 client.0 _ll_register_callbacks cb 0x55bed3898990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5189488640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 0x55bed3989cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5189488640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 0x55bed3989cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49380/0 (socket says 172.21.15.31:49380) 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5189488640 1 -- 172.21.15.31:0/3039175499 learned_addr learned my addr 172.21.15.31:0/3039175499 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5188c87640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35b6390 0x55bed398a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.540+0000 7f5189c89640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3995e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5189488640 1 -- 172.21.15.31:0/3039175499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 msgr2=0x55bed3995e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5189488640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3995e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5189488640 1 -- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35b6390 msgr2=0x55bed398a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5189488640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35b6390 0x55bed398a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5189488640 1 -- 172.21.15.31:0/3039175499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed38acc40 con 0x55bed35ae1b0 2024-09-21T03:02:29.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5189488640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 0x55bed3989cd0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f517c00b920 tx=0x7f517c00bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11308713cd269b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5168ff9640 1 -- 172.21.15.31:0/3039175499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c01b440 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5168ff9640 1 -- 172.21.15.31:0/3039175499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f517c01b5e0 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51780cd420 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5168ff9640 1 -- 172.21.15.31:0/3039175499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f517c01b820 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5191c60740 10 client.17772 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bed3998890 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.541+0000 7f5168ff9640 1 -- 172.21.15.31:0/3039175499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51780cd420 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5168ff9640 1 -- 172.21.15.31:0/3039175499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f517c019020 con 0x55bed35ae1b0 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5191c60740 10 client.17772 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5191c60740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5191c60740 20 client.17772 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5191c60740 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed35959f0 0x55bed399bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5191c60740 4 client.17772 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bed38f6da0 con 0x55bed35959f0 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.542+0000 7f5188c87640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed35959f0 0x55bed399bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.543+0000 7f5168ff9640 10 client.17772 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.543+0000 7f5188c87640 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed35959f0 0x55bed399bd90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88aeda74fc575c91 server_cookie=21362b5b51d8beab in_seq=0 out_seq=0 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.543+0000 7f5168ff9640 1 -- 172.21.15.31:0/3039175499 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bed38f6da0 con 0x55bed35959f0 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.543+0000 7f5168ff9640 10 client.17772 handle_command_reply: tid=0 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.543+0000 7f5191c60740 1 client.17772 shutdown 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 msgr2=0x55bed3989cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 0x55bed3989cd0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f517c00b920 tx=0x7f517c00bd20 comp rx=0 tx=0).stop 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 shutdown_connections 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed35959f0 0x55bed399bd90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3995e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35b6390 0x55bed398a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5168ff9640 0 client.17772 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 --2- 172.21.15.31:0/3039175499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35ae1b0 0x55bed3989cd0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 >> 172.21.15.31:0/3039175499 conn(0x55bed35cb460 msgr2=0x55bed39817e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 shutdown_connections 2024-09-21T03:02:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 1 -- 172.21.15.31:0/3039175499 wait complete. 2024-09-21T03:02:29.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.544+0000 7f5191c60740 20 client.17772 trim_cache size 0 max 16384 2024-09-21T03:02:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.561+0000 7f5191c60740 1 Processor -- start 2024-09-21T03:02:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.561+0000 7f5191c60740 1 -- start start 2024-09-21T03:02:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5191c60740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5191c60740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35b6390 0x55bed3995cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5191c60740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 0x55bed3986ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5191c60740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b1910 con 0x55bed35d7070 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5191c60740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b3450 con 0x55bed35ae1b0 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5191c60740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b1fe0 con 0x55bed35b6390 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 0x55bed3986ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 0x55bed3986ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49396/0 (socket says 172.21.15.31:49396) 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 -- 172.21.15.31:0/31523235 learned_addr learned my addr 172.21.15.31:0/31523235 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5188c87640 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35b6390 0x55bed3995cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189488640 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 -- 172.21.15.31:0/31523235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35b6390 msgr2=0x55bed3995cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35b6390 0x55bed3995cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 -- 172.21.15.31:0/31523235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 msgr2=0x55bed398a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5189c89640 1 -- 172.21.15.31:0/31523235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed3988c30 con 0x55bed35d7070 2024-09-21T03:02:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.562+0000 7f5188c87640 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35b6390 0x55bed3995cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:02:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.563+0000 7f5189c89640 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 0x55bed3986ec0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f517400f8c0 tx=0x7f5174015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9580d703a5475668 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.563+0000 7f516affd640 1 -- 172.21.15.31:0/31523235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517402c070 con 0x55bed35d7070 2024-09-21T03:02:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.563+0000 7f516affd640 1 -- 172.21.15.31:0/31523235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517400fe40 con 0x55bed35d7070 2024-09-21T03:02:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.563+0000 7f516affd640 1 -- 172.21.15.31:0/31523235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174027b90 con 0x55bed35d7070 2024-09-21T03:02:29.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 -- 172.21.15.31:0/31523235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 msgr2=0x55bed3986ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 0x55bed3986ec0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f517400f8c0 tx=0x7f5174015000 comp rx=0 tx=0).stop 2024-09-21T03:02:29.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 -- 172.21.15.31:0/31523235 shutdown_connections 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35b6390 0x55bed3995cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35d7070 0x55bed3986ec0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 --2- 172.21.15.31:0/31523235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 -- 172.21.15.31:0/31523235 >> 172.21.15.31:0/31523235 conn(0x55bed35cb460 msgr2=0x55bed3989a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 -- 172.21.15.31:0/31523235 shutdown_connections 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 -- 172.21.15.31:0/31523235 wait complete. 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 -- start start 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.564+0000 7f5191c60740 1 Processor -- start 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed398e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3a05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b2d80 con 0x55bed35b6390 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b26b0 con 0x55bed35ae1b0 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51841af8f0 con 0x55bed35d7070 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5191c60740 10 client.0 _ll_register_callbacks cb 0x7f517c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed398e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed398e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49412/0 (socket says 172.21.15.31:49412) 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 -- 172.21.15.31:0/1299598725 learned_addr learned my addr 172.21.15.31:0/1299598725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5189488640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5189c89640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3a05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 -- 172.21.15.31:0/1299598725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 msgr2=0x55bed3a05790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3a05790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 -- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 msgr2=0x55bed398dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.565+0000 7f5188c87640 1 -- 172.21.15.31:0/1299598725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed398ece0 con 0x55bed35b6390 2024-09-21T03:02:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5188c87640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed398e4f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f51780cab80 tx=0x7f5178003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1965dcbf7768298f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5168ff9640 1 -- 172.21.15.31:0/1299598725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178049040 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5168ff9640 1 -- 172.21.15.31:0/1299598725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f5178003c50 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517400fe40 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5168ff9640 1 -- 172.21.15.31:0/1299598725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51780ce670 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 10 client.17778 fetch_fsmap learned FSMap version 27 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bed3a06c90 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5168ff9640 1 -- 172.21.15.31:0/1299598725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517400fe40 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5168ff9640 1 -- 172.21.15.31:0/1299598725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f51780ce810 con 0x55bed35b6390 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 10 client.17778 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 20 client.17778 populate_metadata read hostname 'smithi031' 2024-09-21T03:02:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed356ada0 0x55bed3a09230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 4 client.17778 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.566+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bed38f2850 con 0x55bed356ada0 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.567+0000 7f5189488640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed356ada0 0x55bed3a09230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.567+0000 7f5189488640 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed356ada0 0x55bed3a09230 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2f8636e16ba28c server_cookie=63e8d9c908c27dd4 in_seq=0 out_seq=0 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.567+0000 7f5168ff9640 10 client.17778 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.567+0000 7f5168ff9640 1 -- 172.21.15.31:0/1299598725 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55bed38f2850 con 0x55bed356ada0 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.567+0000 7f5168ff9640 10 client.17778 handle_command_reply: tid=0 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.567+0000 7f5191c60740 1 client.17778 shutdown 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 msgr2=0x55bed398e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed398e4f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f51780cab80 tx=0x7f5178003f50 comp rx=0 tx=0).stop 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 shutdown_connections 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55bed356ada0 0x55bed3a09230 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bed35d7070 0x55bed3a05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bed35b6390 0x55bed398e4f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 --2- 172.21.15.31:0/1299598725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bed35ae1b0 0x55bed398dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 >> 172.21.15.31:0/1299598725 conn(0x55bed35cb460 msgr2=0x55bed397f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5168ff9640 0 client.17778 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 shutdown_connections 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 1 -- 172.21.15.31:0/1299598725 wait complete. 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f5191c60740 20 client.17778 trim_cache size 0 max 16384 2024-09-21T03:02:29.574 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "2043732d-b892-4c41-ac8d-903a913c8697": { 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "2043732d-b892-4c41-ac8d-903a913c8697", 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "2043732d-b892-4c41-ac8d-903a913c8697": { 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:02:28.145196+0000", 2024-09-21T03:02:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "6c1e1d72-fabe-4268-a17c-b4153b12ad02": { 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:01:43.102904+0000", 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:58.094612+0000", 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:02:29.578 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.582 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.583 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:02:29.584 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f516c0b66b0 msgr2=0x7f516c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f518b6ec640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f516c0b66b0 0x7f516c0b8bc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5174010620 tx=0x7f5174007520 comp rx=0 tx=0).stop 2024-09-21T03:02:29.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 msgr2=0x7f518419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.568+0000 7f518b6ec640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518419cc80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f517802a930 tx=0x7f5178004270 comp rx=0 tx=0).stop 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 shutdown_connections 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f516c0b66b0 0x7f516c0b8bc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51841101a0 0x7f51841a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5184076080 0x7f518419cc80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 --2- 172.21.15.31:0/3842395028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5184075730 0x7f518419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 >> 172.21.15.31:0/3842395028 conn(0x7f518407b910 msgr2=0x7f5184109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 shutdown_connections 2024-09-21T03:02:29.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:02:29.569+0000 7f518b6ec640 1 -- 172.21.15.31:0/3842395028 wait complete. 2024-09-21T03:02:30.375 INFO:tasks.cephfs.filesystem:scrub status for tag:2043732d-b892-4c41-ac8d-903a913c8697 - {'path': '/', 'tag': '2043732d-b892-4c41-ac8d-903a913c8697', 'options': 'recursive,force'} 2024-09-21T03:02:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:30 smithi178 ceph-mon[34759]: pgmap v619: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:30.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:30 smithi178 ceph-mon[34759]: scrub summary: idle 2024-09-21T03:02:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:30 smithi031 ceph-mon[31209]: pgmap v619: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:31.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:30 smithi031 ceph-mon[31209]: scrub summary: idle 2024-09-21T03:02:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:30 smithi165 ceph-mon[34818]: pgmap v619: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:31.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:30 smithi165 ceph-mon[34818]: scrub summary: idle 2024-09-21T03:02:32.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:32 smithi178 ceph-mon[34759]: pgmap v620: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:02:33.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:32 smithi031 ceph-mon[31209]: pgmap v620: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:02:33.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:32 smithi165 ceph-mon[34818]: pgmap v620: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T03:02:35.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:34 smithi031 ceph-mon[31209]: pgmap v621: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:02:35.327 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:34 smithi178 ceph-mon[34759]: pgmap v621: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:02:35.328 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:34 smithi165 ceph-mon[34818]: pgmap v621: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T03:02:37.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:36 smithi031 ceph-mon[31209]: pgmap v622: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T03:02:37.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:36 smithi165 ceph-mon[34818]: pgmap v622: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T03:02:37.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:36 smithi178 ceph-mon[34759]: pgmap v622: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-09-21T03:02:39.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:38 smithi031 ceph-mon[31209]: pgmap v623: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:02:39.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:38 smithi165 ceph-mon[34818]: pgmap v623: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:02:39.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:38 smithi178 ceph-mon[34759]: pgmap v623: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 10 op/s 2024-09-21T03:02:41.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:40 smithi031 ceph-mon[31209]: pgmap v624: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:02:41.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:40 smithi165 ceph-mon[34818]: pgmap v624: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:02:41.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:40 smithi178 ceph-mon[34759]: pgmap v624: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T03:02:43.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:42 smithi031 ceph-mon[31209]: pgmap v625: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T03:02:43.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:42 smithi165 ceph-mon[34818]: pgmap v625: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T03:02:43.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:42 smithi178 ceph-mon[34759]: pgmap v625: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-21T03:02:45.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:44 smithi031 ceph-mon[31209]: pgmap v626: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:02:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:44 smithi165 ceph-mon[34818]: pgmap v626: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:02:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:44 smithi178 ceph-mon[34759]: pgmap v626: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:02:47.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:46 smithi031 ceph-mon[31209]: pgmap v627: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:47.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:46 smithi165 ceph-mon[34818]: pgmap v627: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:47.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:46 smithi178 ceph-mon[34759]: pgmap v627: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:49.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:48 smithi031 ceph-mon[31209]: pgmap v628: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:02:49.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:48 smithi165 ceph-mon[34818]: pgmap v628: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:02:49.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:48 smithi178 ceph-mon[34759]: pgmap v628: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:02:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:50 smithi031 ceph-mon[31209]: pgmap v629: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:51.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:50 smithi165 ceph-mon[34818]: pgmap v629: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:51.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:50 smithi178 ceph-mon[34759]: pgmap v629: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:53.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:52 smithi031 ceph-mon[31209]: pgmap v630: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:53.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:52 smithi165 ceph-mon[34818]: pgmap v630: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:53.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:52 smithi178 ceph-mon[34759]: pgmap v630: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T03:02:55.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:54 smithi031 ceph-mon[31209]: pgmap v631: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:02:55.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:54 smithi165 ceph-mon[34818]: pgmap v631: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:02:55.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:54 smithi178 ceph-mon[34759]: pgmap v631: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-21T03:02:57.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:56 smithi031 ceph-mon[31209]: pgmap v632: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:57.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:56 smithi165 ceph-mon[34818]: pgmap v632: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:57.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:56 smithi178 ceph-mon[34759]: pgmap v632: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T03:02:59.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:02:58 smithi031 ceph-mon[31209]: pgmap v633: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:02:59.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:02:58 smithi165 ceph-mon[34818]: pgmap v633: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:02:59.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:02:58 smithi178 ceph-mon[34759]: pgmap v633: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-21T03:03:00.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:03:00.667 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:01.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:00 smithi031 ceph-mon[31209]: pgmap v634: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 -- 172.21.15.31:0/3403412279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0102a60 msgr2=0x7ff3c0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 --2- 172.21.15.31:0/3403412279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0102a60 0x7ff3c0102e70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009900 tx=0x7ff3b802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 -- 172.21.15.31:0/3403412279 shutdown_connections 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 --2- 172.21.15.31:0/3403412279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3c01083a0 0x7ff3c01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 --2- 172.21.15.31:0/3403412279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 0x7ff3c010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 --2- 172.21.15.31:0/3403412279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c0102a60 0x7ff3c0102e70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 -- 172.21.15.31:0/3403412279 >> 172.21.15.31:0/3403412279 conn(0x7ff3c0079730 msgr2=0x7ff3c0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 -- 172.21.15.31:0/3403412279 shutdown_connections 2024-09-21T03:03:01.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.167+0000 7ff3c7948640 1 -- 172.21.15.31:0/3403412279 wait complete. 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 Processor -- start 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 -- start start 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3c0102a60 0x7ff3c019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 0x7ff3c019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 0x7ff3c019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0112960 con 0x7ff3c01083a0 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01127e0 con 0x7ff3c0108cf0 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c7948640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0112ae0 con 0x7ff3c0102a60 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 0x7ff3c019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 0x7ff3c019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45424/0 (socket says 172.21.15.31:45424) 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.168+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/745370239 learned_addr learned my addr 172.21.15.31:0/745370239 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3c0102a60 0x7ff3c019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 0x7ff3c019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/745370239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3c0102a60 msgr2=0x7ff3c019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3c0102a60 0x7ff3c019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/745370239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 msgr2=0x7ff3c019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 0x7ff3c019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/745370239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c01a02d0 con 0x7ff3c01083a0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 0x7ff3c019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 0x7ff3c019a9c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400ccb0 tx=0x7ff3b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3be7fc640 1 -- 172.21.15.31:0/745370239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400ce60 con 0x7ff3c01083a0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3be7fc640 1 -- 172.21.15.31:0/745370239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b400fb00 con 0x7ff3c01083a0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c01a05c0 con 0x7ff3c01083a0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3be7fc640 1 -- 172.21.15.31:0/745370239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4018bb0 con 0x7ff3c01083a0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.169+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c01a6de0 con 0x7ff3c01083a0 2024-09-21T03:03:01.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.172+0000 7ff3cdebc740 1 Processor -- start 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.172+0000 7ff3cdebc740 1 -- start start 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.172+0000 7ff3be7fc640 1 -- 172.21.15.31:0/745370239 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff3b400f650 con 0x7ff3c01083a0 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4eff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4effb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557fc4e75e50 con 0x557fc4b37390 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557fc4e65280 con 0x557fc4b2f1b0 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557fc4e69090 con 0x557fc4b58070 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45438/0 (socket says 172.21.15.31:45438) 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/4099436977 learned_addr learned my addr 172.21.15.31:0/4099436977 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4effb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4eff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/4099436977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 msgr2=0x557fc4eff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4eff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/4099436977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 msgr2=0x557fc4effb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4effb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.173+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/4099436977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc4bc3650 con 0x557fc4b37390 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f000d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40e5210 tx=0x7ff3b40ebdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533c26f6d7550460 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3be7fc640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3a80b66b0 0x7ff3a80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3a80b66b0 0x7ff3a80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3a77fe640 1 -- 172.21.15.31:0/4099436977 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40f7020 con 0x557fc4b37390 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3a77fe640 1 -- 172.21.15.31:0/4099436977 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b400f940 con 0x557fc4b37390 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3a77fe640 1 -- 172.21.15.31:0/4099436977 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40ad7a0 con 0x557fc4b37390 2024-09-21T03:03:01.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.174+0000 7ff3be7fc640 1 -- 172.21.15.31:0/745370239 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7ff3b40e7b80 con 0x7ff3c01083a0 2024-09-21T03:03:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 -- 172.21.15.31:0/4099436977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 msgr2=0x557fc4f000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f000d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40e5210 tx=0x7ff3b40ebdf0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3a80b66b0 0x7ff3a80b8bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0005fd0 tx=0x7ff3b0009330 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 -- 172.21.15.31:0/4099436977 shutdown_connections 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f000d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4effb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/4099436977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4eff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 -- 172.21.15.31:0/4099436977 >> 172.21.15.31:0/4099436977 conn(0x557fc4b4c460 msgr2=0x557fc4efd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 -- 172.21.15.31:0/4099436977 shutdown_connections 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 -- 172.21.15.31:0/4099436977 wait complete. 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.175+0000 7ff3cdebc740 1 -- start start 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 Processor -- start 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b37390 0x557fc4f0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 0x557fc4f0fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557fc4cc6700 con 0x557fc4b58070 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557fc4e00110 con 0x557fc4b2f1b0 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557fc4f12190 con 0x557fc4b37390 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3cdebc740 10 client.0 _ll_register_callbacks cb 0x557fc4e19990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3c5ee5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 0x557fc4f0fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3c5ee5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 0x557fc4f0fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45452/0 (socket says 172.21.15.31:45452) 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3c5ee5640 1 -- 172.21.15.31:0/1100927849 learned_addr learned my addr 172.21.15.31:0/1100927849 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.176+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b37390 0x557fc4f0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3c5ee5640 1 -- 172.21.15.31:0/1100927849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b37390 msgr2=0x557fc4f0f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b37390 0x557fc4f0f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3c5ee5640 1 -- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 msgr2=0x557fc4f0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3c5ee5640 1 -- 172.21.15.31:0/1100927849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc4e2dc40 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 0x557fc4f0fab0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00126d0 tx=0x7ff3b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb771fb6c5d326 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/1100927849 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0023070 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/1100927849 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7ff3b00118b0 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fc4e77da0 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/1100927849 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3b002f050 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3cdebc740 10 client.17790 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557fc4f1de90 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.177+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/1100927849 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557fc4e77da0 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/1100927849 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b000f040 con 0x557fc4b58070 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3cdebc740 10 client.17790 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3cdebc740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:01.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3cdebc740 20 client.17790 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4b169f0 0x557fc4f21390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3cdebc740 4 client.17790 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc4e73850 con 0x557fc4b169f0 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4b169f0 0x557fc4f21390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3a4ff9640 10 client.17790 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.178+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4b169f0 0x557fc4f21390 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=244cc7bac866d3f1 server_cookie=aeabd8a4ed8bcf0e in_seq=0 out_seq=0 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/1100927849 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fc4e73850 con 0x557fc4b169f0 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3a4ff9640 10 client.17790 handle_command_reply: tid=0 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3cdebc740 1 client.17790 shutdown 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 msgr2=0x557fc4f0fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 0x557fc4f0fab0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00126d0 tx=0x7ff3b0004730 comp rx=0 tx=0).stop 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 shutdown_connections 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.179+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4b169f0 0x557fc4f21390 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b37390 0x557fc4f0f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b58070 0x557fc4f0fab0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/1100927849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 >> 172.21.15.31:0/1100927849 conn(0x557fc4b4c460 msgr2=0x557fc4efbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3a4ff9640 0 client.17790 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 shutdown_connections 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 1 -- 172.21.15.31:0/1100927849 wait complete. 2024-09-21T03:03:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.180+0000 7ff3cdebc740 20 client.17790 trim_cache size 0 max 16384 2024-09-21T03:03:01.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:00 smithi165 ceph-mon[34818]: pgmap v634: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.197+0000 7ff3cdebc740 1 Processor -- start 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.197+0000 7ff3cdebc740 1 -- start start 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f01150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f10f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4efd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01ad4b0 con 0x557fc4b37390 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01aeff0 con 0x557fc4b2f1b0 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01adb80 con 0x557fc4b58070 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f10f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f10f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45464/0 (socket says 172.21.15.31:45464) 2024-09-21T03:03:01.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/3175388248 learned_addr learned my addr 172.21.15.31:0/3175388248 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4efd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f01150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/3175388248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 msgr2=0x557fc4efd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4efd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/3175388248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 msgr2=0x557fc4f01150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f01150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/3175388248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc4f00440 con 0x557fc4b37390 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.198+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f10f10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400ab20 tx=0x7ff3b400af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383e1a23895475c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3a6ffd640 1 -- 172.21.15.31:0/3175388248 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40207d0 con 0x557fc4b37390 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3a6ffd640 1 -- 172.21.15.31:0/3175388248 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b400fcb0 con 0x557fc4b37390 2024-09-21T03:03:01.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3a6ffd640 1 -- 172.21.15.31:0/3175388248 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4020a10 con 0x557fc4b37390 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 -- 172.21.15.31:0/3175388248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 msgr2=0x557fc4f10f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f10f10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400ab20 tx=0x7ff3b400af60 comp rx=0 tx=0).stop 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 -- 172.21.15.31:0/3175388248 shutdown_connections 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4efd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f10f10 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/3175388248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f01150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 -- 172.21.15.31:0/3175388248 >> 172.21.15.31:0/3175388248 conn(0x557fc4b4c460 msgr2=0x557fc4f047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 -- 172.21.15.31:0/3175388248 shutdown_connections 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.199+0000 7ff3cdebc740 1 -- 172.21.15.31:0/3175388248 wait complete. 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 -- start start 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 Processor -- start 2024-09-21T03:03:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4f137f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01ae920 con 0x557fc4b37390 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01ae250 con 0x557fc4b2f1b0 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01ab490 con 0x557fc4b58070 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3cdebc740 10 client.0 _ll_register_callbacks cb 0x7ff3b0016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3c4ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f13210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45472/0 (socket says 172.21.15.31:45472) 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.200+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/2056314616 learned_addr learned my addr 172.21.15.31:0/2056314616 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c5ee5640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4f137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/2056314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 msgr2=0x557fc4f137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4f137f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 msgr2=0x557fc4f17d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f17d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c4ee3640 1 -- 172.21.15.31:0/2056314616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc4f13f80 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3c4ee3640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f13210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40ebc80 tx=0x7ff3b40e7e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f31a4526b7e755 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/2056314616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40f7070 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/2056314616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7ff3b40b21c0 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b400fcb0 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/2056314616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3b4020c30 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3cdebc740 10 client.17796 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.201+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557fc4f8bd40 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/2056314616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b400fcb0 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/2056314616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b40a9c40 con 0x557fc4b37390 2024-09-21T03:03:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3cdebc740 10 client.17796 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3cdebc740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3cdebc740 20 client.17796 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4aebda0 0x557fc4f8df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3cdebc740 4 client.17796 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fc4e70c80 con 0x557fc4aebda0 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.202+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4aebda0 0x557fc4f8df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3c56e4640 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4aebda0 0x557fc4f8df70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66972ca303c9cee server_cookie=8a06bb4b57cbddcd in_seq=0 out_seq=0 2024-09-21T03:03:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3a4ff9640 10 client.17796 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3a4ff9640 1 -- 172.21.15.31:0/2056314616 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x557fc4e70c80 con 0x557fc4aebda0 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3a4ff9640 10 client.17796 handle_command_reply: tid=0 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 client.17796 shutdown 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 msgr2=0x557fc4f13210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f13210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40ebc80 tx=0x7ff3b40e7e00 comp rx=0 tx=0).stop 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 shutdown_connections 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x557fc4aebda0 0x557fc4f8df70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fc4b58070 0x557fc4f137f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557fc4b37390 0x557fc4f13210 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3a4ff9640 0 client.17796 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 --2- 172.21.15.31:0/2056314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557fc4b2f1b0 0x557fc4f17d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 >> 172.21.15.31:0/2056314616 conn(0x557fc4b4c460 msgr2=0x557fc4f102f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 shutdown_connections 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.203+0000 7ff3cdebc740 1 -- 172.21.15.31:0/2056314616 wait complete. 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3cdebc740 20 client.17796 trim_cache size 0 max 16384 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T03:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "2043732d-b892-4c41-ac8d-903a913c8697": { 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:02:28.145196+0000", 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "6c1e1d72-fabe-4268-a17c-b4153b12ad02": { 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:01:43.102904+0000", 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "db59a392-c804-4e73-833d-e8691cb9fccb": { 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:58.094612+0000", 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "2733a6e0-8560-4410-bda8-b9913220c0b1": { 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:06.574991+0000", 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "682624d1-600d-40b9-ad5a-e348fe3e999b": { 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:38.665561+0000", 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "6f42d9dd-0ce6-4d1a-9e7f-7d629162f423": { 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:54:54.815307+0000", 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "85a5ac9c-d6e6-477d-aeae-fbd2fa198d21": { 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:53:53.093273+0000", 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "70992e9d-1ff5-484f-832c-e890cc2e84da": { 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:09.492077+0000", 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "e8dfcb4e-73ff-483a-8d9e-e6e43875c7ea": { 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T03:00:12.698326+0000", 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "c5ad05d6-d975-460d-b1d3-f1af6af3b302": { 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:55:25.225830+0000", 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "1f3c55b2-f33d-43f4-a2d0-5017d66838d8": { 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:10.667115+0000", 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "a751537f-fbc5-4cd8-b5f6-405f14d5969d": { 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:56:56.485392+0000", 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "affa0fa7-6833-420c-a39f-8c6ea226e382": { 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:57:42.463672+0000", 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "4b117554-5eb7-4b30-bc16-452cb8bcca0d": { 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:58:27.887962+0000", 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "18df0cce-9122-4996-a67b-d5d7fb77b7fb": { 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:12.894260+0000", 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "73ff63e2-bd89-498f-99f1-9c76816e1b1e": { 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T02:59:58.833543+0000", 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3a80b66b0 msgr2=0x7ff3a80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3a80b66b0 0x7ff3a80b8bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0005fd0 tx=0x7ff3b0009330 comp rx=0 tx=0).stop 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 msgr2=0x7ff3c019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 0x7ff3c019a9c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400ccb0 tx=0x7ff3b400a620 comp rx=0 tx=0).stop 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 shutdown_connections 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff3a80b66b0 0x7ff3a80b8bc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3c0108cf0 0x7ff3c019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3c01083a0 0x7ff3c019a9c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 --2- 172.21.15.31:0/745370239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3c0102a60 0x7ff3c019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 >> 172.21.15.31:0/745370239 conn(0x7ff3c0079730 msgr2=0x7ff3c0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 shutdown_connections 2024-09-21T03:03:01.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:01.204+0000 7ff3c7948640 1 -- 172.21.15.31:0/745370239 wait complete. 2024-09-21T03:03:01.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:00 smithi178 ceph-mon[34759]: pgmap v634: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:03:02.054 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:03:02.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:03:02.316 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3345461041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988102ae0 msgr2=0x7fd988102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3345461041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988102ae0 0x7fd988102ef0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd978009900 tx=0x7fd97802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3345461041 shutdown_connections 2024-09-21T03:03:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3345461041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd988108d20 0x7fd98810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3345461041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9881083d0 0x7fd9881087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3345461041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988102ae0 0x7fd988102ef0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3345461041 >> 172.21.15.31:0/3345461041 conn(0x7fd988079760 msgr2=0x7fd988079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3345461041 shutdown_connections 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.785+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3345461041 wait complete. 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 Processor -- start 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 -- start start 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd988102ae0 0x7fd9881a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9881083d0 0x7fd9881a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 0x7fd9881a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112990 con 0x7fd988108d20 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112810 con 0x7fd9881083d0 2024-09-21T03:03:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98e8ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd988112b10 con 0x7fd988102ae0 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98ce8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 0x7fd9881a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98ce8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 0x7fd9881a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45480/0 (socket says 172.21.15.31:45480) 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd98ce8a640 1 -- 172.21.15.31:0/3717065905 learned_addr learned my addr 172.21.15.31:0/3717065905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd97ffff640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9881083d0 0x7fd9881a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.786+0000 7fd987fff640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd988102ae0 0x7fd9881a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98ce8a640 1 -- 172.21.15.31:0/3717065905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd988102ae0 msgr2=0x7fd9881a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98ce8a640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd988102ae0 0x7fd9881a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98ce8a640 1 -- 172.21.15.31:0/3717065905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9881083d0 msgr2=0x7fd9881a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98ce8a640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9881083d0 0x7fd9881a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98ce8a640 1 -- 172.21.15.31:0/3717065905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881a6aa0 con 0x7fd988108d20 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98ce8a640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 0x7fd9881a6370 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd980009fd0 tx=0x7fd98000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980017070 con 0x7fd988108d20 2024-09-21T03:03:02.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881a6d90 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980003e20 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980010460 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.787+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881ad4f0 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.788+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd988103d10 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.790+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fd980003970 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.791+0000 7fd985ffb640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9740b6680 0x7fd9740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.791+0000 7fd987fff640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9740b6680 0x7fd9740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.791+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fd9800e7440 con 0x7fd988108d20 2024-09-21T03:03:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.791+0000 7fd987fff640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9740b6680 0x7fd9740b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd978031440 tx=0x7fd978005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:02.794+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9800ad950 con 0x7fd988108d20 2024-09-21T03:03:03.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.022+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9881a1c60 con 0x7fd988108d20 2024-09-21T03:03:03.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:02 smithi031 ceph-mon[31209]: pgmap v635: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:03:03.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.023+0000 7fd985ffb640 1 -- 172.21.15.31:0/3717065905 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fd9800b1d30 con 0x7fd988108d20 2024-09-21T03:03:03.025 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:03:03.025 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:03:03.025 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9740b6680 msgr2=0x7fd9740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9740b6680 0x7fd9740b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd978031440 tx=0x7fd978005d50 comp rx=0 tx=0).stop 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 msgr2=0x7fd9881a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 0x7fd9881a6370 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd980009fd0 tx=0x7fd98000b260 comp rx=0 tx=0).stop 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 shutdown_connections 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fd9740b6680 0x7fd9740b8b90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd988108d20 0x7fd9881a6370 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9881083d0 0x7fd9881a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 --2- 172.21.15.31:0/3717065905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd988102ae0 0x7fd9881a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 >> 172.21.15.31:0/3717065905 conn(0x7fd988079760 msgr2=0x7fd988101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 shutdown_connections 2024-09-21T03:03:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:03.025+0000 7fd98e8ed640 1 -- 172.21.15.31:0/3717065905 wait complete. 2024-09-21T03:03:03.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:02 smithi165 ceph-mon[34818]: pgmap v635: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:03:03.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:02 smithi178 ceph-mon[34759]: pgmap v635: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T03:03:03.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T03:03:03.999 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:04.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:03 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3717065905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:04.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:03 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3717065905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:04.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:03 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3717065905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 -- 172.21.15.31:0/2083965132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a8105300 msgr2=0x7f76a810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 --2- 172.21.15.31:0/2083965132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a8105300 0x7f76a810d850 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f769c009900 tx=0x7f769c02a410 comp rx=0 tx=0).stop 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 -- 172.21.15.31:0/2083965132 shutdown_connections 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 --2- 172.21.15.31:0/2083965132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76a810de10 0x7f76a8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 --2- 172.21.15.31:0/2083965132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a8105300 0x7f76a810d850 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 --2- 172.21.15.31:0/2083965132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a81049b0 0x7f76a8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 -- 172.21.15.31:0/2083965132 >> 172.21.15.31:0/2083965132 conn(0x7f76a8079730 msgr2=0x7f76a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 -- 172.21.15.31:0/2083965132 shutdown_connections 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.445+0000 7f76aed62640 1 -- 172.21.15.31:0/2083965132 wait complete. 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 Processor -- start 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 -- start start 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 0x7f76a819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76a8105300 0x7f76a819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a810de10 0x7f76a819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113ab0 con 0x7f76a81049b0 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113930 con 0x7f76a8105300 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76aed62640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113c30 con 0x7f76a810de10 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.446+0000 7f76acafe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 0x7f76a819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 0x7f76a819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45500/0 (socket says 172.21.15.31:45500) 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 -- 172.21.15.31:0/3691175851 learned_addr learned my addr 172.21.15.31:0/3691175851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76a8105300 0x7f76a819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a810de10 0x7f76a819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 -- 172.21.15.31:0/3691175851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a810de10 msgr2=0x7f76a819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a810de10 0x7f76a819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 -- 172.21.15.31:0/3691175851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76a8105300 msgr2=0x7f76a819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76a8105300 0x7f76a819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 -- 172.21.15.31:0/3691175851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a81a0390 con 0x7f76a81049b0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76acafe640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 0x7f76a819a4e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f769800c970 tx=0x7f769800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76a5ffb640 1 -- 172.21.15.31:0/3691175851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698007c10 con 0x7f76a81049b0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a81a0680 con 0x7f76a81049b0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76a5ffb640 1 -- 172.21.15.31:0/3691175851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7698007db0 con 0x7f76a81049b0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76a5ffb640 1 -- 172.21.15.31:0/3691175851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769800f7d0 con 0x7f76a81049b0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.447+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a6ea0 con 0x7f76a81049b0 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 Processor -- start 2024-09-21T03:03:04.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 -- start start 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 0x5610685b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681ef390 0x5610685b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56106852dea0 con 0x5610681e71b0 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56106851d2d0 con 0x5610681ef390 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76b52d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610685210e0 con 0x561068210070 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76a5ffb640 1 -- 172.21.15.31:0/3691175851 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f769800fa10 con 0x7f76a81049b0 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76acafe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 0x5610685b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76ad2ff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76acafe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 0x5610685b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45504/0 (socket says 172.21.15.31:45504) 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76acafe640 1 -- 172.21.15.31:0/3446332972 learned_addr learned my addr 172.21.15.31:0/3446332972 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.451+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681ef390 0x5610685b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76acafe640 1 -- 172.21.15.31:0/3446332972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 msgr2=0x5610685b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76acafe640 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685b76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76acafe640 1 -- 172.21.15.31:0/3446332972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681ef390 msgr2=0x5610685b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76acafe640 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681ef390 0x5610685b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76acafe640 1 -- 172.21.15.31:0/3446332972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106827b650 con 0x5610681e71b0 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76acafe640 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 0x5610685b7be0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f76980e51d0 tx=0x7f76980ebc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f235bde303b7d733 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76a5ffb640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76900b66b0 0x7f76900b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76900b66b0 0x7f76900b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f768effd640 1 -- 172.21.15.31:0/3446332972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980f7020 con 0x5610681e71b0 2024-09-21T03:03:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f768effd640 1 -- 172.21.15.31:0/3446332972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76980ad530 con 0x5610681e71b0 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.452+0000 7f76a5ffb640 1 -- 172.21.15.31:0/3691175851 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f76980e7b40 con 0x7f76a81049b0 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 -- 172.21.15.31:0/3446332972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 msgr2=0x5610685b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76900b66b0 0x7f76900b8bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f76a000b4d0 tx=0x7f76a000e040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 0x5610685b7be0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f76980e51d0 tx=0x7f76980ebc20 comp rx=0 tx=0).stop 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 -- 172.21.15.31:0/3446332972 shutdown_connections 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681ef390 0x5610685b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681e71b0 0x5610685b7be0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3446332972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 -- 172.21.15.31:0/3446332972 >> 172.21.15.31:0/3446332972 conn(0x561068204460 msgr2=0x5610685b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 -- 172.21.15.31:0/3446332972 shutdown_connections 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.453+0000 7f76b52d6740 1 -- 172.21.15.31:0/3446332972 wait complete. 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.454+0000 7f76b52d6740 1 -- start start 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.454+0000 7f76b52d6740 1 Processor -- start 2024-09-21T03:03:04.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56106837e700 con 0x561068210070 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5610684b8160 con 0x5610681e71b0 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5610685c5ed0 con 0x5610681ef390 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76b52d6740 10 client.0 _ll_register_callbacks cb 0x5610684d19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685c3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45510/0 (socket says 172.21.15.31:45510) 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3028358193 learned_addr learned my addr 172.21.15.31:0/3028358193 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76acafe640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3028358193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 msgr2=0x5610685c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 msgr2=0x5610685c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3028358193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610684e5c90 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685c3690 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f76a0018b80 tx=0x7f76a0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18bd3f5bfe59ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f7687fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.455+0000 7f7687fff640 1 -- 172.21.15.31:0/3028358193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0028060 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f7687fff640 1 -- 172.21.15.31:0/3028358193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f76a00233f0 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76980ad530 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f7687fff640 1 -- 172.21.15.31:0/3028358193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f76a0007e60 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 10 client.17814 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610685d1bc0 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f7687fff640 1 -- 172.21.15.31:0/3028358193 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76980ad530 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f7687fff640 1 -- 172.21.15.31:0/3028358193 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f76a0022070 con 0x561068210070 2024-09-21T03:03:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 10 client.17814 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 10 client.17814 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 20 client.17814 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681ce9f0 0x5610685d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 4 client.17814 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.456+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56106852fdf0 con 0x5610681ce9f0 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.457+0000 7f76acafe640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681ce9f0 0x5610685d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.457+0000 7f7687fff640 10 client.17814 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.457+0000 7f76acafe640 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681ce9f0 0x5610685d50d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=935d8e67f8ecc4c4 server_cookie=ccf1c4e775a8c421 in_seq=0 out_seq=0 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f7687fff640 1 -- 172.21.15.31:0/3028358193 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56106852fdf0 con 0x5610681ce9f0 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f7687fff640 10 client.17814 handle_command_reply: tid=0 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 client.17814 shutdown 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 msgr2=0x5610685c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685c3690 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f76a0018b80 tx=0x7f76a0004b50 comp rx=0 tx=0).stop 2024-09-21T03:03:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 shutdown_connections 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681ce9f0 0x5610685d50d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685c3690 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3028358193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 >> 172.21.15.31:0/3028358193 conn(0x561068204460 msgr2=0x5610685ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f7687fff640 0 client.17814 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 shutdown_connections 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.458+0000 7f76b52d6740 1 -- 172.21.15.31:0/3028358193 wait complete. 2024-09-21T03:03:04.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.459+0000 7f76b52d6740 20 client.17814 trim_cache size 0 max 16384 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.476+0000 7f76b52d6740 1 Processor -- start 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.476+0000 7f76b52d6740 1 -- start start 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76b52d6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81af7e0 con 0x561068210070 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76b52d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81b1320 con 0x5610681e71b0 2024-09-21T03:03:04.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76b52d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81afeb0 con 0x5610681ef390 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45518/0 (socket says 172.21.15.31:45518) 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3934351267 learned_addr learned my addr 172.21.15.31:0/3934351267 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76acafe640 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3934351267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 msgr2=0x5610685c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3934351267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 msgr2=0x5610685b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 -- 172.21.15.31:0/3934351267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610685b9990 con 0x561068210070 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.477+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685cf1a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f76a002b510 tx=0x7f76a0018810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd4ae34b4a350d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.478+0000 7f768e7fc640 1 -- 172.21.15.31:0/3934351267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a002b980 con 0x561068210070 2024-09-21T03:03:04.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.478+0000 7f768e7fc640 1 -- 172.21.15.31:0/3934351267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a002c410 con 0x561068210070 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.478+0000 7f768e7fc640 1 -- 172.21.15.31:0/3934351267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0036650 con 0x561068210070 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.478+0000 7f76b52d6740 1 -- 172.21.15.31:0/3934351267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 msgr2=0x5610685cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.478+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685cf1a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f76a002b510 tx=0x7f76a0018810 comp rx=0 tx=0).stop 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 -- 172.21.15.31:0/3934351267 shutdown_connections 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610681ef390 0x5610685c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561068210070 0x5610685cf1a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3934351267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 -- 172.21.15.31:0/3934351267 >> 172.21.15.31:0/3934351267 conn(0x561068204460 msgr2=0x5610685c29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 -- 172.21.15.31:0/3934351267 shutdown_connections 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 -- 172.21.15.31:0/3934351267 wait complete. 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 -- start start 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.479+0000 7f76b52d6740 1 Processor -- start 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 0x5610685c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81b0c50 con 0x5610681ef390 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81b0580 con 0x5610681e71b0 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81ad7c0 con 0x561068210070 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76b52d6740 10 client.0 _ll_register_callbacks cb 0x7f76a0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 0x5610685c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 0x5610685c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45534/0 (socket says 172.21.15.31:45534) 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 -- 172.21.15.31:0/3093954741 learned_addr learned my addr 172.21.15.31:0/3093954741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76acafe640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76ad2ff640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 -- 172.21.15.31:0/3093954741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 msgr2=0x5610685c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 -- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 msgr2=0x5610685c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 -- 172.21.15.31:0/3093954741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106863eae0 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f76a7fff640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 0x5610685c7650 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f76a806ad30 tx=0x7f769c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b729515f1a14d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.480+0000 7f7687fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f7687fff640 1 -- 172.21.15.31:0/3093954741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c035040 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a002c410 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f7687fff640 1 -- 172.21.15.31:0/3093954741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f769c0042d0 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f7687fff640 1 -- 172.21.15.31:0/3093954741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f769c0044d0 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 10 client.17820 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56106863fce0 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f7687fff640 1 -- 172.21.15.31:0/3093954741 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a002c410 con 0x5610681ef390 2024-09-21T03:03:04.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f7687fff640 1 -- 172.21.15.31:0/3093954741 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f769c0035f0 con 0x5610681ef390 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 10 client.17820 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 10 client.17820 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 20 client.17820 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681a3da0 0x5610686424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 4 client.17820 mds_command: new command op to 24632 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.481+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 --> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56106852b8a0 con 0x5610681a3da0 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76acafe640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681a3da0 0x5610686424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76acafe640 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681a3da0 0x5610686424d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45282a658b4f7f60 server_cookie=96ff9a91f1fdcaa2 in_seq=0 out_seq=0 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f7687fff640 10 client.17820 ms_handle_connect on v2:172.21.15.165:6834/3816837605 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f7687fff640 1 -- 172.21.15.31:0/3093954741 <== mds.4 v2:172.21.15.165:6834/3816837605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56106852b8a0 con 0x5610681a3da0 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f7687fff640 10 client.17820 handle_command_reply: tid=0 2024-09-21T03:03:04.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 client.17820 shutdown 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 msgr2=0x5610685c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 0x5610685c7650 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f76a806ad30 tx=0x7f769c031a50 comp rx=0 tx=0).stop 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 shutdown_connections 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:6834/3816837605,v1:172.21.15.165:6835/3816837605] conn(0x5610681a3da0 0x5610686424d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561068210070 0x5610685c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f7687fff640 0 client.17820 ms_handle_reset on v2:172.21.15.165:6834/3816837605 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5610681ef390 0x5610685c7650 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 --2- 172.21.15.31:0/3093954741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610681e71b0 0x5610685c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.482+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 >> 172.21.15.31:0/3093954741 conn(0x561068204460 msgr2=0x5610685c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 shutdown_connections 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76b52d6740 1 -- 172.21.15.31:0/3093954741 wait complete. 2024-09-21T03:03:04.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76b52d6740 20 client.17820 trim_cache size 0 max 16384 2024-09-21T03:03:04.491 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:03:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76900b66b0 msgr2=0x7f76900b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76900b66b0 0x7f76900b8bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f76a000b4d0 tx=0x7f76a000e040 comp rx=0 tx=0).stop 2024-09-21T03:03:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 msgr2=0x7f76a819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 0x7f76a819a4e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f769800c970 tx=0x7f769800ce30 comp rx=0 tx=0).stop 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 shutdown_connections 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f76900b66b0 0x7f76900b8bc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76a810de10 0x7f76a819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76a8105300 0x7f76a819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 --2- 172.21.15.31:0/3691175851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76a81049b0 0x7f76a819a4e0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.483+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 >> 172.21.15.31:0/3691175851 conn(0x7f76a8079730 msgr2=0x7f76a80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.484+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 shutdown_connections 2024-09-21T03:03:04.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:04.484+0000 7f76aed62640 1 -- 172.21.15.31:0/3691175851 wait complete. 2024-09-21T03:03:05.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:04 smithi031 ceph-mon[31209]: pgmap v636: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:03:05.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T03:03:05.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:04 smithi165 ceph-mon[34818]: pgmap v636: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:03:05.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:04 smithi178 ceph-mon[34759]: pgmap v636: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-21T03:03:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 -- 172.21.15.31:0/1510468635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 msgr2=0x7f4b50103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 --2- 172.21.15.31:0/1510468635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b50103f70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009900 tx=0x7f4b4c02a410 comp rx=0 tx=0).stop 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 -- 172.21.15.31:0/1510468635 shutdown_connections 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 --2- 172.21.15.31:0/1510468635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b501094a0 0x7f4b501098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 --2- 172.21.15.31:0/1510468635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b50109df0 0x7f4b50110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 --2- 172.21.15.31:0/1510468635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b50103f70 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 -- 172.21.15.31:0/1510468635 >> 172.21.15.31:0/1510468635 conn(0x7f4b5007b850 msgr2=0x7f4b5007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 -- 172.21.15.31:0/1510468635 shutdown_connections 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.004+0000 7f4b581f8640 1 -- 172.21.15.31:0/1510468635 wait complete. 2024-09-21T03:03:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 Processor -- start 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 -- start start 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b5019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b501094a0 0x7f4b5019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b50109df0 0x7f4b5019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50113ab0 con 0x7f4b50103b60 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50113930 con 0x7f4b50109df0 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b581f8640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50113c30 con 0x7f4b501094a0 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b55f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b5019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b55f94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b5019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45546/0 (socket says 172.21.15.31:45546) 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.005+0000 7f4b55f94640 1 -- 172.21.15.31:0/2894488155 learned_addr learned my addr 172.21.15.31:0/2894488155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55793640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b501094a0 0x7f4b5019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b56795640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b50109df0 0x7f4b5019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55f94640 1 -- 172.21.15.31:0/2894488155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b501094a0 msgr2=0x7f4b5019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55f94640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b501094a0 0x7f4b5019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55f94640 1 -- 172.21.15.31:0/2894488155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b50109df0 msgr2=0x7f4b5019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55f94640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b50109df0 0x7f4b5019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55f94640 1 -- 172.21.15.31:0/2894488155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b501a0430 con 0x7f4b50103b60 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b55f94640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b5019a4c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c005940 tx=0x7f4b4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b46ffd640 1 -- 172.21.15.31:0/2894488155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c03f030 con 0x7f4b50103b60 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b46ffd640 1 -- 172.21.15.31:0/2894488155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c003cb0 con 0x7f4b50103b60 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b46ffd640 1 -- 172.21.15.31:0/2894488155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c035070 con 0x7f4b50103b60 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b501a06c0 con 0x7f4b50103b60 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.006+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a6e80 con 0x7f4b50103b60 2024-09-21T03:03:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 Processor -- start 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 -- start start 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b46ffd640 1 -- 172.21.15.31:0/2894488155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4b4c042030 con 0x7f4b50103b60 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 0x564e852186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e481b0 0x564e85218be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e85219120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564e8518eea0 con 0x564e84e71070 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.009+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e8517e2d0 con 0x564e84e481b0 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e851820e0 con 0x564e84e50390 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 0x564e852186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 0x564e852186a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45556/0 (socket says 172.21.15.31:45556) 2024-09-21T03:03:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 -- 172.21.15.31:0/3454805006 learned_addr learned my addr 172.21.15.31:0/3454805006 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b55793640 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e85219120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b55f94640 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e481b0 0x564e85218be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 -- 172.21.15.31:0/3454805006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 msgr2=0x564e85219120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e85219120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 -- 172.21.15.31:0/3454805006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e481b0 msgr2=0x564e85218be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e481b0 0x564e85218be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 -- 172.21.15.31:0/3454805006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e84edc650 con 0x564e84e71070 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b56795640 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 0x564e852186a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4b48008dc0 tx=0x7f4b4800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4133935cbd5237b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b46ffd640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4b3c0b6700 0x7f4b3c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.010+0000 7f4b3bfff640 1 -- 172.21.15.31:0/3454805006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48014070 con 0x564e84e71070 2024-09-21T03:03:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b3bfff640 1 -- 172.21.15.31:0/3454805006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b48003b70 con 0x564e84e71070 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b3bfff640 1 -- 172.21.15.31:0/3454805006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48010700 con 0x564e84e71070 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b55f94640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4b3c0b6700 0x7f4b3c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b46ffd640 1 -- 172.21.15.31:0/2894488155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f4b4c107f50 con 0x7f4b50103b60 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b5e76c740 1 -- 172.21.15.31:0/3454805006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 msgr2=0x564e852186a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 0x564e852186a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4b48008dc0 tx=0x7f4b4800e720 comp rx=0 tx=0).stop 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.011+0000 7f4b55f94640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4b3c0b6700 0x7f4b3c0b8c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x564e85217320 tx=0x7f4b4c0045a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 -- 172.21.15.31:0/3454805006 shutdown_connections 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e85219120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e481b0 0x564e85218be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/3454805006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e71070 0x564e852186a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 -- 172.21.15.31:0/3454805006 >> 172.21.15.31:0/3454805006 conn(0x564e84e65460 msgr2=0x564e85216590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 -- 172.21.15.31:0/3454805006 shutdown_connections 2024-09-21T03:03:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 -- 172.21.15.31:0/3454805006 wait complete. 2024-09-21T03:03:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 -- start start 2024-09-21T03:03:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.012+0000 7f4b5e76c740 1 Processor -- start 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e481b0 0x564e85223e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 0x564e85224340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e71070 0x564e852301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564e84fdf700 con 0x564e84e50390 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e85119160 con 0x564e84e71070 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564e85226e80 con 0x564e84e481b0 2024-09-21T03:03:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b5e76c740 10 client.0 _ll_register_callbacks cb 0x564e851329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 0x564e85224340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 0x564e85224340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45572/0 (socket says 172.21.15.31:45572) 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 -- 172.21.15.31:0/1250296788 learned_addr learned my addr 172.21.15.31:0/1250296788 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b56795640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e71070 0x564e852301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b56795640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e481b0 0x564e85223e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 -- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e481b0 msgr2=0x564e85223e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e481b0 0x564e85223e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 -- 172.21.15.31:0/1250296788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e71070 msgr2=0x564e852301a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e71070 0x564e852301a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b55793640 1 -- 172.21.15.31:0/1250296788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e85146c90 con 0x564e84e50390 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.013+0000 7f4b56795640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e481b0 0x564e85223e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b55793640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 0x564e85224340 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b920 tx=0x7f4b4000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671152b82857c327 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b397fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b397fa640 1 -- 172.21.15.31:0/1250296788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40005020 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b397fa640 1 -- 172.21.15.31:0/1250296788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f4b400051c0 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b48003b70 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b397fa640 1 -- 172.21.15.31:0/1250296788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b400053c0 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b5e76c740 10 client.17832 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e85232c80 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b397fa640 1 -- 172.21.15.31:0/1250296788 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b48003b70 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b397fa640 1 -- 172.21.15.31:0/1250296788 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b4001b470 con 0x564e84e50390 2024-09-21T03:03:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b5e76c740 10 client.17832 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b5e76c740 10 client.17832 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.014+0000 7f4b5e76c740 20 client.17832 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.015+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e2f9f0 0x564e852361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.015+0000 7f4b5e76c740 4 client.17832 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.015+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e85190df0 con 0x564e84e2f9f0 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.015+0000 7f4b56795640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e2f9f0 0x564e852361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.015+0000 7f4b56795640 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e2f9f0 0x564e852361f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f44760e6233b8130 server_cookie=192c0b3d507c13c0 in_seq=0 out_seq=0 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.015+0000 7f4b397fa640 10 client.17832 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b397fa640 1 -- 172.21.15.31:0/1250296788 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e85190df0 con 0x564e84e2f9f0 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b397fa640 10 client.17832 handle_command_reply: tid=0 2024-09-21T03:03:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 client.17832 shutdown 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 msgr2=0x564e85224340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 0x564e85224340 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b920 tx=0x7f4b4000bde0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 shutdown_connections 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e2f9f0 0x564e852361f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e71070 0x564e852301a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e50390 0x564e85224340 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b397fa640 0 client.17832 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1250296788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e481b0 0x564e85223e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 >> 172.21.15.31:0/1250296788 conn(0x564e84e65460 msgr2=0x564e8521b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 shutdown_connections 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1250296788 wait complete. 2024-09-21T03:03:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.016+0000 7f4b5e76c740 20 client.17832 trim_cache size 0 max 16384 2024-09-21T03:03:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 Processor -- start 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 -- start start 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e85224380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e8521beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501af760 con 0x564e84e481b0 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b12a0 con 0x564e84e2f9f0 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501afe30 con 0x564e84e50390 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b55793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e8521beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b55793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e8521beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45584/0 (socket says 172.21.15.31:45584) 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b55793640 1 -- 172.21.15.31:0/1723405502 learned_addr learned my addr 172.21.15.31:0/1723405502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b56795640 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.030+0000 7f4b56795640 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e85224380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b55793640 1 -- 172.21.15.31:0/1723405502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 msgr2=0x564e852300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b55793640 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b55793640 1 -- 172.21.15.31:0/1723405502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 msgr2=0x564e85224380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b55793640 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e85224380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b55793640 1 -- 172.21.15.31:0/1723405502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8521c3f0 con 0x564e84e481b0 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b56795640 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e85224380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b55793640 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e8521beb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4b4001cf60 tx=0x7f4b4001cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ebc1c3dc4f0cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b3b7fe640 1 -- 172.21.15.31:0/1723405502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40028040 con 0x564e84e481b0 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b3b7fe640 1 -- 172.21.15.31:0/1723405502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b40005560 con 0x564e84e481b0 2024-09-21T03:03:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.031+0000 7f4b3b7fe640 1 -- 172.21.15.31:0/1723405502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b400117c0 con 0x564e84e481b0 2024-09-21T03:03:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1723405502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 msgr2=0x564e8521beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e8521beb0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4b4001cf60 tx=0x7f4b4001cf90 comp rx=0 tx=0).stop 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1723405502 shutdown_connections 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e8521beb0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/1723405502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e85224380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1723405502 >> 172.21.15.31:0/1723405502 conn(0x564e84e65460 msgr2=0x564e8521b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1723405502 shutdown_connections 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 -- 172.21.15.31:0/1723405502 wait complete. 2024-09-21T03:03:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 -- start start 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.032+0000 7f4b5e76c740 1 Processor -- start 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e8522c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e85227bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b0bd0 con 0x564e84e481b0 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501b0500 con 0x564e84e2f9f0 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501ad740 con 0x564e84e50390 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b5e76c740 10 client.0 _ll_register_callbacks cb 0x7f4b400042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e85227bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e85227bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45596/0 (socket says 172.21.15.31:45596) 2024-09-21T03:03:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 -- 172.21.15.31:0/118862704 learned_addr learned my addr 172.21.15.31:0/118862704 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b56795640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e8522c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b56795640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 -- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 msgr2=0x564e852280f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852280f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 -- 172.21.15.31:0/118862704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 msgr2=0x564e8522c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e8522c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b55793640 1 -- 172.21.15.31:0/118862704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e852286b0 con 0x564e84e481b0 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.033+0000 7f4b56795640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e8522c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b55793640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e85227bb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4b40003ac0 tx=0x7f4b400057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50325eacf53b254 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b397fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b397fa640 1 -- 172.21.15.31:0/118862704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40028030 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b40005560 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b397fa640 1 -- 172.21.15.31:0/118862704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f4b4001bc90 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b397fa640 1 -- 172.21.15.31:0/118862704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b40043050 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 10 client.17838 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e852a0920 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b397fa640 1 -- 172.21.15.31:0/118862704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b40005560 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b397fa640 1 -- 172.21.15.31:0/118862704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b4002b070 con 0x564e84e481b0 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 10 client.17838 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 10 client.17838 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T03:03:06.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 20 client.17838 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e71070 0x564e852a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 4 client.17838 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.034+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 --> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e8518c8a0 con 0x564e84e71070 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b56795640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e71070 0x564e852a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b56795640 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e71070 0x564e852a2fb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd4df508dcefe318 server_cookie=b7cecaac737fa518 in_seq=0 out_seq=0 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b397fa640 10 client.17838 ms_handle_connect on v2:172.21.15.178:6836/1985506653 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b397fa640 1 -- 172.21.15.31:0/118862704 <== mds.3 v2:172.21.15.178:6836/1985506653 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e8518c8a0 con 0x564e84e71070 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b397fa640 10 client.17838 handle_command_reply: tid=0 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b5e76c740 1 client.17838 shutdown 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 msgr2=0x564e85227bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.035+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e85227bb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4b40003ac0 tx=0x7f4b400057a0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 shutdown_connections 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:6836/1985506653,v1:172.21.15.178:6837/1985506653] conn(0x564e84e71070 0x564e852a2fb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564e84e50390 0x564e852280f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564e84e481b0 0x564e85227bb0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b397fa640 0 client.17838 ms_handle_reset on v2:172.21.15.178:6836/1985506653 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 --2- 172.21.15.31:0/118862704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e84e2f9f0 0x564e8522c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 >> 172.21.15.31:0/118862704 conn(0x564e84e65460 msgr2=0x564e852252b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 shutdown_connections 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 1 -- 172.21.15.31:0/118862704 wait complete. 2024-09-21T03:03:06.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b5e76c740 20 client.17838 trim_cache size 0 max 16384 2024-09-21T03:03:06.045 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:03:06.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4b3c0b6700 msgr2=0x7f4b3c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4b3c0b6700 0x7f4b3c0b8c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x564e85217320 tx=0x7f4b4c0045a0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 msgr2=0x7f4b5019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b5019a4c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c005940 tx=0x7f4b4c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 shutdown_connections 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4b3c0b6700 0x7f4b3c0b8c10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b501094a0 0x7f4b5019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b50109df0 0x7f4b5019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 --2- 172.21.15.31:0/2894488155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50103b60 0x7f4b5019a4c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.036+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 >> 172.21.15.31:0/2894488155 conn(0x7f4b5007b850 msgr2=0x7f4b50107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.037+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 shutdown_connections 2024-09-21T03:03:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:06.037+0000 7f4b581f8640 1 -- 172.21.15.31:0/2894488155 wait complete. 2024-09-21T03:03:06.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:07.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:06 smithi031 ceph-mon[31209]: pgmap v637: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:03:07.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:06 smithi165 ceph-mon[34818]: pgmap v637: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:03:07.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:06 smithi178 ceph-mon[34759]: pgmap v637: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:03:07.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 -- 172.21.15.31:0/4246940246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810faf0 msgr2=0x7fcff810ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 --2- 172.21.15.31:0/4246940246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810faf0 0x7fcff810ff00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009900 tx=0x7fcfec02a410 comp rx=0 tx=0).stop 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 -- 172.21.15.31:0/4246940246 shutdown_connections 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 --2- 172.21.15.31:0/4246940246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff8110440 0x7fcff8116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 --2- 172.21.15.31:0/4246940246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810faf0 0x7fcff810ff00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 --2- 172.21.15.31:0/4246940246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff810a1b0 0x7fcff810a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 -- 172.21.15.31:0/4246940246 >> 172.21.15.31:0/4246940246 conn(0x7fcff80780e0 msgr2=0x7fcff81081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 -- 172.21.15.31:0/4246940246 shutdown_connections 2024-09-21T03:03:07.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.549+0000 7fcfff01b640 1 -- 172.21.15.31:0/4246940246 wait complete. 2024-09-21T03:03:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 Processor -- start 2024-09-21T03:03:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 -- start start 2024-09-21T03:03:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 0x7fcff81a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff810faf0 0x7fcff81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff8110440 0x7fcff81a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcff811a0b0 con 0x7fcff810a1b0 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8119f30 con 0x7fcff810faf0 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcfff01b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcff811a230 con 0x7fcff8110440 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcffcdb7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 0x7fcff81a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcffcdb7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 0x7fcff81a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45612/0 (socket says 172.21.15.31:45612) 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.550+0000 7fcffcdb7640 1 -- 172.21.15.31:0/1883431422 learned_addr learned my addr 172.21.15.31:0/1883431422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff8110440 0x7fcff81a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff810faf0 0x7fcff81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffcdb7640 1 -- 172.21.15.31:0/1883431422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff8110440 msgr2=0x7fcff81a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffcdb7640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff8110440 0x7fcff81a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffcdb7640 1 -- 172.21.15.31:0/1883431422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff810faf0 msgr2=0x7fcff81a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffcdb7640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff810faf0 0x7fcff81a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffcdb7640 1 -- 172.21.15.31:0/1883431422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81a6990 con 0x7fcff810a1b0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff810faf0 0x7fcff81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcffcdb7640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 0x7fcff81a0ae0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcff5ffb640 1 -- 172.21.15.31:0/1883431422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cd80 con 0x7fcff810a1b0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81a6c80 con 0x7fcff810a1b0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcff5ffb640 1 -- 172.21.15.31:0/1883431422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe80108e0 con 0x7fcff810a1b0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcff5ffb640 1 -- 172.21.15.31:0/1883431422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8018980 con 0x7fcff810a1b0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.551+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81ad4a0 con 0x7fcff810a1b0 2024-09-21T03:03:07.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.554+0000 7fd00558f740 1 Processor -- start 2024-09-21T03:03:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.554+0000 7fd00558f740 1 -- start start 2024-09-21T03:03:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fd00558f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bde9070 0x55bd4c1902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fd00558f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c190800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fd00558f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c190d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fd00558f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bd4c106ea0 con 0x55bd4bdc01b0 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fd00558f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd4c0f62d0 con 0x55bd4bdc8390 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fd00558f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd4c0fa0e0 con 0x55bd4bde9070 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fcff7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c190d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fcff5ffb640 1 -- 172.21.15.31:0/1883431422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fcfe8010430 con 0x7fcff810a1b0 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fcff7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c190d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.31:55612/0 (socket says 172.21.15.31:55612) 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fcff7fff640 1 -- 172.21.15.31:0/25302516 learned_addr learned my addr 172.21.15.31:0/25302516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bde9070 0x55bd4c1902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.555+0000 7fcffcdb7640 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c190800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffd5b8640 1 -- 172.21.15.31:0/25302516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 msgr2=0x55bd4c190d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c190d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffd5b8640 1 -- 172.21.15.31:0/25302516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 msgr2=0x55bd4c190800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c190800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffd5b8640 1 -- 172.21.15.31:0/25302516 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4be54650 con 0x55bd4bde9070 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcff7fff640 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c190d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bde9070 0x55bd4c1902c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcff0007d90 tx=0x7fcff000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b430e4b01b2af0e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcff5ffb640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcfe00b6730 0x7fcfe00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcfdeffd640 1 -- 172.21.15.31:0/25302516 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0014070 con 0x55bd4bde9070 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcfdeffd640 1 -- 172.21.15.31:0/25302516 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0003e20 con 0x55bd4bde9070 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcfdeffd640 1 -- 172.21.15.31:0/25302516 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0005020 con 0x55bd4bde9070 2024-09-21T03:03:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.556+0000 7fcffcdb7640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcfe00b6730 0x7fcfe00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fcff5ffb640 1 -- 172.21.15.31:0/1883431422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fcfe80e7b10 con 0x7fcff810a1b0 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 -- 172.21.15.31:0/25302516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bde9070 msgr2=0x55bd4c1902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fcffcdb7640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcfe00b6730 0x7fcfe00b8c40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55bd4c18ef40 tx=0x7fcfe80eb5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bde9070 0x55bd4c1902c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcff0007d90 tx=0x7fcff000e720 comp rx=0 tx=0).stop 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 -- 172.21.15.31:0/25302516 shutdown_connections 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c190d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c190800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 --2- 172.21.15.31:0/25302516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bde9070 0x55bd4c1902c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 -- 172.21.15.31:0/25302516 >> 172.21.15.31:0/25302516 conn(0x55bd4bddd460 msgr2=0x55bd4c18e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 -- 172.21.15.31:0/25302516 shutdown_connections 2024-09-21T03:03:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.557+0000 7fd00558f740 1 -- 172.21.15.31:0/25302516 wait complete. 2024-09-21T03:03:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 -- start start 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 Processor -- start 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 0x55bd4c1a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55bd4bf57700 con 0x55bd4bde9070 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd4c091160 con 0x55bd4bdc8390 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.558+0000 7fd00558f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd4c19eaa0 con 0x55bd4bdc01b0 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fd00558f740 10 client.0 _ll_register_callbacks cb 0x55bd4c0aa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 0x55bd4c1a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 0x55bd4c1a7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45630/0 (socket says 172.21.15.31:45630) 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1468953404 learned_addr learned my addr 172.21.15.31:0/1468953404 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 msgr2=0x55bd4c19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1468953404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 msgr2=0x55bd4c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1468953404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4c0bec90 con 0x55bd4bde9070 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 0x55bd4c1a7df0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x55bd4c192820 tx=0x7fcff0004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656531a197a7ad6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcfd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.559+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1468953404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0014030 con 0x55bd4bde9070 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1468953404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fcff0004c40 con 0x55bd4bde9070 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff0003e20 con 0x55bd4bde9070 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1468953404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcff0022040 con 0x55bd4bde9070 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 10 client.17850 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd4c1aa7b0 con 0x55bd4bde9070 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1468953404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0003e20 con 0x55bd4bde9070 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1468953404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcff0018410 con 0x55bd4bde9070 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 10 client.17850 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 10 client.17850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 20 client.17850 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bda79f0 0x55bd4c1adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 4 client.17850 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.560+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd4c108df0 con 0x55bd4bda79f0 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.561+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bda79f0 0x55bd4c1adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.561+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bda79f0 0x55bd4c1adc20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2b4adb37d1fc479 server_cookie=9af707f1347b4acf in_seq=0 out_seq=0 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.561+0000 7fcfd7fff640 10 client.17850 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1468953404 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd4c108df0 con 0x55bd4bda79f0 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fcfd7fff640 10 client.17850 handle_command_reply: tid=0 2024-09-21T03:03:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 client.17850 shutdown 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 msgr2=0x55bd4c1a7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 0x55bd4c1a7df0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x55bd4c192820 tx=0x7fcff0004420 comp rx=0 tx=0).stop 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 shutdown_connections 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bda79f0 0x55bd4c1adc20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fcfd7fff640 0 client.17850 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bde9070 0x55bd4c1a7df0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 --2- 172.21.15.31:0/1468953404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 >> 172.21.15.31:0/1468953404 conn(0x55bd4bddd460 msgr2=0x55bd4c193450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 shutdown_connections 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 1 -- 172.21.15.31:0/1468953404 wait complete. 2024-09-21T03:03:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.562+0000 7fd00558f740 20 client.17850 trim_cache size 0 max 16384 2024-09-21T03:03:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 Processor -- start 2024-09-21T03:03:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 -- start start 2024-09-21T03:03:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bda79f0 0x55bd4c191cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc8390 0x55bd4c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b5df0 con 0x55bd4bdc01b0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b7930 con 0x55bd4bda79f0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fd00558f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b64c0 con 0x55bd4bdc8390 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45634/0 (socket says 172.21.15.31:45634) 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 -- 172.21.15.31:0/3452509833 learned_addr learned my addr 172.21.15.31:0/3452509833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bda79f0 0x55bd4c191cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc8390 0x55bd4c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 -- 172.21.15.31:0/3452509833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc8390 msgr2=0x55bd4c19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc8390 0x55bd4c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 -- 172.21.15.31:0/3452509833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bda79f0 msgr2=0x55bd4c191cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.576+0000 7fcff7fff640 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bda79f0 0x55bd4c191cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.577+0000 7fcff7fff640 1 -- 172.21.15.31:0/3452509833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4c1895d0 con 0x55bd4bdc01b0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.577+0000 7fcff7fff640 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1a7d60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcfec032e70 tx=0x7fcfec005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810336ea7009ed84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.577+0000 7fcfde7fc640 1 -- 172.21.15.31:0/3452509833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec035040 con 0x55bd4bdc01b0 2024-09-21T03:03:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.577+0000 7fcfde7fc640 1 -- 172.21.15.31:0/3452509833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfec049070 con 0x55bd4bdc01b0 2024-09-21T03:03:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.577+0000 7fcfde7fc640 1 -- 172.21.15.31:0/3452509833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec004ab0 con 0x55bd4bdc01b0 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 -- 172.21.15.31:0/3452509833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 msgr2=0x55bd4c1a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1a7d60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcfec032e70 tx=0x7fcfec005a20 comp rx=0 tx=0).stop 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 -- 172.21.15.31:0/3452509833 shutdown_connections 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc8390 0x55bd4c19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1a7d60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 --2- 172.21.15.31:0/3452509833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bda79f0 0x55bd4c191cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 -- 172.21.15.31:0/3452509833 >> 172.21.15.31:0/3452509833 conn(0x55bd4bddd460 msgr2=0x55bd4c19cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 -- 172.21.15.31:0/3452509833 shutdown_connections 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 -- 172.21.15.31:0/3452509833 wait complete. 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 -- start start 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.578+0000 7fd00558f740 1 Processor -- start 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 0x55bd4c1aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c1af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b7260 con 0x55bd4bda79f0 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b6b90 con 0x55bd4bdc8390 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b3dd0 con 0x55bd4bdc01b0 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fd00558f740 10 client.0 _ll_register_callbacks cb 0x7fcfe80eb490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 0x55bd4c1aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 0x55bd4c1aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45646/0 (socket says 172.21.15.31:45646) 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1408320223 learned_addr learned my addr 172.21.15.31:0/1408320223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c1af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 msgr2=0x55bd4c1af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1408320223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 msgr2=0x55bd4c1af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c1af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 -- 172.21.15.31:0/1408320223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4c217450 con 0x55bd4bda79f0 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcffd5b8640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 0x55bd4c1aecf0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcff000eda0 tx=0x7fcff0004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d925bacc938213d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.579+0000 7fcfd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1408320223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff003a020 con 0x55bd4bda79f0 2024-09-21T03:03:07.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1408320223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fcff0018e00 con 0x55bd4bda79f0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfec049070 con 0x55bd4bda79f0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1408320223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcff0014070 con 0x55bd4bda79f0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 10 client.17856 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bd4c2186d0 con 0x55bd4bda79f0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1408320223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfec049070 con 0x55bd4bda79f0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1408320223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcff0003de0 con 0x55bd4bda79f0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 10 client.17856 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 10 client.17856 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 20 client.17856 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bde9070 0x55bd4c21ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 4 client.17856 mds_command: new command op to 24655 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.580+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 --> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd4c1048a0 con 0x55bd4bde9070 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bde9070 0x55bd4c21ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fcff7fff640 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bde9070 0x55bd4c21ae20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39e32b527ad67c06 server_cookie=99f8ff10ec81d765 in_seq=0 out_seq=0 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fcfd7fff640 10 client.17856 ms_handle_connect on v2:172.21.15.178:6838/730341197 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fcfd7fff640 1 -- 172.21.15.31:0/1408320223 <== mds.2 v2:172.21.15.178:6838/730341197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd4c1048a0 con 0x55bd4bde9070 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fcfd7fff640 10 client.17856 handle_command_reply: tid=0 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 client.17856 shutdown 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 msgr2=0x55bd4c1aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 0x55bd4c1aecf0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcff000eda0 tx=0x7fcff0004380 comp rx=0 tx=0).stop 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 shutdown_connections 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:6838/730341197,v1:172.21.15.178:6839/730341197] conn(0x55bd4bde9070 0x55bd4c21ae20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4bdc01b0 0x55bd4c1af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4bdc8390 0x55bd4c1af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 --2- 172.21.15.31:0/1408320223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd4bda79f0 0x55bd4c1aecf0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.581+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 >> 172.21.15.31:0/1408320223 conn(0x55bd4bddd460 msgr2=0x55bd4c191a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfd7fff640 0 client.17856 ms_handle_reset on v2:172.21.15.178:6838/730341197 2024-09-21T03:03:07.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 shutdown_connections 2024-09-21T03:03:07.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fd00558f740 1 -- 172.21.15.31:0/1408320223 wait complete. 2024-09-21T03:03:07.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fd00558f740 20 client.17856 trim_cache size 0 max 16384 2024-09-21T03:03:07.589 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcfe00b6730 msgr2=0x7fcfe00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcfe00b6730 0x7fcfe00b8c40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55bd4c18ef40 tx=0x7fcfe80eb5c0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 msgr2=0x7fcff81a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 0x7fcff81a0ae0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800e9f0 tx=0x7fcfe800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 shutdown_connections 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fcfe00b6730 0x7fcfe00b8c40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcff8110440 0x7fcff81a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff810faf0 0x7fcff81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 --2- 172.21.15.31:0/1883431422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff810a1b0 0x7fcff81a0ae0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.582+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 >> 172.21.15.31:0/1883431422 conn(0x7fcff80780e0 msgr2=0x7fcff810e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.583+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 shutdown_connections 2024-09-21T03:03:07.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.583+0000 7fcfff01b640 1 -- 172.21.15.31:0/1883431422 wait complete. 2024-09-21T03:03:07.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 -- 172.21.15.31:0/1067695569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 msgr2=0x7f67dc109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 --2- 172.21.15.31:0/1067695569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc109810 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009900 tx=0x7f67d002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 -- 172.21.15.31:0/1067695569 shutdown_connections 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 --2- 172.21.15.31:0/1067695569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67dc109d50 0x7f67dc110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 --2- 172.21.15.31:0/1067695569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc109810 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 --2- 172.21.15.31:0/1067695569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 0x7f67dc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 -- 172.21.15.31:0/1067695569 >> 172.21.15.31:0/1067695569 conn(0x7f67dc0ff510 msgr2=0x7f67dc101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 -- 172.21.15.31:0/1067695569 shutdown_connections 2024-09-21T03:03:07.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.916+0000 7f67e4739640 1 -- 172.21.15.31:0/1067695569 wait complete. 2024-09-21T03:03:07.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 Processor -- start 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 -- start start 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 0x7f67dc19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67dc109d50 0x7f67dc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc113a50 con 0x7f67dc109400 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1138d0 con 0x7f67dc103ad0 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e4739640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc113bd0 con 0x7f67dc109d50 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.917+0000 7f67e1cd4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45660/0 (socket says 172.21.15.31:45660) 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 -- 172.21.15.31:0/281786525 learned_addr learned my addr 172.21.15.31:0/281786525 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e2cd6640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67dc109d50 0x7f67dc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e24d5640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 0x7f67dc19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 -- 172.21.15.31:0/281786525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67dc109d50 msgr2=0x7f67dc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67dc109d50 0x7f67dc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 -- 172.21.15.31:0/281786525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 msgr2=0x7f67dc19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 0x7f67dc19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 -- 172.21.15.31:0/281786525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc1a0400 con 0x7f67dc109400 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e24d5640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 0x7f67dc19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e1cd4640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc19aa70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f67d002a930 tx=0x7f67d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d003f030 con 0x7f67dc109400 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d0003aa0 con 0x7f67dc109400 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0035070 con 0x7f67dc109400 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc1a0690 con 0x7f67dc109400 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.918+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a6e80 con 0x7f67dc109400 2024-09-21T03:03:07.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.919+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc104d40 con 0x7f67dc109400 2024-09-21T03:03:07.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.921+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f67d0042030 con 0x7f67dc109400 2024-09-21T03:03:07.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.922+0000 7f67db7fe640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c40b6700 0x7f67c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.922+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f67d0108210 con 0x7f67dc109400 2024-09-21T03:03:07.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.922+0000 7f67e24d5640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c40b6700 0x7f67c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.922+0000 7f67e24d5640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c40b6700 0x7f67c40b8c10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009fa0 tx=0x7f67cc009570 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:07.925+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f67d00ce720 con 0x7f67dc109400 2024-09-21T03:03:08.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.135+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f67dc10bbb0 con 0x7f67c40b6700 2024-09-21T03:03:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.143+0000 7f67db7fe640 1 -- 172.21.15.31:0/281786525 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f67cc0173f0 con 0x7f67c40b6700 2024-09-21T03:03:08.146 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T02:53:06", "first": "2024-09-21T02:54:00", "last": "2024-09-21T03:03:00", "last_pruned": "2024-09-21T03:03:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c40b6700 msgr2=0x7f67c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c40b6700 0x7f67c40b8c10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009fa0 tx=0x7f67cc009570 comp rx=0 tx=0).stop 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 msgr2=0x7f67dc19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc19aa70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f67d002a930 tx=0x7f67d0004270 comp rx=0 tx=0).stop 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 shutdown_connections 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f67c40b6700 0x7f67c40b8c10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67dc109d50 0x7f67dc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.145+0000 7f67e4739640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67dc109400 0x7f67dc19aa70 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.146+0000 7f67e4739640 1 --2- 172.21.15.31:0/281786525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc103ad0 0x7f67dc19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.146+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 >> 172.21.15.31:0/281786525 conn(0x7f67dc0ff510 msgr2=0x7f67dc101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:08.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.146+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 shutdown_connections 2024-09-21T03:03:08.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.146+0000 7f67e4739640 1 -- 172.21.15.31:0/281786525 wait complete. 2024-09-21T03:03:08.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 -- 172.21.15.31:0/654189297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8301082e0 msgr2=0x7fa8301086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 --2- 172.21.15.31:0/654189297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8301082e0 0x7fa8301086f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa824009900 tx=0x7fa82402a410 comp rx=0 tx=0).stop 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 -- 172.21.15.31:0/654189297 shutdown_connections 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 --2- 172.21.15.31:0/654189297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa830108c30 0x7fa83010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 --2- 172.21.15.31:0/654189297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8301082e0 0x7fa8301086f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 --2- 172.21.15.31:0/654189297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301029b0 0x7fa830102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 -- 172.21.15.31:0/654189297 >> 172.21.15.31:0/654189297 conn(0x7fa83007a740 msgr2=0x7fa83007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 -- 172.21.15.31:0/654189297 shutdown_connections 2024-09-21T03:03:08.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.272+0000 7fa836107640 1 -- 172.21.15.31:0/654189297 wait complete. 2024-09-21T03:03:08.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 Processor -- start 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 -- start start 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8301029b0 0x7fa830199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301082e0 0x7fa8301998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 0x7fa83019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301128f0 con 0x7fa830108c30 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa830112770 con 0x7fa8301082e0 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa836107640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa830112a70 con 0x7fa8301029b0 2024-09-21T03:03:08.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa82ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 0x7fa83019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa82ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 0x7fa83019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45666/0 (socket says 172.21.15.31:45666) 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.273+0000 7fa82ffff640 1 -- 172.21.15.31:0/2153057123 learned_addr learned my addr 172.21.15.31:0/2153057123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82effd640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301082e0 0x7fa8301998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8301029b0 0x7fa830199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82ffff640 1 -- 172.21.15.31:0/2153057123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8301029b0 msgr2=0x7fa830199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82ffff640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8301029b0 0x7fa830199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82ffff640 1 -- 172.21.15.31:0/2153057123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301082e0 msgr2=0x7fa8301998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82ffff640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301082e0 0x7fa8301998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82ffff640 1 -- 172.21.15.31:0/2153057123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83019f280 con 0x7fa830108c30 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82effd640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301082e0 0x7fa8301998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82ffff640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 0x7fa83019eb80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d750 tx=0x7fa82800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:08.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828017070 con 0x7fa830108c30 2024-09-21T03:03:08.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8280054d0 con 0x7fa830108c30 2024-09-21T03:03:08.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83019f570 con 0x7fa830108c30 2024-09-21T03:03:08.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828004c60 con 0x7fa830108c30 2024-09-21T03:03:08.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.274+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8301a5d00 con 0x7fa830108c30 2024-09-21T03:03:08.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.275+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa830102dc0 con 0x7fa830108c30 2024-09-21T03:03:08.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.278+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fa828009bd0 con 0x7fa830108c30 2024-09-21T03:03:08.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.278+0000 7fa82cff9640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa8180b6700 0x7fa8180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:08.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.279+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa8180b6700 0x7fa8180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:08.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.279+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fa8280e74f0 con 0x7fa830108c30 2024-09-21T03:03:08.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.279+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa8180b6700 0x7fa8180b8c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa820005c40 tx=0x7fa820005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:08.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.282+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa8280adac0 con 0x7fa830108c30 2024-09-21T03:03:08.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T03:03:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.497+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 --> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa830063800 con 0x7fa8180b6700 2024-09-21T03:03:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.505+0000 7fa82cff9640 1 -- 172.21.15.31:0/2153057123 <== mgr.14223 v2:172.21.15.31:6800/44765732 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa830063800 con 0x7fa8180b6700 2024-09-21T03:03:08.508 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa8180b6700 msgr2=0x7fa8180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa8180b6700 0x7fa8180b8c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa820005c40 tx=0x7fa820005bd0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 msgr2=0x7fa83019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 0x7fa83019eb80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d750 tx=0x7fa82800dc10 comp rx=0 tx=0).stop 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 shutdown_connections 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fa8180b6700 0x7fa8180b8c10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa830108c30 0x7fa83019eb80 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8301082e0 0x7fa8301998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 --2- 172.21.15.31:0/2153057123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8301029b0 0x7fa830199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:08.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.507+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 >> 172.21.15.31:0/2153057123 conn(0x7fa83007a740 msgr2=0x7fa830106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:08.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.508+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 shutdown_connections 2024-09-21T03:03:08.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:08.508+0000 7fa836107640 1 -- 172.21.15.31:0/2153057123 wait complete. 2024-09-21T03:03:08.524 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T03:03:08.536 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T03:03:08.585 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:09.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:08 smithi031 ceph-mon[31209]: pgmap v638: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:03:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 -- 172.21.15.31:0/3011248560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c4102a60 msgr2=0x7f64c4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 --2- 172.21.15.31:0/3011248560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c4102a60 0x7f64c4102e70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009900 tx=0x7f64c002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 -- 172.21.15.31:0/3011248560 shutdown_connections 2024-09-21T03:03:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 --2- 172.21.15.31:0/3011248560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c4108cf0 0x7f64c410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 --2- 172.21.15.31:0/3011248560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c41083a0 0x7f64c41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 --2- 172.21.15.31:0/3011248560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c4102a60 0x7f64c4102e70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 -- 172.21.15.31:0/3011248560 >> 172.21.15.31:0/3011248560 conn(0x7f64c4079730 msgr2=0x7f64c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 -- 172.21.15.31:0/3011248560 shutdown_connections 2024-09-21T03:03:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.092+0000 7f64cc13d640 1 -- 172.21.15.31:0/3011248560 wait complete. 2024-09-21T03:03:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 Processor -- start 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 -- start start 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c4102a60 0x7f64c419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c41083a0 0x7f64c419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 0x7f64c419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41129c0 con 0x7f64c4108cf0 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64c4112840 con 0x7f64c4102a60 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64cc13d640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64c4112b40 con 0x7f64c41083a0 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64ca6da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 0x7f64c419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64ca6da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 0x7f64c419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55656/0 (socket says 172.21.15.31:55656) 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64ca6da640 1 -- 172.21.15.31:0/2740123147 learned_addr learned my addr 172.21.15.31:0/2740123147 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c41083a0 0x7f64c419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.093+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c4102a60 0x7f64c419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64ca6da640 1 -- 172.21.15.31:0/2740123147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c41083a0 msgr2=0x7f64c419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64ca6da640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c41083a0 0x7f64c419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64ca6da640 1 -- 172.21.15.31:0/2740123147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c4102a60 msgr2=0x7f64c419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64ca6da640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c4102a60 0x7f64c419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64ca6da640 1 -- 172.21.15.31:0/2740123147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c41a0430 con 0x7f64c4108cf0 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64ca6da640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 0x7f64c419fd00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00eb10 tx=0x7f64bc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64baffd640 1 -- 172.21.15.31:0/2740123147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00ce20 con 0x7f64c4108cf0 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c41a0720 con 0x7f64c4108cf0 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64baffd640 1 -- 172.21.15.31:0/2740123147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc005020 con 0x7f64c4108cf0 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64baffd640 1 -- 172.21.15.31:0/2740123147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc010690 con 0x7f64c4108cf0 2024-09-21T03:03:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.094+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c41a6e80 con 0x7f64c4108cf0 2024-09-21T03:03:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 Processor -- start 2024-09-21T03:03:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64baffd640 1 -- 172.21.15.31:0/2740123147 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f64bc005510 con 0x7f64c4108cf0 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 -- start start 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 0x557c3b2e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af1c390 0x557c3b2e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557c3b25aea0 con 0x557c3af3d070 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557c3b24a2d0 con 0x557c3af141b0 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.097+0000 7f64d26b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c3b24e0e0 con 0x557c3af1c390 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 0x557c3b2e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 0x557c3b2e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55662/0 (socket says 172.21.15.31:55662) 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 -- 172.21.15.31:0/2039693258 learned_addr learned my addr 172.21.15.31:0/2039693258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af1c390 0x557c3b2e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64baffd640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64b00b6680 0x7f64b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 -- 172.21.15.31:0/2039693258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af1c390 msgr2=0x557c3b2e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af1c390 0x557c3b2e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 -- 172.21.15.31:0/2039693258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 msgr2=0x557c3b2e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 -- 172.21.15.31:0/2039693258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3afa8650 con 0x557c3af3d070 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64ca6da640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64b00b6680 0x7f64b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 0x557c3b2e46a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009f30 tx=0x7f64b4009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43fef3911e8bbae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64baffd640 1 -- 172.21.15.31:0/2740123147 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f64bc0e7d00 con 0x7f64c4108cf0 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.098+0000 7f64affff640 1 -- 172.21.15.31:0/2039693258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4017610 con 0x557c3af3d070 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.099+0000 7f64affff640 1 -- 172.21.15.31:0/2039693258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b401c070 con 0x557c3af3d070 2024-09-21T03:03:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.099+0000 7f64affff640 1 -- 172.21.15.31:0/2039693258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4015940 con 0x557c3af3d070 2024-09-21T03:03:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.099+0000 7f64d26b1740 1 -- 172.21.15.31:0/2039693258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 msgr2=0x557c3b2e46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.099+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 0x557c3b2e46a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009f30 tx=0x7f64b4009570 comp rx=0 tx=0).stop 2024-09-21T03:03:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.099+0000 7f64ca6da640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64b00b6680 0x7f64b00b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0aadc0 tx=0x7f64bc07ae30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 -- 172.21.15.31:0/2039693258 shutdown_connections 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af1c390 0x557c3b2e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2039693258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af3d070 0x557c3b2e46a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 -- 172.21.15.31:0/2039693258 >> 172.21.15.31:0/2039693258 conn(0x557c3af31460 msgr2=0x557c3b2e2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 -- 172.21.15.31:0/2039693258 shutdown_connections 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 -- 172.21.15.31:0/2039693258 wait complete. 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 -- start start 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.100+0000 7f64d26b1740 1 Processor -- start 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2fc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557c3b0ab700 con 0x557c3af1c390 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557c3b1e5160 con 0x557c3af141b0 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557c3b2f2e20 con 0x557c3af3d070 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64d26b1740 10 client.0 _ll_register_callbacks cb 0x557c3b1fe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2f02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55668/0 (socket says 172.21.15.31:55668) 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 -- 172.21.15.31:0/1743010528 learned_addr learned my addr 172.21.15.31:0/1743010528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c96d8640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c96d8640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 -- 172.21.15.31:0/1743010528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 msgr2=0x557c3b2fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2fc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 -- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 msgr2=0x557c3b2efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 -- 172.21.15.31:0/1743010528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3b212c90 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c96d8640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2f02d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f64c002a930 tx=0x7f64c00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29e506f44ef6534 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.101+0000 7f64ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64ad7fa640 1 -- 172.21.15.31:0/1743010528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0035040 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b401c070 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64ad7fa640 1 -- 172.21.15.31:0/1743010528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f64c0004040 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64ad7fa640 1 -- 172.21.15.31:0/1743010528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64c0004240 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 10 client.17877 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c3b2feb60 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64ad7fa640 1 -- 172.21.15.31:0/1743010528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b401c070 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64ad7fa640 1 -- 172.21.15.31:0/1743010528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f64c00048e0 con 0x557c3af1c390 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 10 client.17877 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 10 client.17877 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:03:09.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 20 client.17877 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aefb9f0 0x557c3b302010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 4 client.17877 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.102+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c3b25cdf0 con 0x557c3aefb9f0 2024-09-21T03:03:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.103+0000 7f64c96d8640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aefb9f0 0x557c3b302010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.103+0000 7f64c96d8640 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aefb9f0 0x557c3b302010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a1f903393eca45 server_cookie=960b91fccb0f0f96 in_seq=0 out_seq=0 2024-09-21T03:03:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.103+0000 7f64ad7fa640 10 client.17877 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:03:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64ad7fa640 1 -- 172.21.15.31:0/1743010528 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c3b25cdf0 con 0x557c3aefb9f0 2024-09-21T03:03:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64ad7fa640 10 client.17877 handle_command_reply: tid=0 2024-09-21T03:03:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 client.17877 shutdown 2024-09-21T03:03:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 msgr2=0x557c3b2f02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2f02d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f64c002a930 tx=0x7f64c00316d0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 shutdown_connections 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aefb9f0 0x557c3b302010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2f02d0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 --2- 172.21.15.31:0/1743010528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af141b0 0x557c3b2efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 >> 172.21.15.31:0/1743010528 conn(0x557c3af31460 msgr2=0x557c3b2e7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64ad7fa640 0 client.17877 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 shutdown_connections 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.104+0000 7f64d26b1740 1 -- 172.21.15.31:0/1743010528 wait complete. 2024-09-21T03:03:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.105+0000 7f64d26b1740 20 client.17877 trim_cache size 0 max 16384 2024-09-21T03:03:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 Processor -- start 2024-09-21T03:03:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 -- start start 2024-09-21T03:03:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af141b0 0x557c3b2f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af3d070 0x557c3b2e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41af780 con 0x557c3af1c390 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41b12c0 con 0x557c3af3d070 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64d26b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41afe50 con 0x557c3af141b0 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64c9ed9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64c9ed9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2e5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55680/0 (socket says 172.21.15.31:55680) 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.118+0000 7f64c9ed9640 1 -- 172.21.15.31:0/2287377298 learned_addr learned my addr 172.21.15.31:0/2287377298 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af141b0 0x557c3b2f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af3d070 0x557c3b2e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c9ed9640 1 -- 172.21.15.31:0/2287377298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af141b0 msgr2=0x557c3b2f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af141b0 0x557c3b2f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c9ed9640 1 -- 172.21.15.31:0/2287377298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af3d070 msgr2=0x557c3b2e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af3d070 0x557c3b2e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c9ed9640 1 -- 172.21.15.31:0/2287377298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3b2e6990 con 0x557c3af1c390 2024-09-21T03:03:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c96d8640 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af3d070 0x557c3b2e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2e5de0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005f70 tx=0x7f64c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9986b0464d2570e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64af7fe640 1 -- 172.21.15.31:0/2287377298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0035070 con 0x557c3af1c390 2024-09-21T03:03:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64af7fe640 1 -- 172.21.15.31:0/2287377298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c0031740 con 0x557c3af1c390 2024-09-21T03:03:09.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.119+0000 7f64af7fe640 1 -- 172.21.15.31:0/2287377298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0005570 con 0x557c3af1c390 2024-09-21T03:03:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 -- 172.21.15.31:0/2287377298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 msgr2=0x557c3b2e5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2e5de0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005f70 tx=0x7f64c002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 -- 172.21.15.31:0/2287377298 shutdown_connections 2024-09-21T03:03:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af3d070 0x557c3b2e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af1c390 0x557c3b2e5de0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 --2- 172.21.15.31:0/2287377298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af141b0 0x557c3b2f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 -- 172.21.15.31:0/2287377298 >> 172.21.15.31:0/2287377298 conn(0x557c3af31460 msgr2=0x557c3b2e56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 -- 172.21.15.31:0/2287377298 shutdown_connections 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 -- 172.21.15.31:0/2287377298 wait complete. 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 -- start start 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.120+0000 7f64d26b1740 1 Processor -- start 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 0x557c3b2f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af1c390 0x557c3b2f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41b0bf0 con 0x557c3af141b0 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41b0520 con 0x557c3af1c390 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64c41ad760 con 0x557c3af3d070 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64d26b1740 10 client.0 _ll_register_callbacks cb 0x7f64c0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 0x557c3b2f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 0x557c3b2f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55686/0 (socket says 172.21.15.31:55686) 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 -- 172.21.15.31:0/3561429880 learned_addr learned my addr 172.21.15.31:0/3561429880 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af1c390 0x557c3b2f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 -- 172.21.15.31:0/3561429880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 msgr2=0x557c3b2f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 -- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af1c390 msgr2=0x557c3b2f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af1c390 0x557c3b2f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.121+0000 7f64c96d8640 1 -- 172.21.15.31:0/3561429880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3b2f4990 con 0x557c3af141b0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64c96d8640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 0x557c3b2f8830 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f64b40155f0 tx=0x7f64b40153e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabcce639834a83d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64ad7fa640 1 -- 172.21.15.31:0/3561429880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4009180 con 0x557c3af141b0 2024-09-21T03:03:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64ad7fa640 1 -- 172.21.15.31:0/3561429880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7f64b4015950 con 0x557c3af141b0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c0031740 con 0x557c3af141b0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64ad7fa640 1 -- 172.21.15.31:0/3561429880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64b4015b50 con 0x557c3af141b0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64d26b1740 10 client.17883 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c3b36c8c0 con 0x557c3af141b0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.122+0000 7f64ad7fa640 1 -- 172.21.15.31:0/3561429880 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c0031740 con 0x557c3af141b0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64ad7fa640 1 -- 172.21.15.31:0/3561429880 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f64b4021400 con 0x557c3af141b0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64d26b1740 10 client.17883 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64d26b1740 10 client.17883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64d26b1740 20 client.17883 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64d26b1740 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aed0da0 0x557c3b36eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64d26b1740 4 client.17883 mds_command: new command op to 24659 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 --> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c3b2588a0 con 0x557c3aed0da0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aed0da0 0x557c3b36eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64ad7fa640 10 client.17883 ms_handle_connect on v2:172.21.15.165:6836/3116204029 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64c9ed9640 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aed0da0 0x557c3b36eda0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a8478cef41f42e0 server_cookie=35a29514b3499163 in_seq=0 out_seq=0 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.123+0000 7f64ad7fa640 1 -- 172.21.15.31:0/3561429880 <== mds.1 v2:172.21.15.165:6836/3116204029 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c3b2588a0 con 0x557c3aed0da0 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64ad7fa640 10 client.17883 handle_command_reply: tid=0 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 client.17883 shutdown 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 msgr2=0x557c3b2f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 0x557c3b2f8830 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f64b40155f0 tx=0x7f64b40153e0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 shutdown_connections 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:6836/3116204029,v1:172.21.15.165:6837/3116204029] conn(0x557c3aed0da0 0x557c3b36eda0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557c3af3d070 0x557c3b2f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c3af1c390 0x557c3b2f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 --2- 172.21.15.31:0/3561429880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c3af141b0 0x557c3b2f8830 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64ad7fa640 0 client.17883 ms_handle_reset on v2:172.21.15.165:6836/3116204029 2024-09-21T03:03:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 >> 172.21.15.31:0/3561429880 conn(0x557c3af31460 msgr2=0x557c3b2e5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 shutdown_connections 2024-09-21T03:03:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 1 -- 172.21.15.31:0/3561429880 wait complete. 2024-09-21T03:03:09.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.124+0000 7f64d26b1740 20 client.17883 trim_cache size 0 max 16384 2024-09-21T03:03:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64b00b6680 msgr2=0x7f64b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64b00b6680 0x7f64b00b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0aadc0 tx=0x7f64bc07ae30 comp rx=0 tx=0).stop 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 msgr2=0x7f64c419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 0x7f64c419fd00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00eb10 tx=0x7f64bc00ef10 comp rx=0 tx=0).stop 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 shutdown_connections 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f64b00b6680 0x7f64b00b8b90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64c41083a0 0x7f64c419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c4108cf0 0x7f64c419fd00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 --2- 172.21.15.31:0/2740123147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64c4102a60 0x7f64c419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 >> 172.21.15.31:0/2740123147 conn(0x7f64c4079730 msgr2=0x7f64c4101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:09.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 shutdown_connections 2024-09-21T03:03:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:09.125+0000 7f64cc13d640 1 -- 172.21.15.31:0/2740123147 wait complete. 2024-09-21T03:03:09.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:08 smithi165 ceph-mon[34818]: pgmap v638: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:03:09.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:08 smithi178 ceph-mon[34759]: pgmap v638: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-21T03:03:09.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:03:10.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:09 smithi031 ceph-mon[31209]: from='client.17862 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:03:10.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:09 smithi031 ceph-mon[31209]: from='client.17868 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:03:10.136 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:10.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:09 smithi165 ceph-mon[34818]: from='client.17862 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:03:10.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:09 smithi165 ceph-mon[34818]: from='client.17868 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:03:10.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:09 smithi178 ceph-mon[34759]: from='client.17862 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:03:10.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:09 smithi178 ceph-mon[34759]: from='client.17868 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:03:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 -- 172.21.15.31:0/634002066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee401083c0 msgr2=0x7fee401087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 --2- 172.21.15.31:0/634002066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee401083c0 0x7fee401087d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fee28009900 tx=0x7fee2802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 -- 172.21.15.31:0/634002066 shutdown_connections 2024-09-21T03:03:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 --2- 172.21.15.31:0/634002066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee40108d10 0x7fee4010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 --2- 172.21.15.31:0/634002066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee401083c0 0x7fee401087d0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 --2- 172.21.15.31:0/634002066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40102a80 0x7fee40102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 -- 172.21.15.31:0/634002066 >> 172.21.15.31:0/634002066 conn(0x7fee40079740 msgr2=0x7fee40079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 -- 172.21.15.31:0/634002066 shutdown_connections 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 -- 172.21.15.31:0/634002066 wait complete. 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.654+0000 7fee456a5640 1 Processor -- start 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 -- start start 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 0x7fee400ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee401083c0 0x7fee400ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40108d10 0x7fee40100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112980 con 0x7fee40102a80 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112800 con 0x7fee401083c0 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee456a5640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112b00 con 0x7fee40108d10 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 0x7fee400ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 0x7fee400ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55718/0 (socket says 172.21.15.31:55718) 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 -- 172.21.15.31:0/2190056136 learned_addr learned my addr 172.21.15.31:0/2190056136 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee367fc640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee401083c0 0x7fee400ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40108d10 0x7fee40100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 -- 172.21.15.31:0/2190056136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40108d10 msgr2=0x7fee40100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40108d10 0x7fee40100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 -- 172.21.15.31:0/2190056136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee401083c0 msgr2=0x7fee400ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee401083c0 0x7fee400ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3effd640 1 -- 172.21.15.31:0/2190056136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee401a8740 con 0x7fee40102a80 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.655+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40108d10 0x7fee40100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.656+0000 7fee3effd640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 0x7fee400ff690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fee3000d770 tx=0x7fee3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.656+0000 7fee3cff9640 1 -- 172.21.15.31:0/2190056136 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30017070 con 0x7fee40102a80 2024-09-21T03:03:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.656+0000 7fee3cff9640 1 -- 172.21.15.31:0/2190056136 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee300054d0 con 0x7fee40102a80 2024-09-21T03:03:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.656+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee401a8a30 con 0x7fee40102a80 2024-09-21T03:03:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.656+0000 7fee3cff9640 1 -- 172.21.15.31:0/2190056136 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30004c30 con 0x7fee40102a80 2024-09-21T03:03:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.656+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a8f40 con 0x7fee40102a80 2024-09-21T03:03:10.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.658+0000 7fee4bc19740 1 Processor -- start 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.658+0000 7fee4bc19740 1 -- start start 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.658+0000 7fee3cff9640 1 -- 172.21.15.31:0/2190056136 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fee30004e10 con 0x7fee40102a80 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597de14070 0x55597e1bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee4bc19740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55597e131ea0 con 0x55597ddf3390 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee4bc19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55597e1212d0 con 0x55597ddeb1b0 2024-09-21T03:03:10.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee4bc19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55597e1250e0 con 0x55597de14070 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee367fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee367fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1bc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55726/0 (socket says 172.21.15.31:55726) 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee367fc640 1 -- 172.21.15.31:0/118201311 learned_addr learned my addr 172.21.15.31:0/118201311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597de14070 0x55597e1bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.659+0000 7fee3effd640 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee367fc640 1 -- 172.21.15.31:0/118201311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597de14070 msgr2=0x55597e1bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee367fc640 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597de14070 0x55597e1bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee367fc640 1 -- 172.21.15.31:0/118201311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 msgr2=0x55597e1bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee367fc640 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1bbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee367fc640 1 -- 172.21.15.31:0/118201311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55597de7f650 con 0x55597ddf3390 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee3effd640 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee3cff9640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fee2c0b6700 0x7fee2c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee367fc640 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1bc120 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fee28009900 tx=0x7fee28032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a2f6b7f35f4228 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fee2c0b6700 0x7fee2c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee3cff9640 1 -- 172.21.15.31:0/2190056136 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fee300e7250 con 0x7fee40102a80 2024-09-21T03:03:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee357fa640 1 -- 172.21.15.31:0/118201311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2803f070 con 0x55597ddf3390 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee357fa640 1 -- 172.21.15.31:0/118201311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee28004030 con 0x55597ddf3390 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.660+0000 7fee357fa640 1 -- 172.21.15.31:0/118201311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28035070 con 0x55597ddf3390 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 -- 172.21.15.31:0/118201311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 msgr2=0x55597e1bc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fee2c0b6700 0x7fee2c0b8c10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fee38005fd0 tx=0x7fee38009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1bc120 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fee28009900 tx=0x7fee28032ba0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 -- 172.21.15.31:0/118201311 shutdown_connections 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1bc120 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 --2- 172.21.15.31:0/118201311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597de14070 0x55597e1bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 -- 172.21.15.31:0/118201311 >> 172.21.15.31:0/118201311 conn(0x55597de08460 msgr2=0x55597e1b9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 -- 172.21.15.31:0/118201311 shutdown_connections 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.661+0000 7fee4bc19740 1 -- 172.21.15.31:0/118201311 wait complete. 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 -- start start 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 Processor -- start 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddf3390 0x55597e1c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 0x55597e1d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55597df82700 con 0x55597de14070 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55597e0bc160 con 0x55597ddeb1b0 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55597e1c9e20 con 0x55597ddf3390 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee4bc19740 10 client.0 _ll_register_callbacks cb 0x55597e0d59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 0x55597e1d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 0x55597e1d3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55738/0 (socket says 172.21.15.31:55738) 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.662+0000 7fee3f7fe640 1 -- 172.21.15.31:0/318683121 learned_addr learned my addr 172.21.15.31:0/318683121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee367fc640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddf3390 0x55597e1c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3effd640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3f7fe640 1 -- 172.21.15.31:0/318683121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddf3390 msgr2=0x55597e1c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddf3390 0x55597e1c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3f7fe640 1 -- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 msgr2=0x55597e1c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3f7fe640 1 -- 172.21.15.31:0/318683121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55597e0e9c90 con 0x55597de14070 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee367fc640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddf3390 0x55597e1c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 0x55597e1d3140 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fee380109e0 tx=0x7fee38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefafa54c2a8feb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee267fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee267fc640 1 -- 172.21.15.31:0/318683121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3801d8f0 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee267fc640 1 -- 172.21.15.31:0/318683121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fee3801da90 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee28004030 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee267fc640 1 -- 172.21.15.31:0/318683121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee3801dc90 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee4bc19740 10 client.17895 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.663+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55597e1d5bc0 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee267fc640 1 -- 172.21.15.31:0/318683121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee28004030 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee267fc640 1 -- 172.21.15.31:0/318683121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fee380294a0 con 0x55597de14070 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee4bc19740 10 client.17895 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee4bc19740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee4bc19740 20 client.17895 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee4bc19740 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597ddd29f0 0x55597e1d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee4bc19740 4 client.17895 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55597e133df0 con 0x55597ddd29f0 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.664+0000 7fee3effd640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597ddd29f0 0x55597e1d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.665+0000 7fee3effd640 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597ddd29f0 0x55597e1d9030 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0b2860d0e6c13d server_cookie=2f62008cd89e0a5 in_seq=0 out_seq=0 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.665+0000 7fee267fc640 10 client.17895 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee267fc640 1 -- 172.21.15.31:0/318683121 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55597e133df0 con 0x55597ddd29f0 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee267fc640 10 client.17895 handle_command_reply: tid=0 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 client.17895 shutdown 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 msgr2=0x55597e1d3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 0x55597e1d3140 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fee380109e0 tx=0x7fee38004a40 comp rx=0 tx=0).stop 2024-09-21T03:03:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 shutdown_connections 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597ddd29f0 0x55597e1d9030 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddf3390 0x55597e1c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee267fc640 0 client.17895 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597de14070 0x55597e1d3140 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 --2- 172.21.15.31:0/318683121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 >> 172.21.15.31:0/318683121 conn(0x55597de08460 msgr2=0x55597e1be830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 shutdown_connections 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 1 -- 172.21.15.31:0/318683121 wait complete. 2024-09-21T03:03:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.666+0000 7fee4bc19740 20 client.17895 trim_cache size 0 max 16384 2024-09-21T03:03:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.679+0000 7fee4bc19740 1 Processor -- start 2024-09-21T03:03:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.679+0000 7fee4bc19740 1 -- start start 2024-09-21T03:03:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee4bc19740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b1910 con 0x55597ddf3390 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee4bc19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b3450 con 0x55597ddeb1b0 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee4bc19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b1fe0 con 0x55597ddd29f0 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55746/0 (socket says 172.21.15.31:55746) 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 -- 172.21.15.31:0/3118770779 learned_addr learned my addr 172.21.15.31:0/3118770779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee367fc640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3effd640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 -- 172.21.15.31:0/3118770779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 msgr2=0x55597e1c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 -- 172.21.15.31:0/3118770779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 msgr2=0x55597e1d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3f7fe640 1 -- 172.21.15.31:0/3118770779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55597e1c9930 con 0x55597ddf3390 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee367fc640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.680+0000 7fee3effd640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.681+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1c3f50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fee380109b0 tx=0x7fee38010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aec936377f1d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.681+0000 7fee34ff9640 1 -- 172.21.15.31:0/3118770779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38029480 con 0x55597ddf3390 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.681+0000 7fee34ff9640 1 -- 172.21.15.31:0/3118770779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee38029620 con 0x55597ddf3390 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.681+0000 7fee34ff9640 1 -- 172.21.15.31:0/3118770779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38004020 con 0x55597ddf3390 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 -- 172.21.15.31:0/3118770779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 msgr2=0x55597e1c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1c3f50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fee380109b0 tx=0x7fee38010f20 comp rx=0 tx=0).stop 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 -- 172.21.15.31:0/3118770779 shutdown_connections 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e1c3f50 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 --2- 172.21.15.31:0/3118770779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 -- 172.21.15.31:0/3118770779 >> 172.21.15.31:0/3118770779 conn(0x55597de08460 msgr2=0x55597e1c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 -- 172.21.15.31:0/3118770779 shutdown_connections 2024-09-21T03:03:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 -- 172.21.15.31:0/3118770779 wait complete. 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 -- start start 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.682+0000 7fee4bc19740 1 Processor -- start 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e242a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b2d80 con 0x55597ddf3390 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee401b26b0 con 0x55597ddeb1b0 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee401af8f0 con 0x55597ddd29f0 2024-09-21T03:03:10.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee4bc19740 10 client.0 _ll_register_callbacks cb 0x7fee38009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:10.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e242a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e242a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55760/0 (socket says 172.21.15.31:55760) 2024-09-21T03:03:10.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 -- 172.21.15.31:0/63377639 learned_addr learned my addr 172.21.15.31:0/63377639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:10.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3effd640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee367fc640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 -- 172.21.15.31:0/63377639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 msgr2=0x55597e1cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 -- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 msgr2=0x55597e1cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 -- 172.21.15.31:0/63377639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55597e242f70 con 0x55597ddf3390 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee367fc640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee3f7fe640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e242a30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fee38017040 tx=0x7fee38015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a572a6897f1508 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.683+0000 7fee267fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee267fc640 1 -- 172.21.15.31:0/63377639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38004a00 con 0x55597ddf3390 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee267fc640 1 -- 172.21.15.31:0/63377639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8528+0+0 (secure 0 0 0) 0x7fee38004ba0 con 0x55597ddf3390 2024-09-21T03:03:10.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee38029620 con 0x55597ddf3390 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee267fc640 1 -- 172.21.15.31:0/63377639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee38004da0 con 0x55597ddf3390 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 10 client.17901 fetch_fsmap learned FSMap version 27 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55597e2440c0 con 0x55597ddf3390 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee267fc640 1 -- 172.21.15.31:0/63377639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee38029620 con 0x55597ddf3390 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee267fc640 1 -- 172.21.15.31:0/63377639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fee38032480 con 0x55597ddf3390 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 10 client.17901 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 20 client.17901 populate_metadata read hostname 'smithi031' 2024-09-21T03:03:10.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597de14070 0x55597e246730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 4 client.17901 mds_command: new command op to 24695 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 --> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55597e12f8a0 con 0x55597de14070 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.684+0000 7fee3effd640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597de14070 0x55597e246730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee3effd640 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597de14070 0x55597e246730 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b0a525d592e74e server_cookie=a8d30b844be89a84 in_seq=0 out_seq=0 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee267fc640 10 client.17901 ms_handle_connect on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee267fc640 1 -- 172.21.15.31:0/63377639 <== mds.0 v2:172.21.15.165:6838/2689629692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55597e12f8a0 con 0x55597de14070 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee267fc640 10 client.17901 handle_command_reply: tid=0 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 client.17901 shutdown 2024-09-21T03:03:10.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 msgr2=0x55597e242a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e242a30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fee38017040 tx=0x7fee38015000 comp rx=0 tx=0).stop 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 shutdown_connections 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:6838/2689629692,v1:172.21.15.165:6839/2689629692] conn(0x55597de14070 0x55597e246730 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55597ddf3390 0x55597e242a30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee267fc640 0 client.17901 ms_handle_reset on v2:172.21.15.165:6838/2689629692 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55597ddeb1b0 0x55597e1cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 --2- 172.21.15.31:0/63377639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55597ddd29f0 0x55597e1cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 >> 172.21.15.31:0/63377639 conn(0x55597de08460 msgr2=0x55597e1be7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.685+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 shutdown_connections 2024-09-21T03:03:10.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee4bc19740 1 -- 172.21.15.31:0/63377639 wait complete. 2024-09-21T03:03:10.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee4bc19740 20 client.17901 trim_cache size 0 max 16384 2024-09-21T03:03:10.700 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T03:03:10.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fee2c0b6700 msgr2=0x7fee2c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fee2c0b6700 0x7fee2c0b8c10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fee38005fd0 tx=0x7fee38009500 comp rx=0 tx=0).stop 2024-09-21T03:03:10.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 msgr2=0x7fee400ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 0x7fee400ff690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fee3000d770 tx=0x7fee3000dc30 comp rx=0 tx=0).stop 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 shutdown_connections 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fee2c0b6700 0x7fee2c0b8c10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee40108d10 0x7fee40100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee401083c0 0x7fee400ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 --2- 172.21.15.31:0/2190056136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee40102a80 0x7fee400ff690 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 >> 172.21.15.31:0/2190056136 conn(0x7fee40079740 msgr2=0x7fee401019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 shutdown_connections 2024-09-21T03:03:10.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:10.686+0000 7fee456a5640 1 -- 172.21.15.31:0/2190056136 wait complete. 2024-09-21T03:03:10.834 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:10 smithi031 ceph-mon[31209]: pgmap v639: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:03:11.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:10 smithi165 ceph-mon[34818]: pgmap v639: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:03:11.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:10 smithi178 ceph-mon[34759]: pgmap v639: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T03:03:11.516 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T03:03:11.517 INFO:tasks.fwd_scrub:done joining 2024-09-21T03:03:11.517 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:03:11.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:03:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 -- 172.21.15.31:0/298402230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 msgr2=0x7f69f0104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 --2- 172.21.15.31:0/298402230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0104d10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009900 tx=0x7f69ec02a410 comp rx=0 tx=0).stop 2024-09-21T03:03:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 -- 172.21.15.31:0/298402230 shutdown_connections 2024-09-21T03:03:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 --2- 172.21.15.31:0/298402230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f0105250 0x7f69f010d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 --2- 172.21.15.31:0/298402230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f010dd60 0x7f69f0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 --2- 172.21.15.31:0/298402230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0104d10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.644+0000 7f69f737b640 1 -- 172.21.15.31:0/298402230 >> 172.21.15.31:0/298402230 conn(0x7f69f00fe3b0 msgr2=0x7f69f0100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.645+0000 7f69f737b640 1 -- 172.21.15.31:0/298402230 shutdown_connections 2024-09-21T03:03:11.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.645+0000 7f69f737b640 1 -- 172.21.15.31:0/298402230 wait complete. 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.645+0000 7f69f737b640 1 Processor -- start 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.645+0000 7f69f737b640 1 -- start start 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f737b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f737b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f0105250 0x7f69f0102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f737b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f010dd60 0x7f69f01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f737b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01139c0 con 0x7f69f0104900 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f737b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0113840 con 0x7f69f0105250 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f737b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0113b40 con 0x7f69f010dd60 2024-09-21T03:03:11.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0102630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55766/0 (socket says 172.21.15.31:55766) 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 -- 172.21.15.31:0/3407655374 learned_addr learned my addr 172.21.15.31:0/3407655374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f4916640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f0105250 0x7f69f0102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5918640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f010dd60 0x7f69f01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 -- 172.21.15.31:0/3407655374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f010dd60 msgr2=0x7f69f01030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f010dd60 0x7f69f01030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 -- 172.21.15.31:0/3407655374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f0105250 msgr2=0x7f69f0102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f0105250 0x7f69f0102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 -- 172.21.15.31:0/3407655374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f01a8710 con 0x7f69f0104900 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5918640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f010dd60 0x7f69f01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69f5117640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0102630 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005940 tx=0x7f69ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.646+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec03f030 con 0x7f69f0104900 2024-09-21T03:03:11.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.647+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01a89a0 con 0x7f69f0104900 2024-09-21T03:03:11.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.647+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ec003cb0 con 0x7f69f0104900 2024-09-21T03:03:11.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.647+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec035070 con 0x7f69f0104900 2024-09-21T03:03:11.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.647+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f01a8ee0 con 0x7f69f0104900 2024-09-21T03:03:11.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.647+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f0104d10 con 0x7f69f0104900 2024-09-21T03:03:11.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.649+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f69ec042030 con 0x7f69f0104900 2024-09-21T03:03:11.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.650+0000 7f69e67fc640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f69dc0b66b0 0x7f69dc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.650+0000 7f69f4916640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f69dc0b66b0 0x7f69dc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.650+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8572+0+0 (secure 0 0 0) 0x7f69ec107f00 con 0x7f69f0104900 2024-09-21T03:03:11.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.650+0000 7f69f4916640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f69dc0b66b0 0x7f69dc0b8bc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f69f0100b50 tx=0x7f69e000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:11.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.653+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69ec0ce420 con 0x7f69f0104900 2024-09-21T03:03:11.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.846+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69f01a9250 con 0x7f69f0104900 2024-09-21T03:03:11.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.847+0000 7f69e67fc640 1 -- 172.21.15.31:0/3407655374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23302 (secure 0 0 0) 0x7f69ec0d2800 con 0x7f69f0104900 2024-09-21T03:03:11.850 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:03:11.850 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T03:03:10.794448+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":85,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f69dc0b66b0 msgr2=0x7f69dc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f69dc0b66b0 0x7f69dc0b8bc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f69f0100b50 tx=0x7f69e000b410 comp rx=0 tx=0).stop 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 msgr2=0x7f69f0102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0102630 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005940 tx=0x7f69ec0042f0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 shutdown_connections 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f69dc0b66b0 0x7f69dc0b8bc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f010dd60 0x7f69f01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69f0105250 0x7f69f0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 --2- 172.21.15.31:0/3407655374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f0104900 0x7f69f0102630 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.849+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 >> 172.21.15.31:0/3407655374 conn(0x7f69f00fe3b0 msgr2=0x7f69f01078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.850+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 shutdown_connections 2024-09-21T03:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.850+0000 7f69f737b640 1 -- 172.21.15.31:0/3407655374 wait complete. 2024-09-21T03:03:11.864 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:46:13.535825+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:50:47.175448+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:50:48.994282+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 85, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:03:11.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 -- 172.21.15.31:0/2303898800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff858104900 msgr2=0x7ff858104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/2303898800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff858104900 0x7ff858104d10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff848009900 tx=0x7ff84802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 -- 172.21.15.31:0/2303898800 shutdown_connections 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/2303898800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858105250 0x7ff85810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/2303898800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff85810dd60 0x7ff858110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/2303898800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff858104900 0x7ff858104d10 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 -- 172.21.15.31:0/2303898800 >> 172.21.15.31:0/2303898800 conn(0x7ff8580fe3f0 msgr2=0x7ff858100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 -- 172.21.15.31:0/2303898800 shutdown_connections 2024-09-21T03:03:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.979+0000 7ff85dfa1640 1 -- 172.21.15.31:0/2303898800 wait complete. 2024-09-21T03:03:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 Processor -- start 2024-09-21T03:03:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 -- start start 2024-09-21T03:03:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858104900 0x7ff858102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff858105250 0x7ff858102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 0x7ff8581030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581139e0 con 0x7ff85810dd60 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff858113860 con 0x7ff858105250 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff85dfa1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff858113b60 con 0x7ff858104900 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff857fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 0x7ff8581030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff857fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 0x7ff8581030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55786/0 (socket says 172.21.15.31:55786) 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff857fff640 1 -- 172.21.15.31:0/3794800914 learned_addr learned my addr 172.21.15.31:0/3794800914 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff8577fe640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858104900 0x7ff858102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.980+0000 7ff84effd640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff858105250 0x7ff858102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff857fff640 1 -- 172.21.15.31:0/3794800914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858104900 msgr2=0x7ff858102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff857fff640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858104900 0x7ff858102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff857fff640 1 -- 172.21.15.31:0/3794800914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff858105250 msgr2=0x7ff858102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff857fff640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff858105250 0x7ff858102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff857fff640 1 -- 172.21.15.31:0/3794800914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8581a86b0 con 0x7ff85810dd60 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff84effd640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff858105250 0x7ff858102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff857fff640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 0x7ff8581030b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff85000eaa0 tx=0x7ff85000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85000cdb0 con 0x7ff85810dd60 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff850005020 con 0x7ff85810dd60 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8581a89a0 con 0x7ff85810dd60 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500116d0 con 0x7ff85810dd60 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.981+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8581a8ef0 con 0x7ff85810dd60 2024-09-21T03:03:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.982+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85806bfe0 con 0x7ff85810dd60 2024-09-21T03:03:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.984+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7ff8500051d0 con 0x7ff85810dd60 2024-09-21T03:03:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.985+0000 7ff8557fa640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff8440b6730 0x7ff8440b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.985+0000 7ff8577fe640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff8440b6730 0x7ff8440b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.985+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8572+0+0 (secure 0 0 0) 0x7ff8500e7c70 con 0x7ff85810dd60 2024-09-21T03:03:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.985+0000 7ff8577fe640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff8440b6730 0x7ff8440b8c40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff848000c00 tx=0x7ff848031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:11.987+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff850016030 con 0x7ff85810dd60 2024-09-21T03:03:12.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:11 smithi031 ceph-mon[31209]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:03:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.175+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff858104d10 con 0x7ff85810dd60 2024-09-21T03:03:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.176+0000 7ff8557fa640 1 -- 172.21.15.31:0/3794800914 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7ff8500ae210 con 0x7ff85810dd60 2024-09-21T03:03:12.178 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-21T03:03:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff8440b6730 msgr2=0x7ff8440b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff8440b6730 0x7ff8440b8c40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff848000c00 tx=0x7ff848031400 comp rx=0 tx=0).stop 2024-09-21T03:03:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 msgr2=0x7ff8581030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 0x7ff8581030b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff85000eaa0 tx=0x7ff85000ef60 comp rx=0 tx=0).stop 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 shutdown_connections 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7ff8440b6730 0x7ff8440b8c40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff85810dd60 0x7ff8581030b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff858105250 0x7ff858102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 --2- 172.21.15.31:0/3794800914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858104900 0x7ff858102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 >> 172.21.15.31:0/3794800914 conn(0x7ff8580fe3f0 msgr2=0x7ff858107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 shutdown_connections 2024-09-21T03:03:12.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.178+0000 7ff85dfa1640 1 -- 172.21.15.31:0/3794800914 wait complete. 2024-09-21T03:03:12.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:11 smithi165 ceph-mon[34818]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:03:12.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:03:12.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:11 smithi178 ceph-mon[34759]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 -- 172.21.15.31:0/2333345421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 msgr2=0x7f4e70102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 --2- 172.21.15.31:0/2333345421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e70102e30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e6002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 -- 172.21.15.31:0/2333345421 shutdown_connections 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 --2- 172.21.15.31:0/2333345421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e70108c50 0x7f4e7010f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 --2- 172.21.15.31:0/2333345421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70108300 0x7f4e70108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 --2- 172.21.15.31:0/2333345421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e70102e30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 -- 172.21.15.31:0/2333345421 >> 172.21.15.31:0/2333345421 conn(0x7f4e70079630 msgr2=0x7f4e70079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:12.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 -- 172.21.15.31:0/2333345421 shutdown_connections 2024-09-21T03:03:12.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.305+0000 7f4e751ed640 1 -- 172.21.15.31:0/2333345421 wait complete. 2024-09-21T03:03:12.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.306+0000 7f4e751ed640 1 Processor -- start 2024-09-21T03:03:12.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.306+0000 7f4e751ed640 1 -- start start 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.306+0000 7f4e751ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e701990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.306+0000 7f4e751ed640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e70108300 0x7f4e701995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e751ed640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70108c50 0x7f4e70199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e751ed640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112910 con 0x7f4e70102a20 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e751ed640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112790 con 0x7f4e70108c50 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e751ed640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112a90 con 0x7f4e70108300 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e701990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e701990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55798/0 (socket says 172.21.15.31:55798) 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 -- 172.21.15.31:0/30205077 learned_addr learned my addr 172.21.15.31:0/30205077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6e59c640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e70108300 0x7f4e701995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6f59e640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70108c50 0x7f4e70199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 -- 172.21.15.31:0/30205077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e70108300 msgr2=0x7f4e701995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e70108300 0x7f4e701995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 -- 172.21.15.31:0/30205077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70108c50 msgr2=0x7f4e70199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70108c50 0x7f4e70199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 -- 172.21.15.31:0/30205077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e701a8760 con 0x7f4e70102a20 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e6ed9d640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e701990b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005940 tx=0x7f4e600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6003f030 con 0x7f4e70102a20 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e60003cb0 con 0x7f4e70102a20 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e701a89f0 con 0x7f4e70102a20 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.307+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60035070 con 0x7f4e70102a20 2024-09-21T03:03:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.308+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a8ee0 con 0x7f4e70102a20 2024-09-21T03:03:12.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.309+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e70102e30 con 0x7f4e70102a20 2024-09-21T03:03:12.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.311+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f4e60042030 con 0x7f4e70102a20 2024-09-21T03:03:12.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.312+0000 7f4e67fff640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.312+0000 7f4e6e59c640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.312+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8572+0+0 (secure 0 0 0) 0x7f4e60108240 con 0x7f4e70102a20 2024-09-21T03:03:12.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.313+0000 7f4e6e59c640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e70107ef0 tx=0x7f4e5800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:12.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.316+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e600d2b40 con 0x7f4e70102a20 2024-09-21T03:03:12.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.506+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e7010a490 con 0x7f4e70102a20 2024-09-21T03:03:12.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.506+0000 7f4e67fff640 1 -- 172.21.15.31:0/30205077 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f4e600d7a80 con 0x7f4e70102a20 2024-09-21T03:03:12.509 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-21T03:03:12.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4e5c0b6680 msgr2=0x7f4e5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4e70107ef0 tx=0x7f4e5800b390 comp rx=0 tx=0).stop 2024-09-21T03:03:12.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 msgr2=0x7f4e701990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e701990b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005940 tx=0x7f4e600042f0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 shutdown_connections 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f4e5c0b6680 0x7f4e5c0b8b90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e70108300 0x7f4e701995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70108c50 0x7f4e70199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 --2- 172.21.15.31:0/30205077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e70102a20 0x7f4e701990b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 >> 172.21.15.31:0/30205077 conn(0x7f4e70079630 msgr2=0x7f4e70106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 shutdown_connections 2024-09-21T03:03:12.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.509+0000 7f4e751ed640 1 -- 172.21.15.31:0/30205077 wait complete. 2024-09-21T03:03:12.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:03:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 -- 172.21.15.31:0/3800027011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d810dd50 msgr2=0x7f78d8110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 --2- 172.21.15.31:0/3800027011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d810dd50 0x7f78d8110240 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009ca0 tx=0x7f78d002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 -- 172.21.15.31:0/3800027011 shutdown_connections 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 --2- 172.21.15.31:0/3800027011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d8105240 0x7f78d810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 --2- 172.21.15.31:0/3800027011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d810dd50 0x7f78d8110240 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 --2- 172.21.15.31:0/3800027011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 0x7f78d8104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 -- 172.21.15.31:0/3800027011 >> 172.21.15.31:0/3800027011 conn(0x7f78d80fe3c0 msgr2=0x7f78d8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 -- 172.21.15.31:0/3800027011 shutdown_connections 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 -- 172.21.15.31:0/3800027011 wait complete. 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.637+0000 7f78e0494640 1 Processor -- start 2024-09-21T03:03:12.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 -- start start 2024-09-21T03:03:12.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 0x7f78d819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 0x7f78d819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d810dd50 0x7f78d819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81139d0 con 0x7f78d8105240 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8113850 con 0x7f78d81048f0 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78e0494640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8113b50 con 0x7f78d810dd50 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 0x7f78d819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 0x7f78d819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55824/0 (socket says 172.21.15.31:55824) 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 -- 172.21.15.31:0/4142968862 learned_addr learned my addr 172.21.15.31:0/4142968862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78de230640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 0x7f78d819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dea31640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d810dd50 0x7f78d819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 -- 172.21.15.31:0/4142968862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d810dd50 msgr2=0x7f78d819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d810dd50 0x7f78d819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 -- 172.21.15.31:0/4142968862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 msgr2=0x7f78d819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 0x7f78d819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 -- 172.21.15.31:0/4142968862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81a0380 con 0x7f78d8105240 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78de230640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 0x7f78d819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dea31640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d810dd50 0x7f78d819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.638+0000 7f78dda2f640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 0x7f78d819a9f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e9c0 tx=0x7f78c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.639+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c800ccc0 con 0x7f78d8105240 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.639+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c80054d0 con 0x7f78d8105240 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.639+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81a0670 con 0x7f78d8105240 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.639+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8010430 con 0x7f78d8105240 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.639+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a6e00 con 0x7f78d8105240 2024-09-21T03:03:12.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.640+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d8104d00 con 0x7f78d8105240 2024-09-21T03:03:12.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.641+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f78c8005000 con 0x7f78d8105240 2024-09-21T03:03:12.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.642+0000 7f78cf7fe640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78c40b6680 0x7f78c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.642+0000 7f78de230640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78c40b6680 0x7f78c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.642+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8572+0+0 (secure 0 0 0) 0x7f78c80e7e00 con 0x7f78d8105240 2024-09-21T03:03:12.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.643+0000 7f78de230640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78c40b6680 0x7f78c40b8b90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009b50 tx=0x7f78d4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:12.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.645+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78c8056710 con 0x7f78d8105240 2024-09-21T03:03:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.834+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78d81a70a0 con 0x7f78d8105240 2024-09-21T03:03:12.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.835+0000 7f78cf7fe640 1 -- 172.21.15.31:0/4142968862 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f78c80052f0 con 0x7f78d8105240 2024-09-21T03:03:12.838 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-21T03:03:12.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78c40b6680 msgr2=0x7f78c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78c40b6680 0x7f78c40b8b90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009b50 tx=0x7f78d4005f90 comp rx=0 tx=0).stop 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 msgr2=0x7f78d819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 0x7f78d819a9f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e9c0 tx=0x7f78c800ee80 comp rx=0 tx=0).stop 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 shutdown_connections 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f78c40b6680 0x7f78c40b8b90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d810dd50 0x7f78d819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8105240 0x7f78d819a9f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 --2- 172.21.15.31:0/4142968862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78d81048f0 0x7f78d819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 >> 172.21.15.31:0/4142968862 conn(0x7f78d80fe3c0 msgr2=0x7f78d81077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 shutdown_connections 2024-09-21T03:03:12.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.837+0000 7f78e0494640 1 -- 172.21.15.31:0/4142968862 wait complete. 2024-09-21T03:03:12.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:03:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 -- 172.21.15.31:0/349645957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec102980 msgr2=0x7f71ec102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/349645957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec102980 0x7f71ec102d90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009900 tx=0x7f71e802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 -- 172.21.15.31:0/349645957 shutdown_connections 2024-09-21T03:03:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/349645957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71ec108c00 0x7f71ec10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/349645957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ec1082b0 0x7f71ec1086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/349645957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec102980 0x7f71ec102d90 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 -- 172.21.15.31:0/349645957 >> 172.21.15.31:0/349645957 conn(0x7f71ec0fe380 msgr2=0x7f71ec1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 -- 172.21.15.31:0/349645957 shutdown_connections 2024-09-21T03:03:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.961+0000 7f71f3aa9640 1 -- 172.21.15.31:0/349645957 wait complete. 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 Processor -- start 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 -- start start 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ec102980 0x7f71ec19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71ec1082b0 0x7f71ec19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 0x7f71ec19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec1128c0 con 0x7f71ec108c00 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec112740 con 0x7f71ec102980 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f3aa9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec112a40 con 0x7f71ec1082b0 2024-09-21T03:03:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 0x7f71ec19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 0x7f71ec19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55844/0 (socket says 172.21.15.31:55844) 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 -- 172.21.15.31:0/986862798 learned_addr learned my addr 172.21.15.31:0/986862798 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f1044640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71ec1082b0 0x7f71ec19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f1845640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ec102980 0x7f71ec19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 -- 172.21.15.31:0/986862798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71ec1082b0 msgr2=0x7f71ec19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71ec1082b0 0x7f71ec19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 -- 172.21.15.31:0/986862798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ec102980 msgr2=0x7f71ec19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.962+0000 7f71f2046640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ec102980 0x7f71ec19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71f2046640 1 -- 172.21.15.31:0/986862798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec1a0290 con 0x7f71ec108c00 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71f2046640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 0x7f71ec19fc20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f71e400eaa0 tx=0x7f71e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e400cdb0 con 0x7f71ec108c00 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e4005020 con 0x7f71ec108c00 2024-09-21T03:03:12.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec1a0580 con 0x7f71ec108c00 2024-09-21T03:03:12.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40116d0 con 0x7f71ec108c00 2024-09-21T03:03:12.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.963+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec1a6da0 con 0x7f71ec108c00 2024-09-21T03:03:12.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.964+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec103bf0 con 0x7f71ec108c00 2024-09-21T03:03:12.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.966+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f71e40051d0 con 0x7f71ec108c00 2024-09-21T03:03:12.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.967+0000 7f71deffd640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f71d80b6680 0x7f71d80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:12.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.967+0000 7f71f1845640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f71d80b6680 0x7f71d80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:12.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.967+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8572+0+0 (secure 0 0 0) 0x7f71e40e79f0 con 0x7f71ec108c00 2024-09-21T03:03:12.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.968+0000 7f71f1845640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f71d80b6680 0x7f71d80b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f71e8031440 tx=0x7f71e8005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:12.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:12.972+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71e4016030 con 0x7f71ec108c00 2024-09-21T03:03:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:12 smithi031 ceph-mon[31209]: pgmap v641: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-21T03:03:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:12 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3407655374' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:03:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:12 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3794800914' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:03:13.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:12 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/30205077' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:03:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:12 smithi165 ceph-mon[34818]: pgmap v641: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-21T03:03:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:12 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3407655374' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:03:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:12 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3794800914' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:03:13.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:12 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/30205077' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:03:13.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.213+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f71ec19b510 con 0x7f71ec108c00 2024-09-21T03:03:13.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.221+0000 7f71deffd640 1 -- 172.21.15.31:0/986862798 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+834048 (secure 0 0 0) 0x7f71e40adeb0 con 0x7f71ec108c00 2024-09-21T03:03:13.231 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:12 smithi178 ceph-mon[34759]: pgmap v641: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-21T03:03:13.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:12 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3407655374' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:03:13.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:12 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3794800914' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:03:13.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:12 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/30205077' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 27, 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 14223, 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 44765732 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 44765732 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/44765732", 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-09-21T02:44:15.049757+0000", 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24151, 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:03:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-21T03:03:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:03:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-21T03:03:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:03:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-21T03:03:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-21T03:03:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:03:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-21T03:03:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:03:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nginx": { 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nginx", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nginx container image", 2024-09-21T03:03:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:03:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-21T03:03:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba_metrics": { 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:03:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-21T03:03:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-21T03:03:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:03:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-21T03:03:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:03:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-21T03:03:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-21T03:03:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-21T03:03:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-21T03:03:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_interval": { 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:03:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:03:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-21T03:03:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:03:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:03:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:03:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:03:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:03:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "sso_oauth2": { 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sso_oauth2", 2024-09-21T03:03:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T03:03:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-21T03:03:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-21T03:03:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-21T03:03:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-21T03:03:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-21T03:03:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-21T03:03:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:03:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-21T03:03:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-21T03:03:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-21T03:03:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-21T03:03:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-21T03:03:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-21T03:03:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-21T03:03:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max_requests": { 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_requests", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:03:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-21T03:03:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-21T03:03:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smb", 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "internal_store_backend": { 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "internal_store_backend", 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "update_orchestration": { 2024-09-21T03:03:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "update_orchestration", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-21T03:03:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:03:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:03:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-21T03:03:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-21T03:03:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-21T03:03:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-21T03:03:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24175, 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-21T03:03:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-21T03:03:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-21T03:03:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-21T03:03:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-21T03:03:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-21T03:03:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-21T03:03:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-21T03:03:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:03:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:03:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:03:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:03:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:03:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:03:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nginx": { 2024-09-21T03:03:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nginx", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nginx container image", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-21T03:03:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:03:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-21T03:03:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba_metrics": { 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-21T03:03:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:03:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:03:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:03:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:03:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_interval": { 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:03:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:03:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:03:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:03:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:03:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-09-21T03:03:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:03:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:03:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:03:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:03:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T03:03:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-21T03:03:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "sso_oauth2": { 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sso_oauth2", 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-21T03:03:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-21T03:03:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:03:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-21T03:03:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-21T03:03:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-21T03:03:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-21T03:03:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-21T03:03:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-21T03:03:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:03:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-21T03:03:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-21T03:03:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-21T03:03:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T03:03:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:03:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T03:03:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max_requests": { 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_requests", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-21T03:03:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smb", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "internal_store_backend": { 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "internal_store_backend", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:03:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "update_orchestration": { 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "update_orchestration", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:03:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:03:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-21T03:03:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-21T03:03:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-21T03:03:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:03:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-21T03:03:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-21T03:03:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:03:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-09-21T03:03:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "restful", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_schedule" 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-21T03:03:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:03:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:03:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-21T03:03:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-21T03:03:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:03:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-21T03:03:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-21T03:03:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nginx": { 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nginx", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nginx container image", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba_metrics": { 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:03:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-21T03:03:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:03:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:03:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-21T03:03:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:03:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-21T03:03:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_interval": { 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:03:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-21T03:03:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-21T03:03:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:03:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:03:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:03:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:03:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:03:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:03:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:03:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:03:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:03:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:03:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-21T03:03:14.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-21T03:03:14.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.489 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.492 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-21T03:03:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.495 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "sso_oauth2": { 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sso_oauth2", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:14.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-21T03:03:14.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.526 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.527 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.528 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.529 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.530 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-21T03:03:14.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.541 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-21T03:03:14.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:14.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.552 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-21T03:03:14.558 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.559 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.562 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:03:14.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.571 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-21T03:03:14.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-21T03:03:14.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f71d80b6680 msgr2=0x7f71d80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f71d80b6680 0x7f71d80b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f71e8031440 tx=0x7f71e8005d50 comp rx=0 tx=0).stop 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.593 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.598 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.599 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.600 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.603 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.604 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.607 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.609 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-21T03:03:14.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.613 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-21T03:03:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.621 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.624 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.629 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.630 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.631 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.634 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.635 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.638 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.647 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-21T03:03:14.655 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:03:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:03:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 msgr2=0x7f71ec19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 0x7f71ec19fc20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f71e400eaa0 tx=0x7f71e400ef60 comp rx=0 tx=0).stop 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 shutdown_connections 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f71d80b6680 0x7f71d80b8b90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71ec1082b0 0x7f71ec19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec108c00 0x7f71ec19fc20 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 --2- 172.21.15.31:0/986862798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71ec102980 0x7f71ec19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 >> 172.21.15.31:0/986862798 conn(0x7f71ec0fe380 msgr2=0x7f71ec06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:14.659 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.660 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.663 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-21T03:03:14.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-21T03:03:14.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-21T03:03:14.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.676 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.678 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.680 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-21T03:03:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max_requests": { 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_requests", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:03:14.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:03:14.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.706 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.709 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.712 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.721 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-21T03:03:14.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 shutdown_connections 2024-09-21T03:03:14.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:13.228+0000 7f71f3aa9640 1 -- 172.21.15.31:0/986862798 wait complete. 2024-09-21T03:03:14.737 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:13 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/4142968862' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:03:14.737 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:13 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/986862798' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:03:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:13 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/4142968862' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:03:14.738 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:13 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/986862798' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:03:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-21T03:03:14.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smb", 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "internal_store_backend": { 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "internal_store_backend", 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "update_orchestration": { 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "update_orchestration", 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T03:03:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.755 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.769 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-21T03:03:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-21T03:03:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-21T03:03:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-21T03:03:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-21T03:03:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-21T03:03:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T03:03:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T03:03:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-09-21T03:03:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-09-21T03:03:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-09-21T03:03:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T03:03:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T03:03:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 5, 2024-09-21T03:03:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3897449423 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1752739616 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T03:03:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1643689360 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3473368718 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3868805791 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1196903843 2024-09-21T03:03:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T03:03:14.851 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:03:14.857 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:03:14.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:03:14.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:13 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/4142968862' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:03:14.858 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:13 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/986862798' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:03:15.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:14 smithi031 ceph-mon[31209]: pgmap v642: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-09-21T03:03:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:15.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:14 smithi165 ceph-mon[34818]: pgmap v642: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-09-21T03:03:15.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:14 smithi178 ceph-mon[34759]: pgmap v642: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-09-21T03:03:15.682 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:03:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1472, 2024-09-21T03:03:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1474, 2024-09-21T03:03:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 441497, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 468097, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.154913861, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.069559619, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.033899062, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.038843618, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1472, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.047346189, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000032164 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 667, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 819, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 166704, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 570488, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.087218116, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.040362197, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.017505218, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.022474362, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 818, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.034095988, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000041682 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 359904 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 228, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 219, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1179516, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 128078, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.042550428, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.019020685, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.034114779, 2024-09-21T03:03:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000020184, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 218, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.012604896, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000057820 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1179248, 2024-09-21T03:03:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 127760 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3498, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.094599033, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000027043 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2024-09-21T03:03:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2024-09-21T03:03:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_write_failed": 0 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 6, 2024-09-21T03:03:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 2, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 2, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 4195956, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 4195815, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 6 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 3, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 3, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 0, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 1, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 1, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 0, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 0, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 0, 2024-09-21T03:03:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 119088, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 316424192 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2024-09-21T03:03:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2024-09-21T03:03:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2024-09-21T03:03:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 0, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 2, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 2, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 1, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 1156, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 189440, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 52, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T03:03:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 8376, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 185, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 8672, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 13, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T03:03:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 2135, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 2135, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2135, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 1.371975998, 2024-09-21T03:03:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000642611 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2135, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 2136 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 2135, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 2135, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 1392, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 741, 2024-09-21T03:03:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2024-09-21T03:03:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 85, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2024-09-21T03:03:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 9, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 5, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 5, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2024-09-21T03:03:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 2363, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1560351, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 2363, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2363, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1560351, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2135, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 1657, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2135, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1657, 2024-09-21T03:03:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2135, 2024-09-21T03:03:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2135, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2135, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2135, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x56013252b520": { 2024-09-21T03:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "trackedop": { 2024-09-21T03:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2024-09-21T03:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:03:16.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:15 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:03:16.137 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:15 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:03:16.147 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:15 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:16.522 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:16.522 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:03:16.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:03:16.795 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:16 smithi031 ceph-mon[31209]: pgmap v643: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:17.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:16 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:16 smithi165 ceph-mon[34818]: pgmap v643: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:17.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:16 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:17.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:16 smithi178 ceph-mon[34759]: pgmap v643: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:17.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:16 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 217, 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 210, 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1145872, 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 128704, 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.044288573, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.019314108, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.034949636, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 209, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.010948400, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000052384 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.340 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1145872, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 128704 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 159, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 108272, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.015637400, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.012462669, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001468819, 2024-09-21T03:03:17.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 157, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.009000382, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000057327 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:17.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 108272 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3040, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.002215471, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000044995, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002177526, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:03:17.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.003902410, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001300803 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:17.345 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:17.346 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:17.347 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148967, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:17.348 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 968, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:17.349 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 218, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1115201, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 218, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 218, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1115201, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:17.350 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:17.351 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:17.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:17.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:17.351 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:03:18.095 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T03:03:18.095 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T03:03:18.095 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-21T03:03:18.095 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-21T03:03:18.095 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T03:03:18.096 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:03:18.096 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:03:18.096 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:03:18.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:03:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:17 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:17 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:17 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:18.929 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:19.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:18 smithi031 ceph-mon[31209]: pgmap v644: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:19.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:18 smithi165 ceph-mon[34818]: pgmap v644: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:19.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:18 smithi178 ceph-mon[34759]: pgmap v644: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 738, 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 740, 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 247913, 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 234685, 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T03:03:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.106363851, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.055733584, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.019300312, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.022322045, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 738, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.040471828, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000054839 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1640, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1629, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1542853, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 576971, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.155927201, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.064145751, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.060783498, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.035101961, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1629, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.066155165, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000040610 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1182832, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 129488 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 159, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1508, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 360613, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014631153, 2024-09-21T03:03:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.011221574, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001916584, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000056475, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 157, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.008029542, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000051143 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 359648 2024-09-21T03:03:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3625, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.098496101, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000027171 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2024-09-21T03:03:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2024-09-21T03:03:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2024-09-21T03:03:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2024-09-21T03:03:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2024-09-21T03:03:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2024-09-21T03:03:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_write_failed": 0 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2024-09-21T03:03:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "evlrg": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 7, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 2, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "segmjr": 2, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2024-09-21T03:03:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 4194304, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 4196347, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 4196206, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 7 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 3, 2024-09-21T03:03:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 3, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 1, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 1, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 116580, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 316416000 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2024-09-21T03:03:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2024-09-21T03:03:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2024-09-21T03:03:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2024-09-21T03:03:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 1, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 3, 2024-09-21T03:03:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 2, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 2, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 2785, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T03:03:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 197463, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 54, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 8376, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 185, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 8672, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 13, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 2148, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 3, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 2148, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2148, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 1.208978808, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000562839 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2148, 2024-09-21T03:03:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 2149 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 2148, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 2148, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 1400, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 746, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2024-09-21T03:03:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2024-09-21T03:03:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2024-09-21T03:03:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 85, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 9, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 6, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 6, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2024-09-21T03:03:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 2378, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1563454, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 2378, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2378, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1563454, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T03:03:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2148, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2048, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2148, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2048, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2148, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2148, 2024-09-21T03:03:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2148, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2148, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55c58723d450": { 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "trackedop": { 2024-09-21T03:03:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2024-09-21T03:03:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:19.546 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:03:20.448 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T03:03:20.449 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T03:03:20.449 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-21T03:03:20.449 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-21T03:03:20.449 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T03:03:20.449 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:20.449 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:20.449 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:03:20.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:03:20.741 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:21.166 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:20 smithi031 ceph-mon[31209]: pgmap v645: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:21.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:21.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:20 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:21.294 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T03:03:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:21.294 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 219, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 212, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1146416, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 129984, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.045925941, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.019538806, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.035842751, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:03:21.295 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 211, 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.012077179, 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000057237 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.296 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1146416, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 129984 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 161, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 108624, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 3, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.016941172, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.013773308, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001385516, 2024-09-21T03:03:21.297 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 159, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.008584133, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000053988 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:21.298 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 108624 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3040, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.002289747, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000050136, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002240488, 2024-09-21T03:03:21.299 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.004059393, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001353131 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:03:21.300 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:21.301 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:21.302 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148967, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 968, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 220, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1115461, 2024-09-21T03:03:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 220, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 220, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1115461, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T03:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T03:03:21.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:20 smithi165 ceph-mon[34818]: pgmap v645: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:21.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:21.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:20 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:21.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:20 smithi178 ceph-mon[34759]: pgmap v645: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 204 B/s wr, 13 op/s 2024-09-21T03:03:21.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:21.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:20 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:22.106 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T03:03:22.106 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T03:03:22.106 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-21T03:03:22.106 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-21T03:03:22.106 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T03:03:22.106 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:03:22.107 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:03:22.107 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:03:22.107 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:03:22.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:03:22.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:03:22.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:21 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:22.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:21 smithi031 ceph-mon[31209]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:03:22.370 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:03:22.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:03:22.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:03:22.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:21 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:22.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:21 smithi165 ceph-mon[34818]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:03:22.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:03:22.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:03:22.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:21 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' 2024-09-21T03:03:22.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:21 smithi178 ceph-mon[34759]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:03:22.931 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-21T03:03:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2612, 2024-09-21T03:03:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1389, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1002010, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1642882, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 32, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 8, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.352269929, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.112386350, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.140112739, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.094873125, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1382, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.220792340, 2024-09-21T03:03:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000159762 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2880, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000699733, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000242 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 351232, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 120848 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 835, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 896, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 363301, 2024-09-21T03:03:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 605888, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 9, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.096356481, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.045392006, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.052604904, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.007824752, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 897, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.828651936, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000923803 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 911, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000274734, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000301 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 367520 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1751, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1475, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1638576, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1900376, 2024-09-21T03:03:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 116, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.245714982, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.101661068, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.158261340, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.011402395, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1477, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.529722894, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000358647 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2020, 2024-09-21T03:03:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000487708, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000241 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1252816, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 155440 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 546, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.194167119, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000355617 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 85, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.003134130, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000036872 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-21T03:03:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 164, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 164, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 164, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.172368237, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001051025 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 12, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-21T03:03:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 37, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 37, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 36, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 2, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 3, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 293, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 285, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 195, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-09-21T03:03:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 195, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 119, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 16, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 103, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 76, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 39, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 43, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 124, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 84, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 3, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 26 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 16, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 32, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 2, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 6, 2024-09-21T03:03:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 4, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 16, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 227, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 227, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-21T03:03:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 2, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 5129516, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 189, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.657676722, 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003479771 2024-09-21T03:03:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 29, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 49, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 20, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 20, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 20, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 37, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 41, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 4, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 3, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 21, 2024-09-21T03:03:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 18, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 122556, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 337395712 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 217, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 1, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 164, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 28, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 2, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006828640, 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000853580 2024-09-21T03:03:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 101, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.021600790, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000213869 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-21T03:03:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000164644, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000164644 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001672795, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000167279 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013833105, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001729138 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-21T03:03:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.054796872, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005479687 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000404253, 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000404253 2024-09-21T03:03:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.003793465, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001896732 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.002972273, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001486136 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.016112800, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008056400 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.018245568, 2024-09-21T03:03:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004561392 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000889768, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000889768 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.024849927, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002484992 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006203337, 2024-09-21T03:03:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001550834 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-09-21T03:03:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 27, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 13, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 591, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 337923, 2024-09-21T03:03:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 141, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 528, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 6, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8376, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 185, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 136949, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 1403, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 1071, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 953362, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 37, 2024-09-21T03:03:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 1071, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1071, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 23.060645123, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.021531881 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1071, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1115 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 1071, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 229, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 842, 2024-09-21T03:03:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 11, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 18, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 191, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 28, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 579, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 211, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 22, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-21T03:03:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-21T03:03:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 86, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 45, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 10 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-21T03:03:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 64, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 3, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 67, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 4, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 5194, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2477459, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 5194, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5194, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2477459, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 1071, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 953798, 2024-09-21T03:03:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1071, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 953798, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 1071, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1071, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1071, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1071, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 4, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 436, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 4, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 436, 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x5652a5abf380": { 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 227, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 935212, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 227, 2024-09-21T03:03:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 189, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 935212, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:22.977 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-21T03:03:23.059 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:22 smithi165 ceph-mon[34818]: pgmap v646: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 193 B/s wr, 13 op/s 2024-09-21T03:03:23.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:22 smithi031 ceph-mon[31209]: pgmap v646: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 193 B/s wr, 13 op/s 2024-09-21T03:03:23.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:22 smithi178 ceph-mon[34759]: pgmap v646: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 193 B/s wr, 13 op/s 2024-09-21T03:03:23.709 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T03:03:23.710 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T03:03:23.710 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=16 2024-09-21T03:03:23.710 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-21T03:03:23.710 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T03:03:23.710 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-21T03:03:23.710 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-21T03:03:23.710 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:03:23.710 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:03:23.962 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 691, 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 772, 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1325980, 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 346574, 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-09-21T03:03:24.508 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.120923589, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.058408131, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.070508201, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.003764366, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 771, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.050397714, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000065366 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 912, 2024-09-21T03:03:24.509 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000253990, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000278 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1189936, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 136736 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 56, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 214, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 12472, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 793530, 2024-09-21T03:03:24.510 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 43, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.030252900, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.010590098, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.017355332, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000190057, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 213, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.020942837, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000098323 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 49, 2024-09-21T03:03:24.511 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000007041, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000143 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 5056, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 364160 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 30, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 46, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 7168, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 39110, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 9, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-09-21T03:03:24.512 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.006527470, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.001830031, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.003859610, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000615168, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 44, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.015352978, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000348931 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000008882, 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000277 2024-09-21T03:03:24.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 5152 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 57, 2024-09-21T03:03:24.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.002197856, 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000038558 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000023352, 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000011676 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-21T03:03:24.515 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-21T03:03:24.516 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 16, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 16, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 13, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 4, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 3, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 7, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 4, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 3, 2024-09-21T03:03:24.517 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-09-21T03:03:24.518 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:24.519 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-09-21T03:03:24.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 11, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 7, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 8, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-09-21T03:03:24.521 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:24.522 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:24.523 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 7, 2024-09-21T03:03:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 7, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 2, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-21T03:03:24.525 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4196347, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6, 2024-09-21T03:03:24.526 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.012591768, 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002098628 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 20, 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 22, 2024-09-21T03:03:24.527 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 2, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 19, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 23, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 4, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 16, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 20, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 4, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-09-21T03:03:24.528 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 118764, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 325857280 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 16, 2024-09-21T03:03:24.529 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.530 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-21T03:03:24.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.532 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.534 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.535 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.536 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.537 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.538 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:24.540 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.542 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-21T03:03:24.543 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 559, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:24.544 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:24.546 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 246135, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 78, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:24.547 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8376, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 185, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 84377, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 413, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-21T03:03:24.548 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 43, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 12435, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 29, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 43, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 43, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.233996409, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005441776 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.549 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 43, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 65 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 43, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 0, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 43, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 0, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 6, 2024-09-21T03:03:24.550 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 8, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 6, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:24.551 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-21T03:03:24.552 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 86, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 9, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 9, 2024-09-21T03:03:24.553 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 14, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 1 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:24.554 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 773, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1258966, 2024-09-21T03:03:24.555 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 773, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 773, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1258966, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:24.556 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 43, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 12435, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 43, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 12435, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 43, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 43, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 43, 2024-09-21T03:03:24.558 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 43, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:24.559 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x558bcd5ad5f0": { 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:24.560 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 7, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2043, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 7, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2043, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:24.561 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:24.562 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-21T03:03:24.562 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-09-21T03:03:24.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:24.562 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-21T03:03:25.181 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:24 smithi165 ceph-mon[34818]: pgmap v648: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=11 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:25.212 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:25.212 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:03:25.213 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:03:25.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:24 smithi031 ceph-mon[31209]: pgmap v648: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:03:25.462 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:03:25.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:24 smithi178 ceph-mon[34759]: pgmap v648: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:03:26.006 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 303, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 317, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1212138, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 201828, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 13, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.064137174, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.024994926, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.048674833, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000503553, 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:26.007 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 314, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.022888389, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000072892 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 58, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000014775, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000254 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1206208, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 136672 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 432, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 681, 2024-09-21T03:03:26.008 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 131498, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 567330, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.085592920, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.050080386, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.029780482, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.003986556, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 681, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.047444155, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000069668 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:26.009 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 923, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000248598, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000269 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 365104 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 52, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 52, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 12073, 2024-09-21T03:03:26.010 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 407770, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 38, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.018670511, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.001969189, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.015450616, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000393774, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 51, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.018592239, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000364553 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:26.011 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000003324, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000103 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 43, 2024-09-21T03:03:26.012 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001344109, 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000031258 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000016202, 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000008101 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.013 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 1, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 1, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000196738, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000196738 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:26.014 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 16, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 16, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 13, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 4, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 1, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 1, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 3, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 7, 2024-09-21T03:03:26.015 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 4, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 3, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:26.016 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:26.017 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-09-21T03:03:26.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 11, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 7, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 8, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:26.019 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:26.020 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 5, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 5, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 2, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-21T03:03:26.021 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4195732, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.008376942, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002094235 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.022 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 20, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 22, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 2, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 19, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 23, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 4, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 16, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 20, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 4, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 1, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 1, 2024-09-21T03:03:26.023 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 120900, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 325857280 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 1, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 1, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 13, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.024 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.025 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000196738, 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000196738 2024-09-21T03:03:26.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.027 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.028 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-21T03:03:26.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:26.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.033 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-21T03:03:26.034 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.035 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-21T03:03:26.036 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 1, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 1, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 615, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:26.037 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:26.038 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 262067, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 79, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:26.040 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8256, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 182, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:26.041 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 84705, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 416, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 41, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 11941, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 33, 2024-09-21T03:03:26.042 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 41, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 41, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.225996521, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005512110 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 41, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 64 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 41, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 0, 2024-09-21T03:03:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 41, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 4, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 7, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 6, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-21T03:03:26.044 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-21T03:03:26.045 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-21T03:03:26.046 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 86, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 9, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 9, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 16, 2024-09-21T03:03:26.047 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 1 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:26.048 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 783, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1268367, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 783, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:26.049 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 783, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1268367, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:26.050 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 41, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 11941, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 41, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 11941, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.051 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 41, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 41, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 41, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 41, 2024-09-21T03:03:26.052 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:26.053 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x556f1df3d520": { 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:26.054 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 5, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1428, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 5, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1428, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:26.055 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-21T03:03:26.056 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-09-21T03:03:26.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:26.056 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-21T03:03:26.767 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T03:03:26.767 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T03:03:26.767 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-21T03:03:26.767 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=11 2024-09-21T03:03:26.767 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T03:03:26.768 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:26.768 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:26.768 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:03:26.768 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:03:27.015 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:03:27.270 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:26 smithi165 ceph-mon[34818]: pgmap v649: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:03:27.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:26 smithi031 ceph-mon[31209]: pgmap v649: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:03:27.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:26 smithi178 ceph-mon[34759]: pgmap v649: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T03:03:27.523 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-21T03:03:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 984, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 975, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1433088, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 371406, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 1, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.158912286, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.076351519, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.063061286, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.023975981, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 974, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.057515597, 2024-09-21T03:03:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000059050 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1182768, 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 134496 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 7, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 167, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 3756, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 366954, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.014572276, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.009612402, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.003269425, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000111597, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 166, 2024-09-21T03:03:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.014571457, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000087779 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 366256 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1424, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1425, 2024-09-21T03:03:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 364493, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 452521, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.113145195, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.042400601, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.026906054, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.037758800, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1423, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.051898622, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000036471 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3662, 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.101887470, 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000027822 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-21T03:03:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-09-21T03:03:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 5, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-21T03:03:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4195732, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 4195424, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 5 2024-09-21T03:03:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-21T03:03:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 3, 2024-09-21T03:03:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 3, 2024-09-21T03:03:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 0, 2024-09-21T03:03:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 1, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 1, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 117020, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 316424192 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-09-21T03:03:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-09-21T03:03:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1942, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 181320, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 47, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 8256, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 182, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-21T03:03:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 8672, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 13, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2170, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2170, 2024-09-21T03:03:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2170, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.199981271, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000552986 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2170, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2171 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2170, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2170, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-21T03:03:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1416, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 752, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-09-21T03:03:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-21T03:03:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-21T03:03:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 86, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 4, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 4, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-09-21T03:03:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 2411, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1569037, 2024-09-21T03:03:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 2411, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2411, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1569037, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-21T03:03:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2170, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1433, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2170, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1433, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2170, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2170, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2170, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2170, 2024-09-21T03:03:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55a691ec15f0": { 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-21T03:03:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-21T03:03:27.574 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-21T03:03:28.330 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T03:03:28.330 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T03:03:28.330 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-21T03:03:28.330 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-21T03:03:28.330 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T03:03:28.331 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:28.331 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:28.331 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:03:28.331 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:03:28.591 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 72, 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 76, 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 14966, 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 437106, 2024-09-21T03:03:29.084 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 44, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.022331937, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.002737171, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.014806357, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000802131, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 74, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.018891991, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000255297 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 58, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000007773, 2024-09-21T03:03:29.085 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000134 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 4544 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 19, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 176, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 6780, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 396042, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 8, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.016707864, 2024-09-21T03:03:29.086 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.009748626, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.005014758, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000319842, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 175, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.028314408, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000161796 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000189, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000094 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.087 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 365728 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 693, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 775, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1327837, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 259391, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.111773451, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.053037054, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.069682107, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005018369, 2024-09-21T03:03:29.088 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 774, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.048070910, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000062107 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 920, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000270492, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000294 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1196656, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 137216 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-21T03:03:29.089 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 36, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.003209963, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000089165 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:29.090 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000033679, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000016839 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 4, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 3, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002484867, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000828289 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.091 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 1, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 16, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 16, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 13, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 4, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-21T03:03:29.092 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 13, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 10, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 3, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-21T03:03:29.093 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:29.094 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 11, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 7, 2024-09-21T03:03:29.095 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 8, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:29.096 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 6, 2024-09-21T03:03:29.097 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 6, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-21T03:03:29.098 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4195956, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.009229181, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001845836 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 20, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 22, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 2, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 19, 2024-09-21T03:03:29.099 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 23, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 4, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 16, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 20, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 4, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 3, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 3, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 118140, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 325861376 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 7, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 4, 2024-09-21T03:03:29.100 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 8, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 1, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.001980655, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001980655 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.101 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000504212, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000252106 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:29.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:29.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.104 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.105 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.106 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:29.107 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.108 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:29.109 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.110 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.111 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 0, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-09-21T03:03:29.112 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 2, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 2, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 72, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:29.113 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:29.114 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:29.115 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 246791, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 80, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 88, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 1, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8256, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 182, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:29.116 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 84489, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 427, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 44, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 12269, 2024-09-21T03:03:29.117 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 44, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 44, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.249995215, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.005681709 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 44, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 68 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 44, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 0, 2024-09-21T03:03:29.118 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 44, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 5, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 8, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 6, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:29.119 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-21T03:03:29.120 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-21T03:03:29.121 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 86, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 10, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 8, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 8, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 17, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 2 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-21T03:03:29.122 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:29.123 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 780, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1262236, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 780, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 780, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1262236, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.124 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 44, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 12269, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 44, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 12269, 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:29.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 44, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 44, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 44, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 44, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:29.127 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.128 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x5636428695f0": { 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 6, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1652, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 6, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1652, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:29.129 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:29.130 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-21T03:03:29.251 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:28 smithi178 ceph-mon[34759]: pgmap v650: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:29.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:28 smithi031 ceph-mon[31209]: pgmap v650: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:29.343 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:28 smithi165 ceph-mon[34818]: pgmap v650: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=11 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:29.968 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:29.968 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:03:29.969 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:03:30.227 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:03:30.733 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1845, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1834, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 2594732, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 643418, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.242232299, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.106620953, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.082095418, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.050429459, 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:30.734 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1834, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.084480328, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000046063 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1184816, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 133024 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:30.735 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 5, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 165, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1816, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 366810, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.013065328, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.009220892, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.002077202, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000118196, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 163, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.014008379, 2024-09-21T03:03:30.736 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000085940 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 365472 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 743, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 744, 2024-09-21T03:03:30.737 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 183080, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 235963, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.091321125, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.042578322, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.017229964, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.024659338, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 742, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.033896936, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000045683 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:30.738 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:30.739 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3869, 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.114219545, 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000029521 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:30.741 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 31, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 31, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 31, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-21T03:03:30.742 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:30.743 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:30.744 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:30.746 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 227, 2024-09-21T03:03:30.747 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 5129516, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 5129404, 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-21T03:03:30.748 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 227 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 20, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 24, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 4, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 11, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 11, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 31, 2024-09-21T03:03:30.749 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 31, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 117364, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 316416000 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-21T03:03:30.750 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.752 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:30.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.754 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.756 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.757 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-21T03:03:30.758 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.759 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.760 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.761 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:30.762 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 4, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 2, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 3, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-21T03:03:30.763 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2566, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:30.764 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:30.765 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:30.766 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189716, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 59, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8256, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 182, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:30.767 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 93533, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 870, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 2351, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 26, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 4, 2024-09-21T03:03:30.768 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 2351, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2351, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.209976685, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000514664 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2351, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2352 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 2351, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 2351, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-21T03:03:30.769 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 1487, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 861, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-21T03:03:30.770 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-21T03:03:30.771 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 86, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 10, 2024-09-21T03:03:30.772 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:30.773 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 2589, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 2531470, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:30.774 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 2589, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2589, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2531470, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:30.775 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2351, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 935222, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2351, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 935222, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:30.776 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2351, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 2351, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2351, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2351, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-21T03:03:30.778 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55d9d07d5450": { 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:30.779 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:30.780 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-21T03:03:31.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:30 smithi031 ceph-mon[31209]: pgmap v651: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:31.360 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:30 smithi178 ceph-mon[34759]: pgmap v651: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:31.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:30 smithi165 ceph-mon[34818]: pgmap v651: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:31.562 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T03:03:31.563 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T03:03:31.563 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-21T03:03:31.563 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-21T03:03:31.563 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-21T03:03:31.563 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:31.563 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:31.563 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:03:31.563 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:03:31.811 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 38, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 52, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 10501, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 64714, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 13, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.018727904, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.003729638, 2024-09-21T03:03:32.324 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.010158512, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000552131, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 51, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.019574923, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000383822 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 37, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000008875, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000239 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.325 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 25, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 198, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 5807, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 408843, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 9, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.017364533, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.010689117, 2024-09-21T03:03:32.326 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.004669083, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000367553, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 196, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.021806873, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000111259 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 32, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000006277, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000196 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 372096 2024-09-21T03:03:32.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 726, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 805, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1336660, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 792473, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 35, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 8, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.127033589, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.055396717, 2024-09-21T03:03:32.328 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.077744652, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005007377, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 804, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.046854777, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000058277 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 957, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000303089, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000316 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1201856, 2024-09-21T03:03:32.329 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 137216 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 37, 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.002183420, 2024-09-21T03:03:32.330 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000059011 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2, 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000024907, 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000012453 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-21T03:03:32.331 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 1, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 1, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-21T03:03:32.332 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 16, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 16, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 13, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 4, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 3, 2024-09-21T03:03:32.333 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 8, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 4, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 1, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 3, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-21T03:03:32.334 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 6, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:32.335 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-21T03:03:32.336 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 11, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 7, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 8, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:32.337 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-21T03:03:32.338 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-21T03:03:32.339 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 4, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 4, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 2, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-21T03:03:32.340 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4195591, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.005966572, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001988857 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 20, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 22, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 2, 2024-09-21T03:03:32.341 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 19, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 23, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 4, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 16, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 20, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 4, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 120900, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 325861376 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 1, 2024-09-21T03:03:32.342 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 1, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 17, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.343 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.346 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.347 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.347 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-21T03:03:32.347 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-21T03:03:32.348 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.350 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-21T03:03:32.351 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.352 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.353 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.354 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 2, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 602, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-21T03:03:32.355 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:32.356 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:32.357 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 238855, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 84, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:32.358 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8256, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 182, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 84401, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 416, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-21T03:03:32.359 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 39, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 11589, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 39, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 39, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.216995844, 2024-09-21T03:03:32.360 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.005563996 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 39, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 61 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 39, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 0, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 39, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-09-21T03:03:32.361 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 3, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 7, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 6, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-21T03:03:32.363 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-21T03:03:32.364 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 86, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 10, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 8, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 8, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 14, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-21T03:03:32.365 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 1 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:32.366 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:32.367 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 785, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1265264, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 785, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 785, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1265264, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.368 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:32.369 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 39, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 11589, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 39, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 11589, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.370 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:32.371 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 39, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 39, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 39, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 39, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:32.372 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.373 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x5646e4d35380": { 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 4, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1287, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 4, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:32.374 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 3, 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1287, 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:32.375 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:33.032 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:33.032 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:03:33.033 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:03:33.279 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:03:33.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:33 smithi031 ceph-mon[31209]: pgmap v652: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:33.400 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:33 smithi178 ceph-mon[34759]: pgmap v652: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:33.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:33 smithi165 ceph-mon[34818]: pgmap v652: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 242, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 233, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 1184146, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 136100, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.048704974, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.020882895, 2024-09-21T03:03:33.801 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.040506494, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000023897, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 232, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.015571079, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000067116 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1183312, 2024-09-21T03:03:33.802 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 135776 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 639, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 803, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 157130, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 570713, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.087326904, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.039818686, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.016459834, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.023623036, 2024-09-21T03:03:33.803 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 801, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.040251098, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000050251 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 367824 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:03:33.804 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1511, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1511, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 454754, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 479732, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.194706782, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.084476972, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.044284424, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.051774986, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1510, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.070503246, 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000046690 2024-09-21T03:03:33.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-21T03:03:33.806 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3583, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.118678664, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000033122 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.807 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-21T03:03:33.808 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-09-21T03:03:33.809 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-21T03:03:33.810 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:03:33.811 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-21T03:03:33.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-21T03:03:33.813 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-21T03:03:33.814 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 4, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-21T03:03:33.815 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 2, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 4195591, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 4195200, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 4 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-21T03:03:33.816 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 3, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 3, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 1, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 1, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 117796, 2024-09-21T03:03:33.817 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 316424192 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.819 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.820 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.821 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.822 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-21T03:03:33.823 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.823 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.823 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.823 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-21T03:03:33.823 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-21T03:03:33.824 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.825 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-21T03:03:33.826 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-21T03:03:33.827 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-21T03:03:33.828 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-21T03:03:33.829 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-21T03:03:33.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-21T03:03:33.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 2, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-21T03:03:33.832 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 2, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1926, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:03:33.833 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-21T03:03:33.834 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:03:33.835 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 189383, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 50, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-21T03:03:33.836 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 8256, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 182, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 8672, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 13, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-21T03:03:33.837 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 2146, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 2146, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2146, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1.212976588, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000565226 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-21T03:03:33.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2146, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2147 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 2146, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 2146, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 1387, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 757, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-21T03:03:33.839 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-21T03:03:33.840 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-21T03:03:33.841 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 86, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 10, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 5, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 5, 2024-09-21T03:03:33.842 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-21T03:03:33.843 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 2388, 2024-09-21T03:03:33.844 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1565696, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 2388, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2388, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1565696, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-21T03:03:33.845 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2146, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1292, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2146, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1292, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 2146, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 2146, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 2146, 2024-09-21T03:03:33.847 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 2146, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:33.848 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55e1fbe95450": { 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-21T03:03:33.849 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-21T03:03:33.850 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-21T03:03:33.851 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-21T03:03:34.565 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T03:03:34.565 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T03:03:34.565 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-21T03:03:34.565 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-21T03:03:34.566 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-21T03:03:34.566 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:03:34.566 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:03:34.566 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:03:34.581 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:03:34.581 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T03:03:34.581 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:03:34.603 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T03:03:34.603 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:03:34.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:03:34.978 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:35.268 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:35 smithi031 ceph-mon[31209]: pgmap v653: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:03:35.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:35 smithi165 ceph-mon[34818]: pgmap v653: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:03:35.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:35 smithi178 ceph-mon[34759]: pgmap v653: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-21T03:03:35.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 -- 172.21.15.31:0/1892004771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518104860 msgr2=0x7f2518104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:35.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 --2- 172.21.15.31:0/1892004771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518104860 0x7f2518104c70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2508009900 tx=0x7f250802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:35.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 -- 172.21.15.31:0/1892004771 shutdown_connections 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 --2- 172.21.15.31:0/1892004771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2518105290 0x7f251810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 --2- 172.21.15.31:0/1892004771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f251810de10 0x7f25181102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 --2- 172.21.15.31:0/1892004771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518104860 0x7f2518104c70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 -- 172.21.15.31:0/1892004771 >> 172.21.15.31:0/1892004771 conn(0x7f2518079730 msgr2=0x7f2518079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 -- 172.21.15.31:0/1892004771 shutdown_connections 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.493+0000 7f251eab6640 1 -- 172.21.15.31:0/1892004771 wait complete. 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 Processor -- start 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 -- start start 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2518104860 0x7f251819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 0x7f251819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f251810de10 0x7f251819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:35.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2518113b00 con 0x7f2518105290 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2518113980 con 0x7f2518104860 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f251eab6640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2518113c80 con 0x7f251810de10 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.494+0000 7f2517fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 0x7f251819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 0x7f251819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49244/0 (socket says 172.21.15.31:49244) 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 -- 172.21.15.31:0/16150259 learned_addr learned my addr 172.21.15.31:0/16150259 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f251c852640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2518104860 0x7f251819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f251d053640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f251810de10 0x7f251819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 -- 172.21.15.31:0/16150259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f251810de10 msgr2=0x7f251819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f251810de10 0x7f251819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 -- 172.21.15.31:0/16150259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2518104860 msgr2=0x7f251819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2518104860 0x7f251819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 -- 172.21.15.31:0/16150259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181a0420 con 0x7f2518105290 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f251c852640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2518104860 0x7f251819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:35.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2517fff640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 0x7f251819aa50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f250c00d750 tx=0x7f250c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c017070 con 0x7f2518105290 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25181a0710 con 0x7f2518105290 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c0054d0 con 0x7f2518105290 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.495+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c004c30 con 0x7f2518105290 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.496+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25181a6e70 con 0x7f2518105290 2024-09-21T03:03:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.497+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2518104c70 con 0x7f2518105290 2024-09-21T03:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.498+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f250c004dd0 con 0x7f2518105290 2024-09-21T03:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.499+0000 7f2515ffb640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25000b6680 0x7f25000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.499+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8528+0+0 (secure 0 0 0) 0x7f250c0e75f0 con 0x7f2518105290 2024-09-21T03:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.499+0000 7f251c852640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25000b6680 0x7f25000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.500+0000 7f251c852640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25000b6680 0x7f25000b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2508031440 tx=0x7f2508009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.502+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f250c0adb40 con 0x7f2518105290 2024-09-21T03:03:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.703+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f251819b5e0 con 0x7f2518105290 2024-09-21T03:03:35.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.704+0000 7f2515ffb640 1 -- 172.21.15.31:0/16150259 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+884 (secure 0 0 0) 0x7f251819b5e0 con 0x7f2518105290 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/944835008 2024-09-22T02:44:15.049623+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/4246092214 2024-09-22T02:44:15.049623+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/262827684 2024-09-22T02:43:13.322815+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4029989165 2024-09-22T02:43:13.322815+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1175221317 2024-09-22T02:42:44.994232+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1876173055 2024-09-22T02:43:13.322815+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1175221317 2024-09-22T02:42:44.994232+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/814395253 2024-09-22T02:42:44.994232+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3796123698 2024-09-22T02:43:13.322815+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1104840323 2024-09-22T02:42:44.994232+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/633109246 2024-09-22T02:42:44.994232+0000 2024-09-21T03:03:35.706 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1876173055 2024-09-22T02:43:13.322815+0000 2024-09-21T03:03:35.707 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3425970992 2024-09-22T02:44:15.049623+0000 2024-09-21T03:03:35.707 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/4246092214 2024-09-22T02:44:15.049623+0000 2024-09-21T03:03:35.707 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1565999986 2024-09-22T02:44:15.049623+0000 2024-09-21T03:03:35.707 INFO:teuthology.orchestra.run.smithi031.stderr:listed 15 entries 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25000b6680 msgr2=0x7f25000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25000b6680 0x7f25000b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2508031440 tx=0x7f2508009680 comp rx=0 tx=0).stop 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 msgr2=0x7f251819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 0x7f251819aa50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f250c00d750 tx=0x7f250c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 shutdown_connections 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f25000b6680 0x7f25000b8b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f251810de10 0x7f251819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2518105290 0x7f251819aa50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 --2- 172.21.15.31:0/16150259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2518104860 0x7f251819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 >> 172.21.15.31:0/16150259 conn(0x7f2518079730 msgr2=0x7f25181007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:35.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 shutdown_connections 2024-09-21T03:03:35.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:35.706+0000 7f251eab6640 1 -- 172.21.15.31:0/16150259 wait complete. 2024-09-21T03:03:36.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:36 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/16150259' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:03:36.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:36 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/16150259' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:03:36.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:36 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/16150259' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:03:36.526 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:03:36.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:03:36.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:03:36.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T03:03:36.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:03:36.577 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:03:36.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:03:36.617 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-21T03:03:36.674 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:03:36.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:03:36.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:03:36.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:03:36.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-21T03:03:36.836 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:03:36.846 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:03:36.846 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:03:36.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:03:37.110 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:37.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:37 smithi031 ceph-mon[31209]: pgmap v654: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:03:37.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:37 smithi165 ceph-mon[34818]: pgmap v654: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:03:37.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:37 smithi178 ceph-mon[34759]: pgmap v654: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-21T03:03:37.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 -- 172.21.15.31:0/277574890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60108d20 msgr2=0x7f2c6010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:37.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 --2- 172.21.15.31:0/277574890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60108d20 0x7f2c6010f550 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009ca0 tx=0x7f2c5802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 -- 172.21.15.31:0/277574890 shutdown_connections 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 --2- 172.21.15.31:0/277574890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c601083d0 0x7f2c601087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 --2- 172.21.15.31:0/277574890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60108d20 0x7f2c6010f550 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 --2- 172.21.15.31:0/277574890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c60102ae0 0x7f2c60102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 -- 172.21.15.31:0/277574890 >> 172.21.15.31:0/277574890 conn(0x7f2c60079790 msgr2=0x7f2c60079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 -- 172.21.15.31:0/277574890 shutdown_connections 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.588+0000 7f2c67abf640 1 -- 172.21.15.31:0/277574890 wait complete. 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 Processor -- start 2024-09-21T03:03:37.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 -- start start 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 0x7f2c6019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c601083d0 0x7f2c6019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c60108d20 0x7f2c6019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60113b00 con 0x7f2c60102ae0 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60113980 con 0x7f2c601083d0 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c67abf640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60113c80 con 0x7f2c60108d20 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.589+0000 7f2c6585b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 0x7f2c6019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:37.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 0x7f2c6019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49262/0 (socket says 172.21.15.31:49262) 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 -- 172.21.15.31:0/1365043014 learned_addr learned my addr 172.21.15.31:0/1365043014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6605c640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c60108d20 0x7f2c6019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6505a640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c601083d0 0x7f2c6019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 -- 172.21.15.31:0/1365043014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c60108d20 msgr2=0x7f2c6019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c60108d20 0x7f2c6019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 -- 172.21.15.31:0/1365043014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c601083d0 msgr2=0x7f2c6019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c601083d0 0x7f2c6019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 -- 172.21.15.31:0/1365043014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c601a02c0 con 0x7f2c60102ae0 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6505a640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c601083d0 0x7f2c6019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:37.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c6585b640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 0x7f2c6019a410 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00d950 tx=0x7f2c5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c017070 con 0x7f2c60102ae0 2024-09-21T03:03:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c5c009c30 con 0x7f2c60102ae0 2024-09-21T03:03:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c601a05b0 con 0x7f2c60102ae0 2024-09-21T03:03:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00f620 con 0x7f2c60102ae0 2024-09-21T03:03:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.590+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c601a6dd0 con 0x7f2c60102ae0 2024-09-21T03:03:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.591+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c60102ef0 con 0x7f2c60102ae0 2024-09-21T03:03:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.593+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f2c5c005030 con 0x7f2c60102ae0 2024-09-21T03:03:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.594+0000 7f2c56ffd640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2c4c0b6730 0x7f2c4c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.594+0000 7f2c6505a640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2c4c0b6730 0x7f2c4c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.594+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8528+0+0 (secure 0 0 0) 0x7f2c5c0e7270 con 0x7f2c60102ae0 2024-09-21T03:03:37.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.594+0000 7f2c6505a640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2c4c0b6730 0x7f2c4c0b8c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2c60079c40 tx=0x7f2c5000a5c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.597+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c5c0ad840 con 0x7f2c60102ae0 2024-09-21T03:03:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.824+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c6019b540 con 0x7f2c60102ae0 2024-09-21T03:03:37.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.825+0000 7f2c56ffd640 1 -- 172.21.15.31:0/1365043014 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7f2c5c0b1c20 con 0x7f2c60102ae0 2024-09-21T03:03:37.826 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:03:37.827 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:03:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2c4c0b6730 msgr2=0x7f2c4c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2c4c0b6730 0x7f2c4c0b8c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2c60079c40 tx=0x7f2c5000a5c0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 msgr2=0x7f2c6019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 0x7f2c6019a410 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00d950 tx=0x7f2c5c00de10 comp rx=0 tx=0).stop 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 shutdown_connections 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f2c4c0b6730 0x7f2c4c0b8c40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c60108d20 0x7f2c6019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c601083d0 0x7f2c6019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 --2- 172.21.15.31:0/1365043014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60102ae0 0x7f2c6019a410 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 >> 172.21.15.31:0/1365043014 conn(0x7f2c60079790 msgr2=0x7f2c60104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 shutdown_connections 2024-09-21T03:03:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:37.827+0000 7f2c67abf640 1 -- 172.21.15.31:0/1365043014 wait complete. 2024-09-21T03:03:38.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:38 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1365043014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:38.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:38 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1365043014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:38.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:03:38.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:38 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1365043014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:38.735 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:39.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:39 smithi031 ceph-mon[31209]: pgmap v655: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 -- 172.21.15.31:0/3168750400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c1094a0 msgr2=0x7fbd3c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3168750400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c1094a0 0x7fbd3c1098b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009900 tx=0x7fbd3002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 -- 172.21.15.31:0/3168750400 shutdown_connections 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3168750400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c109df0 0x7fbd3c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3168750400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c1094a0 0x7fbd3c1098b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3168750400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd3c103b60 0x7fbd3c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.226+0000 7fbd41af9640 1 -- 172.21.15.31:0/3168750400 >> 172.21.15.31:0/3168750400 conn(0x7fbd3c07b850 msgr2=0x7fbd3c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.227+0000 7fbd41af9640 1 -- 172.21.15.31:0/3168750400 shutdown_connections 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.227+0000 7fbd41af9640 1 -- 172.21.15.31:0/3168750400 wait complete. 2024-09-21T03:03:39.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.227+0000 7fbd41af9640 1 Processor -- start 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.227+0000 7fbd41af9640 1 -- start start 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd41af9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c103b60 0x7fbd3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd41af9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd3c1094a0 0x7fbd3c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd41af9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 0x7fbd3c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd41af9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c113ab0 con 0x7fbd3c109df0 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd41af9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c113930 con 0x7fbd3c103b60 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd41af9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c113c30 con 0x7fbd3c1094a0 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 0x7fbd3c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 0x7fbd3c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41622/0 (socket says 172.21.15.31:41622) 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 -- 172.21.15.31:0/3036560110 learned_addr learned my addr 172.21.15.31:0/3036560110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3affd640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd3c1094a0 0x7fbd3c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3b7fe640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c103b60 0x7fbd3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 -- 172.21.15.31:0/3036560110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd3c1094a0 msgr2=0x7fbd3c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:39.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd3c1094a0 0x7fbd3c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 -- 172.21.15.31:0/3036560110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c103b60 msgr2=0x7fbd3c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c103b60 0x7fbd3c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 -- 172.21.15.31:0/3036560110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c1a8780 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3b7fe640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c103b60 0x7fbd3c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd3bfff640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 0x7fbd3c076190 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009fd0 tx=0x7fbd3400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.228+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34017070 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.229+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd34003e20 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.229+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c1a8a70 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.229+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34010460 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.229+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a8f80 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.230+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c103f70 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.231+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7fbd34003970 con 0x7fbd3c109df0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.232+0000 7fbd38ff9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbd240b6700 0x7fbd240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.232+0000 7fbd3b7fe640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbd240b6700 0x7fbd240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:39.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.232+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8528+0+0 (secure 0 0 0) 0x7fbd340e7430 con 0x7fbd3c109df0 2024-09-21T03:03:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.233+0000 7fbd3b7fe640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbd240b6700 0x7fbd240b8c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c005fd0 tx=0x7fbd2c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:39.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.235+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd340ada00 con 0x7fbd3c109df0 2024-09-21T03:03:39.385 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:39 smithi165 ceph-mon[34818]: pgmap v655: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T03:03:39.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.461+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd3c076a10 con 0x7fbd3c109df0 2024-09-21T03:03:39.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.462+0000 7fbd38ff9640 1 -- 172.21.15.31:0/3036560110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11982 (secure 0 0 0) 0x7fbd3c076a10 con 0x7fbd3c109df0 2024-09-21T03:03:39.463 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:03:39.464 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-21T02:52:57:782452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3355921594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3355921594},{"type":"v1","addr":"172.21.15.31:6835","nonce":3355921594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3179894266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3179894266},{"type":"v1","addr":"172.21.15.31:6837","nonce":3179894266}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:50:50.973503+0000","modified":"2024-09-21T02:52:57.214959+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24695,"mds_1":24659,"mds_2":24655,"mds_3":24637,"mds_4":24632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6839/3653301851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3653301851},{"type":"v1","addr":"172.21.15.31:6839","nonce":3653301851}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6841/2144690594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":2144690594},{"type":"v1","addr":"172.21.15.31:6841","nonce":2144690594}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1642580204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1642580204},{"type":"v1","addr":"172.21.15.178:6833","nonce":1642580204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3390770693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3390770693},{"type":"v1","addr":"172.21.15.165:6833","nonce":3390770693}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24625":{"gid":24625,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/178089911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":178089911},{"type":"v1","addr":"172.21.15.178:6835","nonce":178089911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24632":{"gid":24632,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":12,"addr":"172.21.15.165:6835/3816837605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3816837605},{"type":"v1","addr":"172.21.15.165:6835","nonce":3816837605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.178:6837/1985506653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1985506653},{"type":"v1","addr":"172.21.15.178:6837","nonce":1985506653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.178:6839/730341197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":730341197},{"type":"v1","addr":"172.21.15.178:6839","nonce":730341197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24659":{"gid":24659,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.165:6837/3116204029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3116204029},{"type":"v1","addr":"172.21.15.165:6837","nonce":3116204029}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6839/2689629692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":2689629692},{"type":"v1","addr":"172.21.15.165:6839","nonce":2689629692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24695,"qdb_cluster":[24632,24637,24655,24659,24695]},"id":1}]} 2024-09-21T03:03:39.464 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbd240b6700 msgr2=0x7fbd240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbd240b6700 0x7fbd240b8c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c005fd0 tx=0x7fbd2c005850 comp rx=0 tx=0).stop 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 msgr2=0x7fbd3c076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 0x7fbd3c076190 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009fd0 tx=0x7fbd3400b260 comp rx=0 tx=0).stop 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 shutdown_connections 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7fbd240b6700 0x7fbd240b8c10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd3c1094a0 0x7fbd3c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3c109df0 0x7fbd3c076190 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 --2- 172.21.15.31:0/3036560110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd3c103b60 0x7fbd3c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.464+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 >> 172.21.15.31:0/3036560110 conn(0x7fbd3c07b850 msgr2=0x7fbd3c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.465+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 shutdown_connections 2024-09-21T03:03:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:39.465+0000 7fbd41af9640 1 -- 172.21.15.31:0/3036560110 wait complete. 2024-09-21T03:03:39.477 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:39 smithi178 ceph-mon[34759]: pgmap v655: 97 pgs: 97 active+clean; 2.2 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T03:03:40.150 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:40 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/3036560110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:40.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:03:40.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:40 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/3036560110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:40.455 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:40.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:40 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/3036560110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 -- 172.21.15.31:0/918003054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53301094c0 msgr2=0x7f53301098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 --2- 172.21.15.31:0/918003054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53301094c0 0x7f53301098d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f5324009900 tx=0x7f532402a410 comp rx=0 tx=0).stop 2024-09-21T03:03:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 -- 172.21.15.31:0/918003054 shutdown_connections 2024-09-21T03:03:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 --2- 172.21.15.31:0/918003054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5330109e10 0x7f5330110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 --2- 172.21.15.31:0/918003054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53301094c0 0x7f53301098d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 --2- 172.21.15.31:0/918003054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5330103b80 0x7f5330103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 -- 172.21.15.31:0/918003054 >> 172.21.15.31:0/918003054 conn(0x7f53300ff5d0 msgr2=0x7f5330101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 -- 172.21.15.31:0/918003054 shutdown_connections 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.946+0000 7f5338676640 1 -- 172.21.15.31:0/918003054 wait complete. 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 Processor -- start 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 -- start start 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 0x7f533019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301094c0 0x7f533019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5330109e10 0x7f53301a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5330113ad0 con 0x7f5330103b80 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5330113950 con 0x7f5330109e10 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5338676640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5330113c50 con 0x7f53301094c0 2024-09-21T03:03:40.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 0x7f533019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 0x7f533019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41640/0 (socket says 172.21.15.31:41640) 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 -- 172.21.15.31:0/1273987357 learned_addr learned my addr 172.21.15.31:0/1273987357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336c13640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5330109e10 0x7f53301a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5335c11640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301094c0 0x7f533019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 -- 172.21.15.31:0/1273987357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301094c0 msgr2=0x7f533019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301094c0 0x7f533019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 -- 172.21.15.31:0/1273987357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5330109e10 msgr2=0x7f53301a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5330109e10 0x7f53301a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.947+0000 7f5336412640 1 -- 172.21.15.31:0/1273987357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53301a2650 con 0x7f5330103b80 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.948+0000 7f5336412640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 0x7f533019c6e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f532000eb90 tx=0x7f532000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.948+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320004d20 con 0x7f5330103b80 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.948+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532000ee90 con 0x7f5330103b80 2024-09-21T03:03:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.948+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53301a2940 con 0x7f5330103b80 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.948+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320010670 con 0x7f5330103b80 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.948+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53301a90a0 con 0x7f5330103b80 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.949+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5330103f90 con 0x7f5330103b80 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.950+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5320010850 con 0x7f5330103b80 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.951+0000 7f532f7fe640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53180b6700 0x7f53180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.951+0000 7f5335c11640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53180b6700 0x7f53180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.951+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8528+0+0 (secure 0 0 0) 0x7f53200e7fe0 con 0x7f5330103b80 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.952+0000 7f5335c11640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53180b6700 0x7f53180b8c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5324006010 tx=0x7f5324000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:40.955+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5320056710 con 0x7f5330103b80 2024-09-21T03:03:41.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.145+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f533019d810 con 0x7f5330103b80 2024-09-21T03:03:41.370 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:41 smithi031 ceph-mon[31209]: pgmap v656: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T03:03:41.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.146+0000 7f532f7fe640 1 -- 172.21.15.31:0/1273987357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23235 (secure 0 0 0) 0x7f53200ae530 con 0x7f5330103b80 2024-09-21T03:03:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T03:03:41.371 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":86,"fsid":"fdd37b14-77c2-11ef-baf5-efdc52797490","created":"2024-09-21T02:42:18.754603+0000","modified":"2024-09-21T03:03:21.721244+0000","last_up_change":"2024-09-21T02:50:00.655908+0000","last_in_change":"2024-09-21T02:49:38.100585+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:46:13.535825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:50:47.175448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:50:48.994282+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":85,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"23d2c70a-c096-4c33-acfd-b18efa64387a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6803","nonce":3594547707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6805","nonce":3594547707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6809","nonce":3594547707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3594547707},{"type":"v1","addr":"172.21.15.31:6807","nonce":3594547707}]},"public_addr":"172.21.15.31:6803/3594547707","cluster_addr":"172.21.15.31:6805/3594547707","heartbeat_back_addr":"172.21.15.31:6809/3594547707","heartbeat_front_addr":"172.21.15.31:6807/3594547707","state":["exists","up"]},{"osd":1,"uuid":"c12dd8d6-d3b4-4d68-a15f-5f4775fe4ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6801","nonce":3056017088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6803","nonce":3056017088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6807","nonce":3056017088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":3056017088},{"type":"v1","addr":"172.21.15.165:6805","nonce":3056017088}]},"public_addr":"172.21.15.165:6801/3056017088","cluster_addr":"172.21.15.165:6803/3056017088","heartbeat_back_addr":"172.21.15.165:6807/3056017088","heartbeat_front_addr":"172.21.15.165:6805/3056017088","state":["exists","up"]},{"osd":2,"uuid":"eca4fd08-95dd-4326-a6e7-59be134cad7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6801","nonce":730728159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6803","nonce":730728159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6807","nonce":730728159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":730728159},{"type":"v1","addr":"172.21.15.178:6805","nonce":730728159}]},"public_addr":"172.21.15.178:6801/730728159","cluster_addr":"172.21.15.178:6803/730728159","heartbeat_back_addr":"172.21.15.178:6807/730728159","heartbeat_front_addr":"172.21.15.178:6805/730728159","state":["exists","up"]},{"osd":3,"uuid":"68acf89d-75fa-43bf-8181-73e5043cc7a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6811","nonce":3748731073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6813","nonce":3748731073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6817","nonce":3748731073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3748731073},{"type":"v1","addr":"172.21.15.31:6815","nonce":3748731073}]},"public_addr":"172.21.15.31:6811/3748731073","cluster_addr":"172.21.15.31:6813/3748731073","heartbeat_back_addr":"172.21.15.31:6817/3748731073","heartbeat_front_addr":"172.21.15.31:6815/3748731073","state":["exists","up"]},{"osd":4,"uuid":"edf3d333-8a5b-4a48-81ac-50917b7593c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6809","nonce":3903622433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6811","nonce":3903622433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6815","nonce":3903622433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":3903622433},{"type":"v1","addr":"172.21.15.165:6813","nonce":3903622433}]},"public_addr":"172.21.15.165:6809/3903622433","cluster_addr":"172.21.15.165:6811/3903622433","heartbeat_back_addr":"172.21.15.165:6815/3903622433","heartbeat_front_addr":"172.21.15.165:6813/3903622433","state":["exists","up"]},{"osd":5,"uuid":"d68d7120-069f-428b-823a-5fbd7f74f03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6809","nonce":1160605172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6811","nonce":1160605172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6815","nonce":1160605172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":1160605172},{"type":"v1","addr":"172.21.15.178:6813","nonce":1160605172}]},"public_addr":"172.21.15.178:6809/1160605172","cluster_addr":"172.21.15.178:6811/1160605172","heartbeat_back_addr":"172.21.15.178:6815/1160605172","heartbeat_front_addr":"172.21.15.178:6813/1160605172","state":["exists","up"]},{"osd":6,"uuid":"3a7e5212-2bd5-4861-8192-1ab5dcb592c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6819","nonce":787463273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6821","nonce":787463273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6825","nonce":787463273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":787463273},{"type":"v1","addr":"172.21.15.31:6823","nonce":787463273}]},"public_addr":"172.21.15.31:6819/787463273","cluster_addr":"172.21.15.31:6821/787463273","heartbeat_back_addr":"172.21.15.31:6825/787463273","heartbeat_front_addr":"172.21.15.31:6823/787463273","state":["exists","up"]},{"osd":7,"uuid":"a58534f6-f366-4d71-a057-272ba5874b0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6817","nonce":2423292259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6819","nonce":2423292259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6823","nonce":2423292259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2423292259},{"type":"v1","addr":"172.21.15.165:6821","nonce":2423292259}]},"public_addr":"172.21.15.165:6817/2423292259","cluster_addr":"172.21.15.165:6819/2423292259","heartbeat_back_addr":"172.21.15.165:6823/2423292259","heartbeat_front_addr":"172.21.15.165:6821/2423292259","state":["exists","up"]},{"osd":8,"uuid":"5831064f-dc15-4eab-8239-3648a77f4b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6817","nonce":3837162203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6819","nonce":3837162203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6823","nonce":3837162203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3837162203},{"type":"v1","addr":"172.21.15.178:6821","nonce":3837162203}]},"public_addr":"172.21.15.178:6817/3837162203","cluster_addr":"172.21.15.178:6819/3837162203","heartbeat_back_addr":"172.21.15.178:6823/3837162203","heartbeat_front_addr":"172.21.15.178:6821/3837162203","state":["exists","up"]},{"osd":9,"uuid":"2d67c10d-2a66-4175-8ec4-f6178369fff1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6827","nonce":1204611917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6829","nonce":1204611917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6833","nonce":1204611917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1204611917},{"type":"v1","addr":"172.21.15.31:6831","nonce":1204611917}]},"public_addr":"172.21.15.31:6827/1204611917","cluster_addr":"172.21.15.31:6829/1204611917","heartbeat_back_addr":"172.21.15.31:6833/1204611917","heartbeat_front_addr":"172.21.15.31:6831/1204611917","state":["exists","up"]},{"osd":10,"uuid":"d0ae22f5-028f-4723-9374-c88933908b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6825","nonce":3486886499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6827","nonce":3486886499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6831","nonce":3486886499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3486886499},{"type":"v1","addr":"172.21.15.165:6829","nonce":3486886499}]},"public_addr":"172.21.15.165:6825/3486886499","cluster_addr":"172.21.15.165:6827/3486886499","heartbeat_back_addr":"172.21.15.165:6831/3486886499","heartbeat_front_addr":"172.21.15.165:6829/3486886499","state":["exists","up"]},{"osd":11,"uuid":"0fb36d0d-9be8-49f1-b950-2e61cfcef7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6825","nonce":3803011013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6827","nonce":3803011013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6831","nonce":3803011013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":3803011013},{"type":"v1","addr":"172.21.15.178:6829","nonce":3803011013}]},"public_addr":"172.21.15.178:6825/3803011013","cluster_addr":"172.21.15.178:6827/3803011013","heartbeat_back_addr":"172.21.15.178:6831/3803011013","heartbeat_front_addr":"172.21.15.178:6829/3803011013","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:21.587135+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:45:46.713406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:10.854150+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:46:35.655712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:00.937024+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:26.122653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:47:53.178549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:18.393266+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:48:43.020006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:09.163999+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:34.769999+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:49:59.299781+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/944835008":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6801/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/262827684":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/4029989165":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6801/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6801/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:6800/1175221317":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/814395253":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/3796123698":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/1104840323":"2024-09-22T02:42:44.994232+0000","172.21.15.31:0/633109246":"2024-09-22T02:42:44.994232+0000","172.21.15.31:6800/1876173055":"2024-09-22T02:43:13.322815+0000","172.21.15.31:0/3425970992":"2024-09-22T02:44:15.049623+0000","172.21.15.31:6800/4246092214":"2024-09-22T02:44:15.049623+0000","172.21.15.31:0/1565999986":"2024-09-22T02:44:15.049623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:03:41.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53180b6700 msgr2=0x7f53180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:41.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53180b6700 0x7f53180b8c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5324006010 tx=0x7f5324000bc0 comp rx=0 tx=0).stop 2024-09-21T03:03:41.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 msgr2=0x7f533019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:41.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 0x7f533019c6e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f532000eb90 tx=0x7f532000c6a0 comp rx=0 tx=0).stop 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 shutdown_connections 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f53180b6700 0x7f53180b8c10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53301094c0 0x7f533019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5330109e10 0x7f53301a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 --2- 172.21.15.31:0/1273987357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5330103b80 0x7f533019c6e0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 >> 172.21.15.31:0/1273987357 conn(0x7f53300ff5d0 msgr2=0x7f5330106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 shutdown_connections 2024-09-21T03:03:41.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:41.149+0000 7f5338676640 1 -- 172.21.15.31:0/1273987357 wait complete. 2024-09-21T03:03:41.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:41 smithi165 ceph-mon[34818]: pgmap v656: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T03:03:41.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:41 smithi178 ceph-mon[34759]: pgmap v656: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T03:03:41.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:03:42.120 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:42.241 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1273987357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:03:42.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1273987357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:03:42.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1273987357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 -- 172.21.15.31:0/87163052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 msgr2=0x7f370810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 --2- 172.21.15.31:0/87163052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370810d910 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009900 tx=0x7f36f802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 -- 172.21.15.31:0/87163052 shutdown_connections 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 --2- 172.21.15.31:0/87163052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f370810de50 0x7f3708110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 --2- 172.21.15.31:0/87163052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370810d910 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 --2- 172.21.15.31:0/87163052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37081048a0 0x7f3708104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 -- 172.21.15.31:0/87163052 >> 172.21.15.31:0/87163052 conn(0x7f3708079740 msgr2=0x7f3708079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 -- 172.21.15.31:0/87163052 shutdown_connections 2024-09-21T03:03:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.639+0000 7f370fc68640 1 -- 172.21.15.31:0/87163052 wait complete. 2024-09-21T03:03:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 Processor -- start 2024-09-21T03:03:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 -- start start 2024-09-21T03:03:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37081048a0 0x7f370819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f370810de50 0x7f370819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3708113ae0 con 0x7f37081052d0 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3708113960 con 0x7f370810de50 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370fc68640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3708113c60 con 0x7f37081048a0 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370d203640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370d203640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41666/0 (socket says 172.21.15.31:41666) 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.640+0000 7f370d203640 1 -- 172.21.15.31:0/1339059386 learned_addr learned my addr 172.21.15.31:0/1339059386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370e205640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f370810de50 0x7f370819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370da04640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37081048a0 0x7f370819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370d203640 1 -- 172.21.15.31:0/1339059386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37081048a0 msgr2=0x7f370819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370d203640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37081048a0 0x7f370819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370d203640 1 -- 172.21.15.31:0/1339059386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f370810de50 msgr2=0x7f370819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370d203640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f370810de50 0x7f370819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370d203640 1 -- 172.21.15.31:0/1339059386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37081a0440 con 0x7f37081052d0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370d203640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370819aa70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005ab0 tx=0x7f36f8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f803f070 con 0x7f37081052d0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f8031b50 con 0x7f37081052d0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37081a06d0 con 0x7f37081052d0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8035030 con 0x7f37081052d0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.641+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a6e90 con 0x7f37081052d0 2024-09-21T03:03:42.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.642+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370806c1a0 con 0x7f37081052d0 2024-09-21T03:03:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.644+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f36f8042070 con 0x7f37081052d0 2024-09-21T03:03:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.645+0000 7f36feffd640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f36f40b6680 0x7f36f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.645+0000 7f370da04640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f36f40b6680 0x7f36f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.645+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8528+0+0 (secure 0 0 0) 0x7f36f8108110 con 0x7f37081052d0 2024-09-21T03:03:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.645+0000 7f370da04640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f36f40b6680 0x7f36f40b8b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f370806c8a0 tx=0x7f3704005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.648+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36f80d2a40 con 0x7f37081052d0 2024-09-21T03:03:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.877+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3708104cb0 con 0x7f37081052d0 2024-09-21T03:03:42.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.879+0000 7f36feffd640 1 -- 172.21.15.31:0/1339059386 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f36f80d7980 con 0x7f37081052d0 2024-09-21T03:03:42.881 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f36f40b6680 msgr2=0x7f36f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f36f40b6680 0x7f36f40b8b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f370806c8a0 tx=0x7f3704005f90 comp rx=0 tx=0).stop 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 msgr2=0x7f370819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370819aa70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005ab0 tx=0x7f36f8003b90 comp rx=0 tx=0).stop 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 shutdown_connections 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f36f40b6680 0x7f36f40b8b90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f370810de50 0x7f370819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37081052d0 0x7f370819aa70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 --2- 172.21.15.31:0/1339059386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37081048a0 0x7f370819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 >> 172.21.15.31:0/1339059386 conn(0x7f3708079740 msgr2=0x7f37081069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 shutdown_connections 2024-09-21T03:03:42.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:42.883+0000 7f370fc68640 1 -- 172.21.15.31:0/1339059386 wait complete. 2024-09-21T03:03:43.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a[31205]: 2024-09-21T03:03:42.878+0000 7f66f5d78640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:03:43.226 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: pgmap v657: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1339059386' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/1339059386' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:43 smithi031 ceph-mon[31209]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a[74267]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a[74267]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-mds[74271]: -- 172.21.15.31:0/2778338726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56265122f860 con 0x5626553d7600 2024-09-21T03:03:43.227 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a[74267]: starting mds.a at 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g[73903]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g[73903]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-mds[73919]: -- 172.21.15.31:0/2175843623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc79e7d860 con 0x55fc7e025600 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:03:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g[73903]: starting mds.g at 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l[63406]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l[63406]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.228 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l[63406]: starting mds.l at 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i[64221]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i[64221]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-mds[64225]: -- 172.21.15.178:0/2845060062 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577ab67bc20 con 0x5577af823a80 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i[64221]: starting mds.i at 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c[63941]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c[63941]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-mds[63945]: -- 172.21.15.178:0/3883142277 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3fb05b860 con 0x55d3ff203180 2024-09-21T03:03:43.229 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c[63941]: starting mds.c at 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: pgmap v657: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1339059386' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/1339059386' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:43 smithi178 ceph-mon[34759]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f[63673]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.230 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f[63673]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.231 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-mds[63677]: -- 172.21.15.178:0/4260421469 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fce8fe4ea0 con 0x55fced183180 2024-09-21T03:03:43.231 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-mds[63677]: -- 172.21.15.178:0/4260421469 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fce8fdb860 con 0x55fced183180 2024-09-21T03:03:43.231 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:03:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f[63673]: starting mds.f at 2024-09-21T03:03:43.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: pgmap v657: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-09-21T03:03:43.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1339059386' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/1339059386' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:43 smithi165 ceph-mon[34818]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-mds[63771]: -- 172.21.15.165:0/3421686179 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588f8af5860 con 0x5588fcc9d600 2024-09-21T03:03:43.442 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: starting mds.b at 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e[63212]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e[63212]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-mds[63216]: -- 172.21.15.165:0/1127976143 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a793a33860 con 0x55a797bdba80 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e[63212]: starting mds.e at 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h[63497]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h[63497]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-mds[63501]: -- 172.21.15.165:0/1897290974 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a080c93860 con 0x55a084e3b600 2024-09-21T03:03:43.443 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h[63497]: starting mds.h at 2024-09-21T03:03:43.444 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k[62931]: ignoring --setuser ceph since I am not root 2024-09-21T03:03:43.444 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k[62931]: ignoring --setgroup ceph since I am not root 2024-09-21T03:03:43.444 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-mds[62935]: -- 172.21.15.165:0/2694799216 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b30b6b9860 con 0x55b30f861a80 2024-09-21T03:03:43.444 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:03:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k[62931]: starting mds.k at 2024-09-21T03:03:43.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:03:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 -- 172.21.15.31:0/2364961845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488109e10 msgr2=0x7f5488110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 --2- 172.21.15.31:0/2364961845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488109e10 0x7f5488110640 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5480009ca0 tx=0x7f548002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 -- 172.21.15.31:0/2364961845 shutdown_connections 2024-09-21T03:03:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 --2- 172.21.15.31:0/2364961845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54881094c0 0x7f54881098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 --2- 172.21.15.31:0/2364961845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488109e10 0x7f5488110640 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 --2- 172.21.15.31:0/2364961845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5488101fe0 0x7f54881023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 -- 172.21.15.31:0/2364961845 >> 172.21.15.31:0/2364961845 conn(0x7f5488077bc0 msgr2=0x7f5488078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 -- 172.21.15.31:0/2364961845 shutdown_connections 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 -- 172.21.15.31:0/2364961845 wait complete. 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.306+0000 7f548daf7640 1 Processor -- start 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 -- start start 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 0x7f548819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:44.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54881094c0 0x7f548819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5488109e10 0x7f548819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5488113a80 con 0x7f5488101fe0 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5488113900 con 0x7f5488109e10 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f548daf7640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5488113c00 con 0x7f54881094c0 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f54877fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 0x7f548819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f54877fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 0x7f548819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41722/0 (socket says 172.21.15.31:41722) 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f54877fe640 1 -- 172.21.15.31:0/722742709 learned_addr learned my addr 172.21.15.31:0/722742709 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f547effd640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54881094c0 0x7f548819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.307+0000 7f5487fff640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5488109e10 0x7f548819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54877fe640 1 -- 172.21.15.31:0/722742709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54881094c0 msgr2=0x7f548819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:44.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54877fe640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54881094c0 0x7f548819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54877fe640 1 -- 172.21.15.31:0/722742709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5488109e10 msgr2=0x7f548819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54877fe640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5488109e10 0x7f548819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54877fe640 1 -- 172.21.15.31:0/722742709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54881a0400 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54877fe640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 0x7f548819a490 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f547800e9f0 tx=0x7f547800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800cd80 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54881a06f0 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54780108e0 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478018980 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.308+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881a6e50 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.309+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54881023f0 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.311+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f5478010430 con 0x7f5488101fe0 2024-09-21T03:03:44.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.312+0000 7f54857fa640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f54740b6680 0x7f54740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:44.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.312+0000 7f547effd640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f54740b6680 0x7f54740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:44.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.312+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8914+0+0 (secure 0 0 0) 0x7f54780e8060 con 0x7f5488101fe0 2024-09-21T03:03:44.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.313+0000 7f547effd640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f54740b6680 0x7f54740b8b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f548806b170 tx=0x7f547000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:44.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.315+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54780b2800 con 0x7f5488101fe0 2024-09-21T03:03:44.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:44.541+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f548802e640 con 0x7f5488101fe0 2024-09-21T03:03:45.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.042+0000 7f54857fa640 1 -- 172.21.15.31:0/722742709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f54780b7740 con 0x7f5488101fe0 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f54740b6680 msgr2=0x7f54740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f54740b6680 0x7f54740b8b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f548806b170 tx=0x7f547000b390 comp rx=0 tx=0).stop 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 msgr2=0x7f548819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 0x7f548819a490 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f547800e9f0 tx=0x7f547800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 shutdown_connections 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f54740b6680 0x7f54740b8b90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54881094c0 0x7f548819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5488109e10 0x7f548819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 --2- 172.21.15.31:0/722742709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101fe0 0x7f548819a490 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 >> 172.21.15.31:0/722742709 conn(0x7f5488077bc0 msgr2=0x7f5488107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 shutdown_connections 2024-09-21T03:03:45.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:45.045+0000 7f548daf7640 1 -- 172.21.15.31:0/722742709 wait complete. 2024-09-21T03:03:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: pgmap v659: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-21T03:03:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.31:6838/4111378057,v1:172.21.15.31:6839/4111378057] up:boot 2024-09-21T03:03:45.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6832/906084760,v1:172.21.15.165:6833/906084760] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.31:6840/2773932055,v1:172.21.15.31:6841/2773932055] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6834/597276258,v1:172.21.15.165:6835/597276258] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6832/2212083015,v1:172.21.15.178:6834/2212083015] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6833/3842513797,v1:172.21.15.178:6835/3842513797] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6836/3411106752,v1:172.21.15.165:6837/3411106752] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6836/1461784844,v1:172.21.15.178:6838/1461784844] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.178:6837/341400669,v1:172.21.15.178:6839/341400669] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: mds.? [v2:172.21.15.165:6838/2586043955,v1:172.21.15.165:6839/2586043955] up:boot 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:03:45.192 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:03:45.193 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:44 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/722742709' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: pgmap v659: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.31:6838/4111378057,v1:172.21.15.31:6839/4111378057] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6832/906084760,v1:172.21.15.165:6833/906084760] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.31:6840/2773932055,v1:172.21.15.31:6841/2773932055] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6834/597276258,v1:172.21.15.165:6835/597276258] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6832/2212083015,v1:172.21.15.178:6834/2212083015] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6833/3842513797,v1:172.21.15.178:6835/3842513797] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6836/3411106752,v1:172.21.15.165:6837/3411106752] up:boot 2024-09-21T03:03:45.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6836/1461784844,v1:172.21.15.178:6838/1461784844] up:boot 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.178:6837/341400669,v1:172.21.15.178:6839/341400669] up:boot 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: mds.? [v2:172.21.15.165:6838/2586043955,v1:172.21.15.165:6839/2586043955] up:boot 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:03:45.229 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:44 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/722742709' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: pgmap v659: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.31:6838/4111378057,v1:172.21.15.31:6839/4111378057] up:boot 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6832/906084760,v1:172.21.15.165:6833/906084760] up:boot 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.31:6840/2773932055,v1:172.21.15.31:6841/2773932055] up:boot 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6834/597276258,v1:172.21.15.165:6835/597276258] up:boot 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6832/2212083015,v1:172.21.15.178:6834/2212083015] up:boot 2024-09-21T03:03:45.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6833/3842513797,v1:172.21.15.178:6835/3842513797] up:boot 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6836/3411106752,v1:172.21.15.165:6837/3411106752] up:boot 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6836/1461784844,v1:172.21.15.178:6838/1461784844] up:boot 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.178:6837/341400669,v1:172.21.15.178:6839/341400669] up:boot 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: mds.? [v2:172.21.15.165:6838/2586043955,v1:172.21.15.165:6839/2586043955] up:boot 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:03:45.335 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='mgr.14223 172.21.15.31:0/3788205885' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T03:03:45.336 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:44 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/722742709' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:03:45.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:03:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:46.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:45 smithi031 ceph-mon[31209]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:03:46.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:45 smithi031 ceph-mon[31209]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:03:46.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:45 smithi031 ceph-mon[31209]: Cluster is now healthy 2024-09-21T03:03:46.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:45 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/722742709' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:03:46.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:45 smithi031 ceph-mon[31209]: fsmap 12 up:standby 2024-09-21T03:03:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:45 smithi165 ceph-mon[34818]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:03:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:45 smithi165 ceph-mon[34818]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:03:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:45 smithi165 ceph-mon[34818]: Cluster is now healthy 2024-09-21T03:03:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:45 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/722742709' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:03:46.191 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:45 smithi165 ceph-mon[34818]: fsmap 12 up:standby 2024-09-21T03:03:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:45 smithi178 ceph-mon[34759]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:03:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:45 smithi178 ceph-mon[34759]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:03:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:45 smithi178 ceph-mon[34759]: Cluster is now healthy 2024-09-21T03:03:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:45 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/722742709' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:03:46.228 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:45 smithi178 ceph-mon[34759]: fsmap 12 up:standby 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 -- 172.21.15.31:0/2111446189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 msgr2=0x7f6e201055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 --2- 172.21.15.31:0/2111446189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e201055f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009900 tx=0x7f6e1402a410 comp rx=0 tx=0).stop 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 -- 172.21.15.31:0/2111446189 shutdown_connections 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 --2- 172.21.15.31:0/2111446189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e20105b30 0x7f6e2010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 --2- 172.21.15.31:0/2111446189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e201055f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 --2- 172.21.15.31:0/2111446189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e20103210 0x7f6e20103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 -- 172.21.15.31:0/2111446189 >> 172.21.15.31:0/2111446189 conn(0x7f6e20079f90 msgr2=0x7f6e2007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 -- 172.21.15.31:0/2111446189 shutdown_connections 2024-09-21T03:03:46.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.462+0000 7f6e261f0640 1 -- 172.21.15.31:0/2111446189 wait complete. 2024-09-21T03:03:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 Processor -- start 2024-09-21T03:03:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 -- start start 2024-09-21T03:03:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e20103210 0x7f6e2019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e2019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e20105b30 0x7f6e2019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20113b10 con 0x7f6e201051e0 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20113990 con 0x7f6e20105b30 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e261f0640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20113c90 con 0x7f6e20103210 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e2019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e2019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41742/0 (socket says 172.21.15.31:41742) 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1effd640 1 -- 172.21.15.31:0/913989185 learned_addr learned my addr 172.21.15.31:0/913989185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1ffff640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e20105b30 0x7f6e2019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1f7fe640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e20103210 0x7f6e2019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1effd640 1 -- 172.21.15.31:0/913989185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e20103210 msgr2=0x7f6e2019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1effd640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e20103210 0x7f6e2019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.463+0000 7f6e1effd640 1 -- 172.21.15.31:0/913989185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e20105b30 msgr2=0x7f6e2019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e1effd640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e20105b30 0x7f6e2019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e1effd640 1 -- 172.21.15.31:0/913989185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e201a0390 con 0x7f6e201051e0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e1effd640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e2019aa80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6e14031a90 tx=0x7f6e14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1403f030 con 0x7f6e201051e0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e14003cb0 con 0x7f6e201051e0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201a0620 con 0x7f6e201051e0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14035070 con 0x7f6e201051e0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.464+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a6ea0 con 0x7f6e201051e0 2024-09-21T03:03:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.465+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e20103620 con 0x7f6e201051e0 2024-09-21T03:03:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.467+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f6e14042030 con 0x7f6e201051e0 2024-09-21T03:03:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.467+0000 7f6e1cff9640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6e080b6730 0x7f6e080b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.468+0000 7f6e1f7fe640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6e080b6730 0x7f6e080b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.468+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8914+0+0 (secure 0 0 0) 0x7f6e14108290 con 0x7f6e201051e0 2024-09-21T03:03:46.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.468+0000 7f6e1f7fe640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6e080b6730 0x7f6e080b8c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000ba40 tx=0x7f6e10005cc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.471+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e140ce650 con 0x7f6e201051e0 2024-09-21T03:03:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:46.659+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e2019b610 con 0x7f6e201051e0 2024-09-21T03:03:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.058+0000 7f6e1cff9640 1 -- 172.21.15.31:0/913989185 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7f6e140d2a30 con 0x7f6e201051e0 2024-09-21T03:03:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:03:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6e080b6730 msgr2=0x7f6e080b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6e080b6730 0x7f6e080b8c40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000ba40 tx=0x7f6e10005cc0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 msgr2=0x7f6e2019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e2019aa80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6e14031a90 tx=0x7f6e14004270 comp rx=0 tx=0).stop 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 shutdown_connections 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f6e080b6730 0x7f6e080b8c40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e20105b30 0x7f6e2019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e201051e0 0x7f6e2019aa80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 --2- 172.21.15.31:0/913989185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e20103210 0x7f6e2019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 >> 172.21.15.31:0/913989185 conn(0x7f6e20079f90 msgr2=0x7f6e20106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 shutdown_connections 2024-09-21T03:03:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:47.061+0000 7f6e261f0640 1 -- 172.21.15.31:0/913989185 wait complete. 2024-09-21T03:03:47.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:47 smithi031 ceph-mon[31209]: pgmap v660: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-21T03:03:47.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:47 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/913989185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:03:47.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:47 smithi165 ceph-mon[34818]: pgmap v660: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-21T03:03:47.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:47 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/913989185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:03:47.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:47 smithi178 ceph-mon[34759]: pgmap v660: 97 pgs: 97 active+clean; 2.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-21T03:03:47.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:47 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/913989185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:03:47.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:03:47.994 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:48.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:48 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/913989185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:03:48.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:48 smithi031 ceph-mon[31209]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:03:48.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:48 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/913989185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:03:48.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:48 smithi165 ceph-mon[34818]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:03:48.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:48 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/913989185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:03:48.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:48 smithi178 ceph-mon[34759]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 -- 172.21.15.31:0/2409646257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 msgr2=0x7f380c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 --2- 172.21.15.31:0/2409646257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c10f520 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3804009ca0 tx=0x7f380402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 -- 172.21.15.31:0/2409646257 shutdown_connections 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 --2- 172.21.15.31:0/2409646257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c10f520 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 --2- 172.21.15.31:0/2409646257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f380c1083a0 0x7f380c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 --2- 172.21.15.31:0/2409646257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102a60 0x7f380c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 -- 172.21.15.31:0/2409646257 >> 172.21.15.31:0/2409646257 conn(0x7f380c079730 msgr2=0x7f380c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 -- 172.21.15.31:0/2409646257 shutdown_connections 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.493+0000 7f3812293640 1 -- 172.21.15.31:0/2409646257 wait complete. 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 Processor -- start 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 -- start start 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102a60 0x7f380c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f380c1083a0 0x7f380c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f380c113a70 con 0x7f380c108cf0 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1138f0 con 0x7f380c1083a0 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.494+0000 7f3812293640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f380c113bf0 con 0x7f380c102a60 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41754/0 (socket says 172.21.15.31:41754) 2024-09-21T03:03:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 -- 172.21.15.31:0/2329766999 learned_addr learned my addr 172.21.15.31:0/2329766999 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3803fff640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f380c1083a0 0x7f380c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f380bfff640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102a60 0x7f380c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 -- 172.21.15.31:0/2329766999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102a60 msgr2=0x7f380c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102a60 0x7f380c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 -- 172.21.15.31:0/2329766999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f380c1083a0 msgr2=0x7f380c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f380c1083a0 0x7f380c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 -- 172.21.15.31:0/2329766999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c1a03f0 con 0x7f380c108cf0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3810830640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c19fcc0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3804005cc0 tx=0x7f380402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804003bf0 con 0x7f380c108cf0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3804003d90 con 0x7f380c108cf0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380402fbc0 con 0x7f380c108cf0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c1a0680 con 0x7f380c108cf0 2024-09-21T03:03:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.495+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1a6e40 con 0x7f380c108cf0 2024-09-21T03:03:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.497+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c102e70 con 0x7f380c108cf0 2024-09-21T03:03:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.498+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 27) ==== 154643+0+0 (secure 0 0 0) 0x7f380402fd60 con 0x7f380c108cf0 2024-09-21T03:03:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.499+0000 7f3809ffb640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37f80b6680 0x7f37f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.499+0000 7f380bfff640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37f80b6680 0x7f37f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:48.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.499+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8360+0+0 (secure 0 0 0) 0x7f3804108c60 con 0x7f380c108cf0 2024-09-21T03:03:48.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.499+0000 7f380bfff640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37f80b6680 0x7f37f80b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005fd0 tx=0x7f37fc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:48.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.502+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38040cf250 con 0x7f380c108cf0 2024-09-21T03:03:48.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:48.691+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f380c19b5b0 con 0x7f380c108cf0 2024-09-21T03:03:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.072+0000 7f3809ffb640 1 -- 172.21.15.31:0/2329766999 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7f380c19b5b0 con 0x7f380c108cf0 2024-09-21T03:03:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-09-21T03:03:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.075+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37f80b6680 msgr2=0x7f37f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.075+0000 7f3812293640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37f80b6680 0x7f37f80b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005fd0 tx=0x7f37fc00b390 comp rx=0 tx=0).stop 2024-09-21T03:03:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.075+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 msgr2=0x7f380c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.075+0000 7f3812293640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c19fcc0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3804005cc0 tx=0x7f380402f380 comp rx=0 tx=0).stop 2024-09-21T03:03:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.076+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 shutdown_connections 2024-09-21T03:03:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:6800/44765732,v1:172.21.15.31:6801/44765732] conn(0x7f37f80b6680 0x7f37f80b8b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f380c108cf0 0x7f380c19fcc0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f380c1083a0 0x7f380c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 --2- 172.21.15.31:0/2329766999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f380c102a60 0x7f380c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 >> 172.21.15.31:0/2329766999 conn(0x7f380c079730 msgr2=0x7f380c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 shutdown_connections 2024-09-21T03:03:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T03:03:49.077+0000 7f3812293640 1 -- 172.21.15.31:0/2329766999 wait complete. 2024-09-21T03:03:49.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:49 smithi031 ceph-mon[31209]: pgmap v662: 65 pgs: 65 active+clean; 577 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:03:49.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:49 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2329766999' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:03:49.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:49 smithi165 ceph-mon[34818]: pgmap v662: 65 pgs: 65 active+clean; 577 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:03:49.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:49 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2329766999' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:03:49.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:49 smithi178 ceph-mon[34759]: pgmap v662: 65 pgs: 65 active+clean; 577 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:03:49.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:49 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2329766999' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:03:49.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:03:49.764 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:03:49.796 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:03:49.829 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:03:49.829 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:03:49.847 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:03:49.864 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:03:49.887 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:03:49.887 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:03:49.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:03:50.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:49 smithi031 systemd[1]: Stopping Ceph mon.a for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 ceph-mon[31209]: from='client.? 172.21.15.31:0/2329766999' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:03:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 ceph-mon[31209]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:03:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 ceph-mon[31209]: pgmap v664: 1 pgs: 1 active+clean; 577 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:03:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a[31205]: 2024-09-21T03:03:50.214+0000 7f66fb583640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:03:50.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a[31205]: 2024-09-21T03:03:50.214+0000 7f66fb583640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:03:50.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:50 smithi165 ceph-mon[34818]: from='client.? 172.21.15.31:0/2329766999' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:03:50.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:50 smithi165 ceph-mon[34818]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:03:50.441 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:50 smithi165 ceph-mon[34818]: pgmap v664: 1 pgs: 1 active+clean; 577 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:03:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:50 smithi178 ceph-mon[34759]: from='client.? 172.21.15.31:0/2329766999' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:03:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:50 smithi178 ceph-mon[34759]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:03:50.478 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:50 smithi178 ceph-mon[34759]: pgmap v664: 1 pgs: 1 active+clean; 577 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T03:03:50.779 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 podman[109239]: 2024-09-21 03:03:50.4399683 +0000 UTC m=+0.345846024 container died ecef9b21124f577615900b8fe09289a90c81ead6b1de62e2fa7ac7f53c59cfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:03:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 podman[109239]: 2024-09-21 03:03:50.881399284 +0000 UTC m=+0.787277052 container cleanup ecef9b21124f577615900b8fe09289a90c81ead6b1de62e2fa7ac7f53c59cfa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:03:51.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 03:03:50 smithi031 bash[109239]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:03:51.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:03:51.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:03:51.373 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:03:51.374 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:03:51.374 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:03:51.691 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:51 smithi165 systemd[1]: Stopping Ceph mon.b for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:52.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b[34814]: 2024-09-21T03:03:51.710+0000 7fbc66f52640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:03:52.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b[34814]: 2024-09-21T03:03:51.710+0000 7fbc66f52640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:03:52.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:51 smithi165 podman[67067]: 2024-09-21 03:03:51.783329686 +0000 UTC m=+0.214784769 container died b9f87f32b7ceb1f234a3cd5c5a955c52834ca8bd06fa8c9f8335db293d84b6d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:03:52.413 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:52 smithi165 podman[67067]: 2024-09-21 03:03:52.219145281 +0000 UTC m=+0.650600364 container cleanup b9f87f32b7ceb1f234a3cd5c5a955c52834ca8bd06fa8c9f8335db293d84b6d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:03:52.413 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:52 smithi165 bash[67067]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:03:52.647 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:03:52.669 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:52 smithi165 podman[67079]: 2024-09-21 03:03:52.412491875 +0000 UTC m=+0.623932157 container remove b9f87f32b7ceb1f234a3cd5c5a955c52834ca8bd06fa8c9f8335db293d84b6d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:03:52.669 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:52 smithi165 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.b.service: Deactivated successfully. 2024-09-21T03:03:52.669 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:52 smithi165 systemd[1]: Stopped Ceph mon.b for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T03:03:52.669 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 21 03:03:52 smithi165 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.b.service: Consumed 13.117s CPU time. 2024-09-21T03:03:52.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:03:52.693 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:03:52.693 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:03:52.693 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:03:52.978 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:52 smithi178 systemd[1]: Stopping Ceph mon.c for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:53.377 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c[34755]: 2024-09-21T03:03:52.990+0000 7fa051781640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:03:53.377 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c[34755]: 2024-09-21T03:03:52.990+0000 7fa051781640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:03:53.377 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:53 smithi178 podman[67615]: 2024-09-21 03:03:53.092992601 +0000 UTC m=+0.207441602 container died cd588d01e868c640bb855b4847552caa5f39aabe5bb903ed3edeb0a06b324376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:03:53.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:53 smithi178 podman[67615]: 2024-09-21 03:03:53.483760817 +0000 UTC m=+0.598209816 container cleanup cd588d01e868c640bb855b4847552caa5f39aabe5bb903ed3edeb0a06b324376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:03:53.728 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 21 03:03:53 smithi178 bash[67615]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:03:53.868 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:03:53.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:03:53.914 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:03:53.914 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:03:53.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:03:54.229 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 03:03:53 smithi031 systemd[1]: Stopping Ceph mgr.x for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:54.584 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 03:03:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x[31434]: 2024-09-21T03:03:54.227+0000 7f097cfd6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:03:54.584 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 03:03:54 smithi031 podman[109357]: 2024-09-21 03:03:54.2691643 +0000 UTC m=+0.155567496 container died 1b306d23b1662a15555937d3c9c6076c353bd3b04f5f0a0b8d9e22a4762bbc34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T03:03:55.015 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 03:03:54 smithi031 podman[109357]: 2024-09-21 03:03:54.728928464 +0000 UTC m=+0.615331658 container cleanup 1b306d23b1662a15555937d3c9c6076c353bd3b04f5f0a0b8d9e22a4762bbc34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:03:55.015 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 03:03:54 smithi031 bash[109357]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:03:55.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:03:55.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:03:55.180 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:03:55.180 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:03:55.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:03:55.486 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 03:03:55 smithi165 systemd[1]: Stopping Ceph mgr.y for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:55.766 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 03:03:55 smithi165 podman[67190]: 2024-09-21 03:03:55.486160708 +0000 UTC m=+0.115530728 container died 41c6c7592df232b8c598c0b574ae4bf747c4ae9d8b800030214c4ad7f6a81769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T03:03:56.117 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 03:03:55 smithi165 podman[67190]: 2024-09-21 03:03:55.865579671 +0000 UTC m=+0.494949697 container cleanup 41c6c7592df232b8c598c0b574ae4bf747c4ae9d8b800030214c4ad7f6a81769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:03:56.117 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 21 03:03:55 smithi165 bash[67190]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:03:56.299 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:03:56.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:03:56.333 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:03:56.334 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:03:56.334 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:03:56.654 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 03:03:56 smithi178 systemd[1]: Stopping Ceph mgr.z for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:56.950 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 03:03:56 smithi178 podman[67736]: 2024-09-21 03:03:56.65357006 +0000 UTC m=+0.126786000 container died 9fc30afc7849b264bd92e2c082d09a999f1689d65abc0c248d86972686c729b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T03:03:57.228 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 03:03:57 smithi178 podman[67736]: 2024-09-21 03:03:57.048503634 +0000 UTC m=+0.521719573 container cleanup 9fc30afc7849b264bd92e2c082d09a999f1689d65abc0c248d86972686c729b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:03:57.228 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 21 03:03:57 smithi178 bash[67736]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:03:57.472 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:03:57.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:03:57.506 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:03:57.506 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:03:57.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:03:57.826 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:03:57 smithi031 systemd[1]: Stopping Ceph osd.0 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:03:58.084 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:03:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0[47986]: 2024-09-21T03:03:57.824+0000 7f68797c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:03:58.084 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:03:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0[47986]: 2024-09-21T03:03:57.824+0000 7f68797c5640 -1 osd.0 89 *** Got signal Terminated *** 2024-09-21T03:03:58.084 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:03:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0[47986]: 2024-09-21T03:03:57.824+0000 7f68797c5640 -1 osd.0 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:03.444 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:03 smithi031 podman[109478]: 2024-09-21 03:04:03.14822672 +0000 UTC m=+5.441661038 container died df18da783d6dbf4bd80c62fa3ef18c7ce6a4fb79d9e834f211ab05b57c26cab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:04:03.834 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:03 smithi031 podman[109478]: 2024-09-21 03:04:03.642644207 +0000 UTC m=+5.936078567 container cleanup df18da783d6dbf4bd80c62fa3ef18c7ce6a4fb79d9e834f211ab05b57c26cab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:03.834 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:03 smithi031 bash[109478]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:04:04.274 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:03 smithi031 podman[109496]: 2024-09-21 03:04:03.85257691 +0000 UTC m=+0.694202898 container remove df18da783d6dbf4bd80c62fa3ef18c7ce6a4fb79d9e834f211ab05b57c26cab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:04:04.584 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:04 smithi031 podman[109583]: 2024-09-21 03:04:04.176941503 +0000 UTC m=+0.016745161 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:04.584 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:04 smithi031 podman[109583]: 2024-09-21 03:04:04.348339234 +0000 UTC m=+0.188142888 container create 9fb8cd3bec96896ca51ca18b21d4f4c9bed01ed5ef93fe9879718f04f38521b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:04:05.084 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:04 smithi031 podman[109583]: 2024-09-21 03:04:04.618265702 +0000 UTC m=+0.458069357 container init 9fb8cd3bec96896ca51ca18b21d4f4c9bed01ed5ef93fe9879718f04f38521b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:04:05.084 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:04 smithi031 podman[109583]: 2024-09-21 03:04:04.621844015 +0000 UTC m=+0.461647671 container start 9fb8cd3bec96896ca51ca18b21d4f4c9bed01ed5ef93fe9879718f04f38521b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T03:04:05.085 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:04 smithi031 podman[109583]: 2024-09-21 03:04:04.697885676 +0000 UTC m=+0.537689325 container attach 9fb8cd3bec96896ca51ca18b21d4f4c9bed01ed5ef93fe9879718f04f38521b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:04:05.085 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 03:04:04 smithi031 podman[109583]: 2024-09-21 03:04:04.815903877 +0000 UTC m=+0.655707528 container died 9fb8cd3bec96896ca51ca18b21d4f4c9bed01ed5ef93fe9879718f04f38521b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:04:05.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:04:05.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:05.601 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:04:05.602 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:04:05.602 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:04:05.941 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:05 smithi165 systemd[1]: Stopping Ceph osd.1 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:06.441 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1[39455]: 2024-09-21T03:04:06.060+0000 7f78d60b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:06.441 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1[39455]: 2024-09-21T03:04:06.060+0000 7f78d60b5640 -1 osd.1 89 *** Got signal Terminated *** 2024-09-21T03:04:06.441 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1[39455]: 2024-09-21T03:04:06.060+0000 7f78d60b5640 -1 osd.1 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:11.863 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:11 smithi165 podman[67324]: 2024-09-21 03:04:11.561595674 +0000 UTC m=+5.765993377 container died 3bef73e04cb3f58d08cab6b7beeec736304c85780ab7af16f1d6786cfec311f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:04:12.192 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:11 smithi165 podman[67324]: 2024-09-21 03:04:11.952795754 +0000 UTC m=+6.157193502 container cleanup 3bef73e04cb3f58d08cab6b7beeec736304c85780ab7af16f1d6786cfec311f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:04:12.192 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:11 smithi165 bash[67324]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:04:12.578 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:12 smithi165 podman[67340]: 2024-09-21 03:04:12.24579929 +0000 UTC m=+0.679922831 container remove 3bef73e04cb3f58d08cab6b7beeec736304c85780ab7af16f1d6786cfec311f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T03:04:12.833 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:12 smithi165 podman[67429]: 2024-09-21 03:04:12.480405188 +0000 UTC m=+0.015844371 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:12.833 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:12 smithi165 podman[67429]: 2024-09-21 03:04:12.580054753 +0000 UTC m=+0.115493980 container create 8714c6889cdb9040044e2dd69d41dd3f1b9d0da7764aeefafacc7d855e9e5719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T03:04:12.833 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:12 smithi165 podman[67429]: 2024-09-21 03:04:12.832539345 +0000 UTC m=+0.367978540 container init 8714c6889cdb9040044e2dd69d41dd3f1b9d0da7764aeefafacc7d855e9e5719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:13.192 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:12 smithi165 podman[67429]: 2024-09-21 03:04:12.835986848 +0000 UTC m=+0.371426033 container start 8714c6889cdb9040044e2dd69d41dd3f1b9d0da7764aeefafacc7d855e9e5719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:04:13.192 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:12 smithi165 podman[67429]: 2024-09-21 03:04:12.941031993 +0000 UTC m=+0.476471176 container attach 8714c6889cdb9040044e2dd69d41dd3f1b9d0da7764aeefafacc7d855e9e5719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:04:13.192 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 21 03:04:13 smithi165 podman[67429]: 2024-09-21 03:04:13.024128644 +0000 UTC m=+0.559567834 container died 8714c6889cdb9040044e2dd69d41dd3f1b9d0da7764aeefafacc7d855e9e5719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:13.690 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:04:13.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:13.729 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:04:13.729 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:04:13.729 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:04:14.037 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:13 smithi178 systemd[1]: Stopping Ceph osd.2 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:14.478 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2[39921]: 2024-09-21T03:04:14.035+0000 7f1df26f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:14.478 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2[39921]: 2024-09-21T03:04:14.035+0000 7f1df26f9640 -1 osd.2 89 *** Got signal Terminated *** 2024-09-21T03:04:14.478 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2[39921]: 2024-09-21T03:04:14.035+0000 7f1df26f9640 -1 osd.2 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:19.728 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:19 smithi178 podman[67869]: 2024-09-21 03:04:19.438501622 +0000 UTC m=+5.513823009 container died 824a66cda9e28a2c8d42a644b0d14c100fcb568145ebdc61536c5ca53f3414e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T03:04:20.079 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:19 smithi178 podman[67869]: 2024-09-21 03:04:19.836074807 +0000 UTC m=+5.911396195 container cleanup 824a66cda9e28a2c8d42a644b0d14c100fcb568145ebdc61536c5ca53f3414e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:20.080 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:19 smithi178 bash[67869]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:04:20.401 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67886]: 2024-09-21 03:04:20.079064438 +0000 UTC m=+0.639399151 container remove 824a66cda9e28a2c8d42a644b0d14c100fcb568145ebdc61536c5ca53f3414e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T03:04:20.686 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67974]: 2024-09-21 03:04:20.303167251 +0000 UTC m=+0.016088069 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:20.686 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67974]: 2024-09-21 03:04:20.421931271 +0000 UTC m=+0.134852089 container create c94fdebc46e96de916321b59c373548105a72820c613fdf5928ff345a968e8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T03:04:20.978 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67974]: 2024-09-21 03:04:20.685368891 +0000 UTC m=+0.398289709 container init c94fdebc46e96de916321b59c373548105a72820c613fdf5928ff345a968e8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:04:20.978 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67974]: 2024-09-21 03:04:20.689010938 +0000 UTC m=+0.401931757 container start c94fdebc46e96de916321b59c373548105a72820c613fdf5928ff345a968e8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:04:20.979 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67974]: 2024-09-21 03:04:20.791591224 +0000 UTC m=+0.504512052 container attach c94fdebc46e96de916321b59c373548105a72820c613fdf5928ff345a968e8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:04:20.979 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 21 03:04:20 smithi178 podman[67974]: 2024-09-21 03:04:20.877211016 +0000 UTC m=+0.590131835 container died c94fdebc46e96de916321b59c373548105a72820c613fdf5928ff345a968e8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:04:21.561 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:04:21.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:21.607 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:04:21.607 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:04:21.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:04:21.923 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:21 smithi031 systemd[1]: Stopping Ceph osd.3 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:22.335 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:21 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:21.921+0000 7f15908d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:22.335 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:21 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:21.921+0000 7f15908d3640 -1 osd.3 89 *** Got signal Terminated *** 2024-09-21T03:04:22.335 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:21 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:21.921+0000 7f15908d3640 -1 osd.3 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:23.834 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:23 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:23.539+0000 7f1588eb6640 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.202766+0000 front 2024-09-21T03:03:59.202803+0000 (oldest deadline 2024-09-21T03:04:23.303013+0000) 2024-09-21T03:04:24.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:23.869+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:24.834 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:24 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:24.563+0000 7f1588eb6640 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.202766+0000 front 2024-09-21T03:03:59.202803+0000 (oldest deadline 2024-09-21T03:04:23.303013+0000) 2024-09-21T03:04:24.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:24 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:24.576+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:25.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:24.846+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:25.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:24 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:24.893+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:25.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:25 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:25.620+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:25.834 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:25 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:25.562+0000 7f1588eb6640 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.202766+0000 front 2024-09-21T03:03:59.202803+0000 (oldest deadline 2024-09-21T03:04:23.303013+0000) 2024-09-21T03:04:25.941 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:25 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:25.463+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:26.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:25.834+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:26.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:25 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:25.931+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:26.823 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:26.526+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:26.939 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:26 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3[54384]: 2024-09-21T03:04:26.599+0000 7f1588eb6640 -1 osd.3 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.202766+0000 front 2024-09-21T03:03:59.202803+0000 (oldest deadline 2024-09-21T03:04:23.303013+0000) 2024-09-21T03:04:26.939 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:26 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:26.632+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:26.941 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:26 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:26.488+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:27.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:26.822+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:27.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:26 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:26.937+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:27.747 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:27 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:27.626+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:27.748 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:27 smithi031 podman[109687]: 2024-09-21 03:04:27.475114115 +0000 UTC m=+5.669337163 container died e2fa7add2ccb3fdf3751faf647aba8a1df5284ca7a05d205d5f5264fc5cc75d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:04:27.815 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:27.505+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:27.941 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:27 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:27.458+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:28.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:27 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:27.965+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:28.084 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:27 smithi031 podman[109687]: 2024-09-21 03:04:27.95436622 +0000 UTC m=+6.148589260 container cleanup e2fa7add2ccb3fdf3751faf647aba8a1df5284ca7a05d205d5f5264fc5cc75d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:04:28.084 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:27 smithi031 bash[109687]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:04:28.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:27.814+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:28.581 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:28 smithi031 podman[109705]: 2024-09-21 03:04:28.247772615 +0000 UTC m=+0.765811445 container remove e2fa7add2ccb3fdf3751faf647aba8a1df5284ca7a05d205d5f5264fc5cc75d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T03:04:28.834 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:28 smithi031 podman[109794]: 2024-09-21 03:04:28.493489609 +0000 UTC m=+0.016422470 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:28.835 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:28 smithi031 podman[109794]: 2024-09-21 03:04:28.59862879 +0000 UTC m=+0.121561642 container create c3b1be71f8a4843b6927496e6ae35494a454eaf10712d5df458cc0cf57e6318b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:28.835 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:28 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:28.580+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:28.843 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:28.555+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:28.941 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:28 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:28.508+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:29.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:28.842+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:29.330 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:28 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:28.917+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:29.330 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:28 smithi031 podman[109794]: 2024-09-21 03:04:28.859520358 +0000 UTC m=+0.382453228 container init c3b1be71f8a4843b6927496e6ae35494a454eaf10712d5df458cc0cf57e6318b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:04:29.330 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:28 smithi031 podman[109794]: 2024-09-21 03:04:28.863069787 +0000 UTC m=+0.386002647 container start c3b1be71f8a4843b6927496e6ae35494a454eaf10712d5df458cc0cf57e6318b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:04:29.330 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:28 smithi031 podman[109794]: 2024-09-21 03:04:28.943083328 +0000 UTC m=+0.466016192 container attach c3b1be71f8a4843b6927496e6ae35494a454eaf10712d5df458cc0cf57e6318b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-21T03:04:29.330 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 03:04:29 smithi031 podman[109794]: 2024-09-21 03:04:29.059421018 +0000 UTC m=+0.582353869 container died c3b1be71f8a4843b6927496e6ae35494a454eaf10712d5df458cc0cf57e6318b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:04:29.584 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:29 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:29.554+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:29.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:04:29.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:29.708 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:04:29.708 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:04:29.708 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:04:29.756 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:29 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:29.502+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:29.805 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:29.517+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:30.023 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:29 smithi165 systemd[1]: Stopping Ceph osd.4 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:30.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:29 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:29.869+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:30.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:29.803+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:30.442 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:30 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4[46041]: 2024-09-21T03:04:30.021+0000 7fb148a83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:30.442 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:30 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4[46041]: 2024-09-21T03:04:30.021+0000 7fb148a83640 -1 osd.4 89 *** Got signal Terminated *** 2024-09-21T03:04:30.442 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:30 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4[46041]: 2024-09-21T03:04:30.021+0000 7fb148a83640 -1 osd.4 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:30.764 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:30 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:30.493+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:30.830 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:30 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:30.516+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:30.941 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:30 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:30.458+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:31.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:30 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:30.829+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:31.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:30 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:30.763+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:31.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:31 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:31.415+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:31.806 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:31 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:31.535+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:31.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:31 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:31.548+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:31.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:31 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:31.791+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:32.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:31 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:31.805+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:32.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:32 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:32.415+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:32.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:32 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:32.528+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:32.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:32 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:32.826+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:32.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:32 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:32.577+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:32.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:32 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:32.577+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:33.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:32 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:32.777+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:33.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:33 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:33.434+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:33.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:33 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:33.434+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:33.812 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:33 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:33.528+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:33.812 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:33 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:33.528+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:33.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:33 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:33.536+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:33.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:33 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:33.753+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:33.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:33 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:33.753+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:33.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:33 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:33.764+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:34.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:33 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:33.811+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:34.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:33 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:33.811+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:34.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:33 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:33.803+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:34.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:34 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:34.410+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:34.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:34 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:34.410+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:34.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:34 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:34.525+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:34.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:34 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:34.525+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:34.879 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:34 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:34.754+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:34.880 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:34 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:34.526+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:34.880 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:34 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:34.754+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:34.880 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:34 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:34.754+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:35.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:34 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:34.767+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:35.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:34 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:34.836+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:35.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:34 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:34.836+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:35.691 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:35 smithi165 podman[67531]: 2024-09-21 03:04:35.565212255 +0000 UTC m=+5.665234483 container died e61c6596732adeb0b7566bdabfca9bc1ffc5b18723043c63cf9118009a28b401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T03:04:35.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:35 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:35.420+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:35.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:35 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:35.420+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:35.818 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:35 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:35.522+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:35.819 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:35 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:35.522+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:35.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:35 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:35.755+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:35.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:35 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:35.793+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:35.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:35 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:35.793+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:35.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:35 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:35.545+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:35.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:35 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:35.545+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:36.013 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:35 smithi165 podman[67531]: 2024-09-21 03:04:35.996227913 +0000 UTC m=+6.096250133 container cleanup e61c6596732adeb0b7566bdabfca9bc1ffc5b18723043c63cf9118009a28b401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:04:36.013 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:35 smithi165 bash[67531]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:04:36.014 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:35 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:35.760+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:36.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:35 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:35.817+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:36.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:35 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:35.817+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:36.659 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:36 smithi165 podman[67549]: 2024-09-21 03:04:36.214421299 +0000 UTC m=+0.643962431 container remove e61c6596732adeb0b7566bdabfca9bc1ffc5b18723043c63cf9118009a28b401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:04:36.659 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:36 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:36.445+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:36.660 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:36 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:36.445+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:36.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:36.559+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:36.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:36.559+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:36.941 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:36 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:36.759+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:36.941 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:36 smithi165 podman[67640]: 2024-09-21 03:04:36.561309761 +0000 UTC m=+0.015773483 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:36.941 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:36 smithi165 podman[67640]: 2024-09-21 03:04:36.681965576 +0000 UTC m=+0.136429353 container create 9440a4ea321ce8e1bd8660704d4dbd964ea64c9ed0bc1a93543ad0664de3a9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:04:36.941 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:36 smithi165 podman[67640]: 2024-09-21 03:04:36.884314177 +0000 UTC m=+0.338777889 container init 9440a4ea321ce8e1bd8660704d4dbd964ea64c9ed0bc1a93543ad0664de3a9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T03:04:36.941 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:36 smithi165 podman[67640]: 2024-09-21 03:04:36.887602795 +0000 UTC m=+0.342066510 container start 9440a4ea321ce8e1bd8660704d4dbd964ea64c9ed0bc1a93543ad0664de3a9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T03:04:36.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:36 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:36.757+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:36.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:36 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:36.594+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:36.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:36 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:36.594+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:36.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:36 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:36.809+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:36.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:36 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:36.809+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:37.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:36.864+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:37.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:36.864+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:37.439 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:36 smithi165 podman[67640]: 2024-09-21 03:04:36.984311066 +0000 UTC m=+0.438774789 container attach 9440a4ea321ce8e1bd8660704d4dbd964ea64c9ed0bc1a93543ad0664de3a9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:04:37.439 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 21 03:04:37 smithi165 podman[67640]: 2024-09-21 03:04:37.08671452 +0000 UTC m=+0.541178243 container died 9440a4ea321ce8e1bd8660704d4dbd964ea64c9ed0bc1a93543ad0664de3a9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-21T03:04:37.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:37 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:37.480+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:37.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:37 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:37.480+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:37.774 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:04:37.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:37.814 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:04:37.814 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:04:37.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:04:37.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:37 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:37.519+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:37.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:37 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:37.519+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:37.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:37 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:37.749+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:37.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:37 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:37.848+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:37.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:37 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:37.848+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:37.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:37 smithi178 systemd[1]: Stopping Ceph osd.5 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:37.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:37 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:37.640+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:37.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:37 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:37.640+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:38.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:37 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:37.774+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:38.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:37 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:37.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:38.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:37 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:37.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:38.478 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:38.132+0000 7f01f2f16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:38.478 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:38.132+0000 7f01f2f16640 -1 osd.5 89 *** Got signal Terminated *** 2024-09-21T03:04:38.478 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:38.132+0000 7f01f2f16640 -1 osd.5 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:38.826 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:38 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:38.461+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:38.826 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:38 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:38.461+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:38.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:38 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:38.495+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:38.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:38 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:38.495+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:38.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:38.893+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:38.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:38.893+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:38.979 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:38.769+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:38.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:38.647+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:38.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:38 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:38.647+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:39.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:38 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:38.824+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:39.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:38 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:38.938+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:39.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:38 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:38.938+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:39.831 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:39 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:39.495+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:39.831 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:39 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:39.495+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:39.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:39 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:39.453+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:39.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:39 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:39.453+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:39.872 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:39 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:39.787+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:39.872 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:39 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:39.603+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:39.872 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:39 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:39.603+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:40.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:39 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:39.830+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:40.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:39 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:39.871+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:40.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:39 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:39.871+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:40.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:39 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:39.953+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:40.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:39 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:39.953+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:40.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:40 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:40.442+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:40.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:40 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:40.442+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:40.841 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:40 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:40.450+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:40.841 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:40 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:40.450+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:40.903 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:40 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:40.819+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:40.903 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:40 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:40.589+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:40.903 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:40 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:40.589+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:41.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:40 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:40.840+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:41.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:40 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:40.840+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:41.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:40 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:40.902+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:41.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:40 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:40.902+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:41.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:40 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:40.902+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.812616+0000 front 2024-09-21T03:04:17.812640+0000 (oldest deadline 2024-09-21T03:04:40.712730+0000) 2024-09-21T03:04:41.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:40 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:40.987+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:41.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:40 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:40.987+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:41.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:41 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:41.467+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:41.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:41 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:41.467+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:41.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:41 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:41.467+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:41.844 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:41 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:41.552+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:41.844 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:41 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:41.552+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:41.859 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:41 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:41.442+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:41.859 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:41 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:41.442+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:41.859 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:41 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:41.442+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:42.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:41 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:41.858+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:42.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:41 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:41.858+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:42.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:41 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:41.842+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:42.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:41 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:41.936+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:42.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:41 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:41.936+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:42.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:41 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:41.936+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.812616+0000 front 2024-09-21T03:04:17.812640+0000 (oldest deadline 2024-09-21T03:04:40.712730+0000) 2024-09-21T03:04:42.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:41 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:41.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:42.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:41 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:41.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:42.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:42.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:42.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:42.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:42.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:42.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:42.796 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:42.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:42.796 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:42.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:42.796 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:42.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:42.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:42.512+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:42.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:42.512+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:42.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:42.512+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:43.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:42.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:43.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:42 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:42.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:43.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:42.795+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:43.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:42.795+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:43.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:42.930+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.709632+0000 front 2024-09-21T03:03:59.709657+0000 (oldest deadline 2024-09-21T03:04:23.809875+0000) 2024-09-21T03:04:43.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:42.930+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.211606+0000 front 2024-09-21T03:04:07.211640+0000 (oldest deadline 2024-09-21T03:04:33.111721+0000) 2024-09-21T03:04:43.228 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:42 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5[46034]: 2024-09-21T03:04:42.930+0000 7f01ef501640 -1 osd.5 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.812616+0000 front 2024-09-21T03:04:17.812640+0000 (oldest deadline 2024-09-21T03:04:40.712730+0000) 2024-09-21T03:04:43.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:42.908+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:43.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:42.908+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:43.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:42 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:42.908+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:43.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:43 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:43.430+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:43.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:43 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:43.430+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:43.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:43 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:43.430+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:43.775 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:43 smithi178 podman[68089]: 2024-09-21 03:04:43.507451624 +0000 UTC m=+5.495404163 container died 1c3f733fc592a4e169276f4451987054401aa81d5106c758b1f17a23dde27cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:43.775 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:43.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:43.776 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:43.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:43.776 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:43.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:43.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:43 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:43.497+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:43.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:43 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:43.497+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:43.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:43 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:43.497+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:44.150 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:43 smithi178 podman[68089]: 2024-09-21 03:04:43.865391206 +0000 UTC m=+5.853343740 container cleanup 1c3f733fc592a4e169276f4451987054401aa81d5106c758b1f17a23dde27cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T03:04:44.151 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:43 smithi178 bash[68089]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:04:44.151 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:43.797+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:44.151 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:43 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:43.797+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:44.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:43 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:43.831+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:44.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:43 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:43.831+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:44.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:43 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:43.869+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:44.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:43 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:43.869+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:44.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:43 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:43.869+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:44.478 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68105]: 2024-09-21 03:04:44.150002066 +0000 UTC m=+0.640349609 container remove 1c3f733fc592a4e169276f4451987054401aa81d5106c758b1f17a23dde27cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:04:44.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:44 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:44.407+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:44.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:44 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:44.407+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:44.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:44 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:44.407+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:44.761 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:44 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:44.478+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:44.761 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:44 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:44.478+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:44.761 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:44 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:44.478+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:44.761 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68194]: 2024-09-21 03:04:44.392018233 +0000 UTC m=+0.016031426 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:44.762 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68194]: 2024-09-21 03:04:44.500977727 +0000 UTC m=+0.124990913 container create 1364cbd4c7bdc3d363c0a81a764d6644a40db2898af397cf1b17896d17753df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:04:44.762 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68194]: 2024-09-21 03:04:44.703504885 +0000 UTC m=+0.327518079 container init 1364cbd4c7bdc3d363c0a81a764d6644a40db2898af397cf1b17896d17753df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:04:44.762 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68194]: 2024-09-21 03:04:44.706901835 +0000 UTC m=+0.330915021 container start 1364cbd4c7bdc3d363c0a81a764d6644a40db2898af397cf1b17896d17753df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:04:44.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:44.521+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:44.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:44.521+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:44.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:44.521+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:45.182 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68194]: 2024-09-21 03:04:44.787197992 +0000 UTC m=+0.411211185 container attach 1364cbd4c7bdc3d363c0a81a764d6644a40db2898af397cf1b17896d17753df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T03:04:45.183 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 21 03:04:44 smithi178 podman[68194]: 2024-09-21 03:04:44.899520584 +0000 UTC m=+0.523533776 container died 1364cbd4c7bdc3d363c0a81a764d6644a40db2898af397cf1b17896d17753df4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T03:04:45.183 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:44 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:44.759+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:45.183 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:44 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:44.759+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:45.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:44 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:44.811+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:45.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:44 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:44.811+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:45.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:44.860+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:45.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:44.860+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:45.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:44.860+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:45.542 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:04:45.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:45.580 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:04:45.580 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:04:45.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:04:45.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:45 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:45.450+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:45.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:45 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:45.450+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:45.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:45 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:45.450+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:45.796 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:45 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:45.458+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:45.797 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:45 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:45.458+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:45.797 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:45 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:45.458+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:45.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:45.561+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:45.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:45.561+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:45.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:45.561+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:45.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 systemd[1]: Stopping Ceph osd.6 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:46.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:45 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:45.796+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:46.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:45 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:45.796+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:46.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:45 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:45.757+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:46.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:45 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:45.757+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:46.334 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:45.915+0000 7fb2d1f4f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:46.334 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:45.915+0000 7fb2d1f4f640 -1 osd.6 89 *** Got signal Terminated *** 2024-09-21T03:04:46.334 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:45.915+0000 7fb2d1f4f640 -1 osd.6 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:46.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:45.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:46.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:45.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:46.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:45 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:45.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:46.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:46 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:46.435+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:46.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:46 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:46.435+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:46.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:46 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:46.435+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:46.804 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:46 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:46.485+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:46.804 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:46 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:46.485+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:46.804 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:46 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:46.485+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:46.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:46 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:46.565+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:46.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:46 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:46.565+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:46.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:46 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:46.565+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:47.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:46 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:46.803+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:47.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:46 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:46.803+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:47.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:46 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:46.729+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:47.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:46 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:46.729+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:47.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:46 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:46.862+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:47.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:46 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:46.862+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:47.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:46 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:46.862+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:47.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:47 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:47.450+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:47.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:47 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:47.450+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:47.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:47 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:47.450+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:47.776 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:47.446+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:47.776 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:47.446+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:47.776 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:47.446+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:47.815 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:47.562+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:47.816 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:47.562+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:47.816 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:47.562+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:48.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:47.815+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:48.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:47.815+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:48.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:47 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:47.815+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:48.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:47.775+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:48.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:47.775+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:48.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:47 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:47.756+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:48.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:47 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:47.756+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:48.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:48 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:48.466+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:48.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:48 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:48.466+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:48.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:48 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:48.466+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:48.779 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:48.461+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:48.779 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:48.461+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:48.779 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:48.461+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:48.780 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:48.461+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:49.084 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:48.604+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:49.084 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:48.604+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:49.084 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:48.604+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:49.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:48.852+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:49.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:48.852+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:49.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:48.852+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:49.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:48 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:48.852+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:49.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:48.777+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:49.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:48.777+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:49.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:48 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:48.777+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:49.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:48 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:48.731+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:49.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:48 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:48.731+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:49.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:49.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:49.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:49.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:49.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:49.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:49.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:49.417+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:49.639+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:49.639+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:49.639+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:49.889+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:49.889+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:49.889+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:49.891 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:49 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:49.889+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:49.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:49 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:49.686+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:49.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:49 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:49.686+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:49.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:49 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:49.485+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:49.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:49 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:49.485+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:49.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:49 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:49.485+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:50.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:49.822+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:50.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:49.822+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:50.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:49 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:49.822+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:50.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:50 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:50.446+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:50.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:50 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:50.446+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:50.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:50 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:50.446+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:50.805 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:50.468+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:50.805 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:50.468+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:50.805 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:50.468+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:50.805 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:50.468+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:50.935 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:50.633+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.213762+0000 front 2024-09-21T03:04:01.213870+0000 (oldest deadline 2024-09-21T03:04:24.113911+0000) 2024-09-21T03:04:50.935 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:50.633+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:06.415070+0000 front 2024-09-21T03:04:06.415537+0000 (oldest deadline 2024-09-21T03:04:32.314998+0000) 2024-09-21T03:04:50.936 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6[60823]: 2024-09-21T03:04:50.633+0000 7fb2ca532640 -1 osd.6 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.916741+0000 front 2024-09-21T03:04:16.916669+0000 (oldest deadline 2024-09-21T03:04:41.016855+0000) 2024-09-21T03:04:51.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:50.803+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:51.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:50.803+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:51.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:50 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:50.803+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:51.220 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:50.934+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:51.220 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:50.934+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:51.220 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:50.934+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:51.220 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:50 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:50.934+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:51.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:50 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:50.732+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:51.228 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:50 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:50.732+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:51.508 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:51 smithi031 podman[109897]: 2024-09-21 03:04:51.219252804 +0000 UTC m=+5.438689669 container died df30cc4ea2ef6db44b1ec20c415e2c756096a8b06ee0ab4ddc596ca27931c416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:04:51.685 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:51 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:51.430+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:51.685 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:51 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:51.430+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:51.685 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:51 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:51.430+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:51.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:51.435+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:51.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:51.435+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:51.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:51.435+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:51.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:51.435+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:51.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:51 smithi031 podman[109897]: 2024-09-21 03:04:51.707192494 +0000 UTC m=+5.926629357 container cleanup df30cc4ea2ef6db44b1ec20c415e2c756096a8b06ee0ab4ddc596ca27931c416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:04:51.834 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:51 smithi031 bash[109897]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:04:51.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:51 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:51.682+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:51.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:51 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:51.682+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:51.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:51 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:51.682+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:52.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:51.812+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:52.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:51.812+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:52.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:51 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:51.812+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:52.332 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:51 smithi031 podman[109915]: 2024-09-21 03:04:51.983641252 +0000 UTC m=+0.756149168 container remove df30cc4ea2ef6db44b1ec20c415e2c756096a8b06ee0ab4ddc596ca27931c416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:52.332 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:51 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:51.886+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:52.332 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:51 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:51.886+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:52.333 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:51 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:51.886+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:52.333 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:51 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:51.886+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:52.584 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:52 smithi031 podman[110004]: 2024-09-21 03:04:52.234303228 +0000 UTC m=+0.016692008 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:04:52.584 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:52 smithi031 podman[110004]: 2024-09-21 03:04:52.334272889 +0000 UTC m=+0.116661697 container create d53729ac927dc997d4c405cf3449331f200325449bfcbc05dd2a0fb9bf8917f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T03:04:52.584 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:52 smithi031 podman[110004]: 2024-09-21 03:04:52.578359241 +0000 UTC m=+0.360748015 container init d53729ac927dc997d4c405cf3449331f200325449bfcbc05dd2a0fb9bf8917f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T03:04:52.584 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:52 smithi031 podman[110004]: 2024-09-21 03:04:52.581822473 +0000 UTC m=+0.364211241 container start d53729ac927dc997d4c405cf3449331f200325449bfcbc05dd2a0fb9bf8917f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:04:52.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:52.402+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:52.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:52.402+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:52.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:52.402+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:52.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:52.402+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:52.692 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:52.393+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:52.692 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:52.393+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:52.692 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:52.393+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:52.977 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:52.690+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:52.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:52.690+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:52.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:52 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:52.690+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:53.084 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:52 smithi031 podman[110004]: 2024-09-21 03:04:52.678582257 +0000 UTC m=+0.460971023 container attach d53729ac927dc997d4c405cf3449331f200325449bfcbc05dd2a0fb9bf8917f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:04:53.084 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 03:04:52 smithi031 podman[110004]: 2024-09-21 03:04:52.774436993 +0000 UTC m=+0.556825776 container died d53729ac927dc997d4c405cf3449331f200325449bfcbc05dd2a0fb9bf8917f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:04:53.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:52.928+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:53.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:52.928+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:53.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:52.928+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:53.085 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:52 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:52.928+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:53.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:52.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:53.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:52.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:53.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:52.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:53.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:04:53.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:04:53.593 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:04:53.593 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:04:53.593 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:04:53.619 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:53.363+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:53.619 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:53.363+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:53.619 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:53.363+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:53.620 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:53.363+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:53.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:53 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:53.436+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:53.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:53 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:53.436+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:53.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:53 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:53.436+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:53.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:53 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:53.656+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:53.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:53 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:53.656+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:53.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:53 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:53.656+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:53.899 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 systemd[1]: Stopping Ceph osd.7 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:04:53.900 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:53.845+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:53.900 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:53.845+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:53.900 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:53.845+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:53.900 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:53.897+0000 7fcda8781640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:04:53.900 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:53.897+0000 7fcda8781640 -1 osd.7 89 *** Got signal Terminated *** 2024-09-21T03:04:53.900 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:53 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:53.897+0000 7fcda8781640 -1 osd.7 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:04:54.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:53 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:53.888+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:54.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:53 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:53.888+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:54.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:53 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:53.888+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:54.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:53 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:53.888+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:54.654 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:54 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:54.391+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:54.654 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:54 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:54.391+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:54.654 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:54 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:54.391+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:54.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:54.412+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:54.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:54.412+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:54.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:54.412+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:54.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:54.412+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:54.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:54 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:54.652+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:54.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:54 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:54.652+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:54.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:54 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:54.652+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:55.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:54.867+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:55.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:54.867+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:55.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:54 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:54.867+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:55.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:54.845+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:55.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:54.845+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:55.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:54.845+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:55.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:54 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:54.845+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:55.689 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:55.366+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:55.689 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:55.366+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:55.689 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:55.366+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:55.867 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:55.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:55.867 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:55.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:55.868 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:55.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:55.868 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:55.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:55.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:55.687+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:55.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:55.687+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:55.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:55.687+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:55.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:55 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:55.687+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:04:56.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:55.866+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:56.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:55.866+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:56.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:55 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:55.866+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:56.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:55 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:55.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:56.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:55 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:55.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:56.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:55 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:55.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:56.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:55 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:55.858+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:56.655 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:56.319+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:56.655 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:56.319+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:56.655 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:56.319+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:56.857 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:56.448+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:56.857 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:56.448+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:56.857 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:56.448+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:56.857 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:56.448+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:56.857 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:56.448+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:04:56.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:56.654+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:56.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:56.654+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:56.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:56.654+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:56.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:56 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:56.654+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:04:57.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:56.855+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:57.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:56.855+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:57.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:56.855+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:57.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:56 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:56.895+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:57.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:56 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:56.895+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:57.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:56 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:56.895+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:57.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:56 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:56.895+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:57.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:56 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:56.895+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:04:57.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:57.432+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:57.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:57.432+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:57.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:57.432+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:57.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:57.432+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:57.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:57.432+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:04:57.701 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:57.360+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:57.701 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:57.360+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:57.701 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:57.360+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:57.702 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:57.360+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:04:57.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:57.700+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:57.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:57.700+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:57.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:57.700+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:57.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:57 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:57.700+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:04:58.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:57.813+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:58.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:57.813+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:58.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:57.813+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:58.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:57 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:57.813+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:34.447691+0000 front 2024-09-21T03:04:34.447739+0000 (oldest deadline 2024-09-21T03:04:57.347900+0000) 2024-09-21T03:04:58.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:57.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:58.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:57.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:58.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:57.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:58.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:57.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:58.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:57 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:57.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:04:58.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:58.401+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:58.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:58.401+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:58.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:58.401+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:58.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:58.401+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:58.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:58.401+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:04:58.699 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:58.355+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:58.699 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:58.355+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:58.699 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:58.355+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:58.699 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:58.355+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:04:58.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:58.697+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:58.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:58.697+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:58.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:58.697+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:58.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:58 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:58.697+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:04:59.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:58.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:08.541548+0000 front 2024-09-21T03:04:08.541511+0000 (oldest deadline 2024-09-21T03:04:33.241645+0000) 2024-09-21T03:04:59.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:58.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.043030+0000 front 2024-09-21T03:04:16.043198+0000 (oldest deadline 2024-09-21T03:04:40.143107+0000) 2024-09-21T03:04:59.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:58.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:25.245317+0000 front 2024-09-21T03:04:25.245301+0000 (oldest deadline 2024-09-21T03:04:48.745372+0000) 2024-09-21T03:04:59.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:58 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7[52210]: 2024-09-21T03:04:58.836+0000 7fcda4d6c640 -1 osd.7 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:34.447691+0000 front 2024-09-21T03:04:34.447739+0000 (oldest deadline 2024-09-21T03:04:57.347900+0000) 2024-09-21T03:04:59.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:58.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:04:59.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:58.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:04:59.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:58.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:04:59.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:58.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:04:59.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:58 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:58.909+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:04:59.549 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:59 smithi165 podman[67743]: 2024-09-21 03:04:59.268198412 +0000 UTC m=+5.485516469 container died 98aeeb769708a9172fda5afa2ca4f0aebcc2993bdd8ca6128e52409bb918c419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:04:59.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:59 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:59.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:04:59.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:59 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:59.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:04:59.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:59 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:59.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:04:59.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:59 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:59.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:04:59.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:04:59 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:04:59.444+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:04:59.668 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:59.387+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:04:59.668 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:59.387+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:04:59.668 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:59.387+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:04:59.668 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:04:59.387+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:04:59.907 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:59 smithi165 podman[67743]: 2024-09-21 03:04:59.647670383 +0000 UTC m=+5.864988444 container cleanup 98aeeb769708a9172fda5afa2ca4f0aebcc2993bdd8ca6128e52409bb918c419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:04:59.908 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:59 smithi165 bash[67743]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:04:59.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:59.666+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:04:59.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:59.666+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:04:59.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:59.666+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:04:59.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:04:59 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:04:59.666+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:00.191 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:04:59 smithi165 podman[67759]: 2024-09-21 03:04:59.907093712 +0000 UTC m=+0.630121415 container remove 98aeeb769708a9172fda5afa2ca4f0aebcc2993bdd8ca6128e52409bb918c419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:05:00.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:59 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:59.921+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:00.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:59 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:59.921+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:00.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:59 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:59.921+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:00.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:59 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:59.921+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:00.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:04:59 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:04:59.921+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:00.431+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:00.431+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:00.431+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:00.431+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:00.431+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:05:00 smithi165 podman[67847]: 2024-09-21 03:05:00.16440334 +0000 UTC m=+0.016260097 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:05:00 smithi165 podman[67847]: 2024-09-21 03:05:00.274601627 +0000 UTC m=+0.126458384 container create b9a4ab57b034157f7ebfd6fda2a5215fa0070b16d2b8b35df209f40d49f40279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:05:00 smithi165 podman[67847]: 2024-09-21 03:05:00.468759199 +0000 UTC m=+0.320615967 container init b9a4ab57b034157f7ebfd6fda2a5215fa0070b16d2b8b35df209f40d49f40279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:05:00.561 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:05:00 smithi165 podman[67847]: 2024-09-21 03:05:00.472007483 +0000 UTC m=+0.323864250 container start b9a4ab57b034157f7ebfd6fda2a5215fa0070b16d2b8b35df209f40d49f40279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T03:05:00.562 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:05:00 smithi165 podman[67847]: 2024-09-21 03:05:00.560317562 +0000 UTC m=+0.412174333 container attach b9a4ab57b034157f7ebfd6fda2a5215fa0070b16d2b8b35df209f40d49f40279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:05:00.650 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:00.342+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:00.650 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:00.342+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:00.650 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:00.342+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:00.650 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:00.342+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:00.941 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 21 03:05:00 smithi165 podman[67847]: 2024-09-21 03:05:00.664335771 +0000 UTC m=+0.516192531 container died b9a4ab57b034157f7ebfd6fda2a5215fa0070b16d2b8b35df209f40d49f40279 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:05:00.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:00.648+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:05:00.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:00.648+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:05:00.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:00.648+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:05:00.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:00 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:00.648+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:01.295 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:05:01.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:00.942+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:01.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:00.942+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:01.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:00.942+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:01.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:00.942+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:01.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:00 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:00.942+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:01.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:01.373 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:05:01.373 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:05:01.373 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:05:01.548 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:01 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:01.422+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:01.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:01 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:01.422+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:01.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:01 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:01.422+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:01.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:01 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:01.422+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:01.549 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:01 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:01.422+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:01.686 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:01.380+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:01.686 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:01.380+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:01.380+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:01.380+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 systemd[1]: Stopping Ceph osd.8 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.612+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.612+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.612+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.612+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.685+0000 7fe9b79fb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.685+0000 7fe9b79fb640 -1 osd.8 89 *** Got signal Terminated *** 2024-09-21T03:05:01.687 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:01 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:01.685+0000 7fe9b79fb640 -1 osd.8 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:05:02.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:01 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:01.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:02.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:01 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:01.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:02.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:01 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:01.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:02.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:01 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:01.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:02.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:01 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:01.952+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:02.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:02.375+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:02.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:02.375+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:02.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:02.375+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:02.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:02.375+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:02.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:02 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:02.375+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:02.562+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:02.562+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:02.562+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:02.562+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:02.428+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:02.428+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:02.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:02.428+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:02.729 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:02 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:02.428+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:03.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:02 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:02.957+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:03.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:02 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:02.957+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:03.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:02 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:02.957+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:03.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:02 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:02.957+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:03.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:02 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:02.957+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:03.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:03 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:03.334+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:03.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:03 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:03.334+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:03.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:03 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:03.334+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:03.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:03 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:03.334+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:03.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:03 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:03.334+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:03.451+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:03.451+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:03.451+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:03.451+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:03.586+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:03.586+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:03.586+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:03.586+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:03.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:03 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:03.586+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:42.486350+0000 front 2024-09-21T03:04:42.486303+0000 (oldest deadline 2024-09-21T03:05:03.586422+0000) 2024-09-21T03:05:04.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:03 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:03.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:04.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:03 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:03.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:04.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:03 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:03.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:04.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:03 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:03.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:04.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:03 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:03.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:04.592 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:04 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:04.306+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:04.592 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:04 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:04.306+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:04.592 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:04 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:04.306+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:04.593 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:04 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:04.306+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:04.593 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:04 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:04.306+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:04.902 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:04.635+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:05:04.902 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:04.635+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:05:04.902 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:04.635+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:05:04.902 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:04.635+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:04.902 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:04.635+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:42.486350+0000 front 2024-09-21T03:04:42.486303+0000 (oldest deadline 2024-09-21T03:05:03.586422+0000) 2024-09-21T03:05:04.903 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:04.498+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:04.903 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:04.498+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:04.903 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:04.498+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:04.903 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:04.498+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:05.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:04.933+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:05.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:04.933+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:05.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:04.933+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:05.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:04.933+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:05.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:04 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:04.933+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:05.604 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:05 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:05.262+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:05.604 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:05 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:05.262+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:05.604 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:05 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:05.262+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:05.604 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:05 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:05.262+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:05.604 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:05 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:05.262+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:05.862 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:05.680+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.980478+0000 front 2024-09-21T03:04:09.980553+0000 (oldest deadline 2024-09-21T03:04:33.480484+0000) 2024-09-21T03:05:05.862 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:05.680+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:17.482044+0000 front 2024-09-21T03:04:17.482011+0000 (oldest deadline 2024-09-21T03:04:42.782220+0000) 2024-09-21T03:05:05.862 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:05.680+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:26.184204+0000 front 2024-09-21T03:04:26.184139+0000 (oldest deadline 2024-09-21T03:04:50.883901+0000) 2024-09-21T03:05:05.862 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:05.680+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:30.884434+0000 front 2024-09-21T03:04:30.884497+0000 (oldest deadline 2024-09-21T03:04:55.584434+0000) 2024-09-21T03:05:05.863 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8[52471]: 2024-09-21T03:05:05.680+0000 7fe9affde640 -1 osd.8 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:42.486350+0000 front 2024-09-21T03:04:42.486303+0000 (oldest deadline 2024-09-21T03:05:03.586422+0000) 2024-09-21T03:05:05.863 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:05.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:05.863 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:05.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:05.863 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:05.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:05.863 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:05.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:05.863 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:05 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:05.508+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:06.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:05 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:05.897+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:06.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:05 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:05.897+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:06.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:05 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:05.897+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:06.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:05 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:05.897+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:06.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:05 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:05.897+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:06.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:06.270+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:06.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:06.270+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:06.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:06.270+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:06.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:06.270+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:06.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:06.270+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:06.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:06 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:06.270+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:06.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:06 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:06.502+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:06.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:06 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:06.502+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:06.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:06 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:06.502+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:06.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:06 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:06.502+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:06.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:06 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:06.502+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:07.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:06 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:06.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:07.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:06 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:06.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:07.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:06 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:06.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:07.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:06 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:06.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:07.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:06 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:06.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:07.346 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:07 smithi178 podman[68299]: 2024-09-21 03:05:07.008883428 +0000 UTC m=+5.435978062 container died ff40fd1a44d7a37b7e88d3a7b2865278765e6a70618d28cab08f69cf1412b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:05:07.671 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:07 smithi178 podman[68299]: 2024-09-21 03:05:07.443981541 +0000 UTC m=+5.871076177 container cleanup ff40fd1a44d7a37b7e88d3a7b2865278765e6a70618d28cab08f69cf1412b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:05:07.671 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:07 smithi178 bash[68299]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:05:07.672 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:07 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:07.522+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:07.672 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:07 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:07.522+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:07.672 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:07 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:07.522+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:07.672 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:07 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:07.522+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:07.672 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:07 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:07.522+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:07.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:07 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:07.312+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:07.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:07 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:07.312+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:07.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:07 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:07.312+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:07.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:07 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:07.312+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:07.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:07 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:07.312+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:07.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:07 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:07.312+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:07.977 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:07 smithi178 podman[68326]: 2024-09-21 03:05:07.670449393 +0000 UTC m=+0.655200929 container remove ff40fd1a44d7a37b7e88d3a7b2865278765e6a70618d28cab08f69cf1412b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T03:05:08.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:07 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:07.906+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:08.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:07 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:07.906+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:08.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:07 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:07.906+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:08.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:07 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:07.906+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:08.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:07 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:07.906+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:08.408 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:08 smithi178 podman[68414]: 2024-09-21 03:05:07.99175548 +0000 UTC m=+0.015842252 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:05:08.408 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:08 smithi178 podman[68414]: 2024-09-21 03:05:08.096424277 +0000 UTC m=+0.120511085 container create 051654cb41b4106482070d765100f1fb742e10b77ad5c1221c6b8bd08af33fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:08.408 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:08 smithi178 podman[68414]: 2024-09-21 03:05:08.30722478 +0000 UTC m=+0.331311543 container init 051654cb41b4106482070d765100f1fb742e10b77ad5c1221c6b8bd08af33fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-21T03:05:08.408 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:08 smithi178 podman[68414]: 2024-09-21 03:05:08.310505986 +0000 UTC m=+0.334592749 container start 051654cb41b4106482070d765100f1fb742e10b77ad5c1221c6b8bd08af33fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:08.408 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:08 smithi178 podman[68414]: 2024-09-21 03:05:08.407147452 +0000 UTC m=+0.431234221 container attach 051654cb41b4106482070d765100f1fb742e10b77ad5c1221c6b8bd08af33fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-21T03:05:08.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:08 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:08.271+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:08.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:08 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:08.271+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:08.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:08 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:08.271+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:08.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:08 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:08.271+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:08.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:08 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:08.271+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:08.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:08 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:08.271+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:08.728 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 21 03:05:08 smithi178 podman[68414]: 2024-09-21 03:05:08.500859478 +0000 UTC m=+0.524946255 container died 051654cb41b4106482070d765100f1fb742e10b77ad5c1221c6b8bd08af33fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:05:08.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:08.569+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:08.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:08.569+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:08.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:08.569+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:08.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:08.569+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:08.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:08.569+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:09.205 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:05:09.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:09.245 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:05:09.245 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:05:09.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:05:09.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:08.882+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:09.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:08.882+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:09.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:08.882+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:09.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:08.882+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:09.273 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:08.882+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:09.274 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:08 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:08.882+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.701460+0000 front 2024-09-21T03:04:39.701553+0000 (oldest deadline 2024-09-21T03:05:08.501748+0000) 2024-09-21T03:05:09.569 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 systemd[1]: Stopping Ceph osd.9 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:09.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:09 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:09.261+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:09.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:09 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:09.261+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:09.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:09 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:09.261+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:09.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:09 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:09.261+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:09.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:09 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:09.261+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:09.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:09 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:09.261+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:09.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.567+0000 7fb109cf0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:09.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.567+0000 7fb109cf0640 -1 osd.9 89 *** Got signal Terminated *** 2024-09-21T03:05:09.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.567+0000 7fb109cf0640 -1 osd.9 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:05:09.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:09 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:09.570+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:09.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:09 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:09.570+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:09.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:09 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:09.570+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:09.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:09 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:09.570+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:09.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:09 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:09.570+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:10.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:10.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:10.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:10.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:10.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:10.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:09 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:09.887+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.701460+0000 front 2024-09-21T03:04:39.701553+0000 (oldest deadline 2024-09-21T03:05:08.501748+0000) 2024-09-21T03:05:10.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:10 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:10.285+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:10.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:10 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:10.285+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:10.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:10 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:10.285+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:10.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:10 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:10.285+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:10.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:10 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:10.285+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:10.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:10 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:10.285+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:10.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:10 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:10.531+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:10.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:10 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:10.531+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:10.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:10 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:10.531+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:10.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:10 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:10.531+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:10.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:10 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:10.531+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:11.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:10 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:10.930+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:11.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:10 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:10.930+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:11.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:10 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:10.930+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:11.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:10 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:10.930+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:11.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:10 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:10.930+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:11.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:10 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:10.930+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.701460+0000 front 2024-09-21T03:04:39.701553+0000 (oldest deadline 2024-09-21T03:05:08.501748+0000) 2024-09-21T03:05:11.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:11.333+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:11.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:11.333+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:11.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:11.333+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:11.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:11.333+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:11.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:11.333+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:11.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:11 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:11.333+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:11.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:11.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:11.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:11.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:11.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:11.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:11.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:11.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:11.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:11 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:11.490+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:12.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:11 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:11.891+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:12.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:11 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:11.891+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:12.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:11 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:11.891+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:12.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:11 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:11.891+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:12.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:11 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:11.891+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:12.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:11 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:11.891+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.701460+0000 front 2024-09-21T03:04:39.701553+0000 (oldest deadline 2024-09-21T03:05:08.501748+0000) 2024-09-21T03:05:12.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:12.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:12.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:12.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:12.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:12.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:12.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:12 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:12.310+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:12.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:12.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:12.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:12.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:12.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:12.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:12.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:12.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:12.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:12.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:13.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:12 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:12.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:13.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:12 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:12.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:13.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:12 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:12.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:13.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:12 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:12.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:13.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:12 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:12.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:13.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:12 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:12.918+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.701460+0000 front 2024-09-21T03:04:39.701553+0000 (oldest deadline 2024-09-21T03:05:08.501748+0000) 2024-09-21T03:05:13.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:13.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:13.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:13.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:13.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:13.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:13.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:13 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:13.316+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:13.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:13 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:13.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:13.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:13 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:13.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:13.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:13 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:13.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:13.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:13 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:13.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:13.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:13 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:13.480+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:14.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:13 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:13.943+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:03:59.395632+0000 front 2024-09-21T03:03:59.395482+0000 (oldest deadline 2024-09-21T03:04:24.695619+0000) 2024-09-21T03:05:14.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:13 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:13.943+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:09.397006+0000 front 2024-09-21T03:04:09.396961+0000 (oldest deadline 2024-09-21T03:04:33.397561+0000) 2024-09-21T03:05:14.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:13 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:13.943+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.298705+0000 front 2024-09-21T03:04:16.298906+0000 (oldest deadline 2024-09-21T03:04:42.198627+0000) 2024-09-21T03:05:14.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:13 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:13.943+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:22.199005+0000 front 2024-09-21T03:04:22.199064+0000 (oldest deadline 2024-09-21T03:04:48.099103+0000) 2024-09-21T03:05:14.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:13 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:13.943+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.800062+0000 front 2024-09-21T03:04:32.800030+0000 (oldest deadline 2024-09-21T03:04:56.200566+0000) 2024-09-21T03:05:14.334 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:13 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9[67487]: 2024-09-21T03:05:13.943+0000 7fb1022d3640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.701460+0000 front 2024-09-21T03:04:39.701553+0000 (oldest deadline 2024-09-21T03:05:08.501748+0000) 2024-09-21T03:05:14.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:14.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:14.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:14.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:14.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:14.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:14.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:14 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:14.355+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:14.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:14.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:14.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:14.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:14.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:14.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:14.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:14.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:14.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:14.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:14.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:14 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:14.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:15.370 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:15 smithi031 podman[110107]: 2024-09-21 03:05:15.094192461 +0000 UTC m=+5.643120528 container died c87a663ca623cccd1adc4544c9695bcc0d545c427632b6c9ec2273595f985388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:15.646 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:15 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:15.387+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:15.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:15 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:15.447+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:15.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:15 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:15.447+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:15.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:15 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:15.447+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:15.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:15 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:15.447+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:15.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:15 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:15.447+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:15.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:15 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:15.447+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:15.819 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:15 smithi031 podman[110107]: 2024-09-21 03:05:15.542442046 +0000 UTC m=+6.091370148 container cleanup c87a663ca623cccd1adc4544c9695bcc0d545c427632b6c9ec2273595f985388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:15.819 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:15 smithi031 bash[110107]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:05:16.084 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:15 smithi031 podman[110122]: 2024-09-21 03:05:15.818617715 +0000 UTC m=+0.723315816 container remove c87a663ca623cccd1adc4544c9695bcc0d545c427632b6c9ec2273595f985388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T03:05:16.438 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:16 smithi031 podman[110209]: 2024-09-21 03:05:16.152777393 +0000 UTC m=+0.092231296 container create adf8cb8f9de0d3d953bdb6d0bc4376b3e20f53400d5c07342b3f4a641ff74985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T03:05:16.439 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:16 smithi031 podman[110209]: 2024-09-21 03:05:16.077101833 +0000 UTC m=+0.016555699 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:05:16.439 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:16 smithi031 podman[110209]: 2024-09-21 03:05:16.362964102 +0000 UTC m=+0.302417976 container init adf8cb8f9de0d3d953bdb6d0bc4376b3e20f53400d5c07342b3f4a641ff74985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:05:16.439 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:16 smithi031 podman[110209]: 2024-09-21 03:05:16.366524916 +0000 UTC m=+0.305978779 container start adf8cb8f9de0d3d953bdb6d0bc4376b3e20f53400d5c07342b3f4a641ff74985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:05:16.439 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:16 smithi031 podman[110209]: 2024-09-21 03:05:16.438035475 +0000 UTC m=+0.377489350 container attach adf8cb8f9de0d3d953bdb6d0bc4376b3e20f53400d5c07342b3f4a641ff74985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:16.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:16.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:16.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:16.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:16.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:16.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:16.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:16 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:16.409+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:16.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:16.445+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:16.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:16.445+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:16.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:16.445+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:16.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:16.445+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:16.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:16.445+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:16.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:16.445+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:16.834 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 03:05:16 smithi031 podman[110209]: 2024-09-21 03:05:16.561623732 +0000 UTC m=+0.501077590 container died adf8cb8f9de0d3d953bdb6d0bc4376b3e20f53400d5c07342b3f4a641ff74985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T03:05:17.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:05:17.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:17.233 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:05:17.234 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:05:17.234 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 systemd[1]: Stopping Ceph osd.10 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:17.525 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:17.526 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.404+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:17.526 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.523+0000 7fbfec0af640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:17.526 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.523+0000 7fbfec0af640 -1 osd.10 89 *** Got signal Terminated *** 2024-09-21T03:05:17.526 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:17 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:17.523+0000 7fbfec0af640 -1 osd.10 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:05:17.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:17 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:17.416+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:17.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:17 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:17.416+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:17.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:17 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:17.416+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:17.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:17 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:17.416+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:17.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:17 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:17.416+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:17.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:17 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:17.416+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:18.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:18.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:18.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:18.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:18.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:18.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:18.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:18 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:18.391+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:18.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:18 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:18.460+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:18.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:18 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:18.460+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:18.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:18 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:18.460+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:18.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:18 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:18.460+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:18.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:18 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:18.460+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:18.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:18 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:18.460+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:19.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:19 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:19.365+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:19.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:19 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:19.427+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:19.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:19 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:19.427+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:19.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:19 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:19.427+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:19.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:19 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:19.427+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:19.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:19 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:19.427+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:19.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:19 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:19.427+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:20.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:20.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:20.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:20.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:20.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:20.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:20.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:20 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:20.350+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:20.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:20 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:20.474+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:20.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:20 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:20.474+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:20.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:20 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:20.474+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:20.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:20 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:20.474+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:20.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:20 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:20.474+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:20.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:20 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:20.474+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:21.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:21.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:21.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:21.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:21.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:21.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:21.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:21 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:21.399+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:21.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:21 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:21.504+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:22.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:01.972402+0000 front 2024-09-21T03:04:01.972154+0000 (oldest deadline 2024-09-21T03:04:24.872252+0000) 2024-09-21T03:05:22.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:07.173197+0000 front 2024-09-21T03:04:07.173232+0000 (oldest deadline 2024-09-21T03:04:33.073370+0000) 2024-09-21T03:05:22.691 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.475218+0000 front 2024-09-21T03:04:16.475146+0000 (oldest deadline 2024-09-21T03:04:40.575249+0000) 2024-09-21T03:05:22.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T03:04:24.076311+0000 front 2024-09-21T03:04:24.076248+0000 (oldest deadline 2024-09-21T03:04:48.176340+0000) 2024-09-21T03:05:22.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.777775+0000 front 2024-09-21T03:04:32.777797+0000 (oldest deadline 2024-09-21T03:04:56.178749+0000) 2024-09-21T03:05:22.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:39.679973+0000 front 2024-09-21T03:04:39.679958+0000 (oldest deadline 2024-09-21T03:05:05.579530+0000) 2024-09-21T03:05:22.692 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10[58888]: 2024-09-21T03:05:22.433+0000 7fbfe869a640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:47.880405+0000 front 2024-09-21T03:04:47.880350+0000 (oldest deadline 2024-09-21T03:05:11.380299+0000) 2024-09-21T03:05:22.729 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:22.730 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:22.730 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:22.730 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:22.730 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:22.730 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:22.730 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:22 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:22.462+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:23.233 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:22 smithi165 podman[67963]: 2024-09-21 03:05:22.971594255 +0000 UTC m=+5.545115308 container died 2260c719d9d7424c9bc08d82dd44e2b81cc83c7148f76831e2bfb66090642feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:05:23.508 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:23 smithi165 podman[67963]: 2024-09-21 03:05:23.322930922 +0000 UTC m=+5.896451964 container cleanup 2260c719d9d7424c9bc08d82dd44e2b81cc83c7148f76831e2bfb66090642feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:05:23.508 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:23 smithi165 bash[67963]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:05:23.928 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:23 smithi165 podman[67981]: 2024-09-21 03:05:23.507401425 +0000 UTC m=+0.527346932 container remove 2260c719d9d7424c9bc08d82dd44e2b81cc83c7148f76831e2bfb66090642feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T03:05:23.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:23.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:23.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:23.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:23.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:23.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:23.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:23 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:23.497+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:24.191 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:23 smithi165 podman[68067]: 2024-09-21 03:05:23.83075471 +0000 UTC m=+0.016047930 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:05:24.191 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:23 smithi165 podman[68067]: 2024-09-21 03:05:23.933020224 +0000 UTC m=+0.118313483 container create a90ad5d27e7c126ff0975eb66bbe0ff644bbd26e36a35d712e4e0373beb10e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T03:05:24.192 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:24 smithi165 podman[68067]: 2024-09-21 03:05:24.143714924 +0000 UTC m=+0.329008148 container init a90ad5d27e7c126ff0975eb66bbe0ff644bbd26e36a35d712e4e0373beb10e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:05:24.192 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:24 smithi165 podman[68067]: 2024-09-21 03:05:24.146942619 +0000 UTC m=+0.332235836 container start a90ad5d27e7c126ff0975eb66bbe0ff644bbd26e36a35d712e4e0373beb10e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:05:24.630 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:24 smithi165 podman[68067]: 2024-09-21 03:05:24.235234751 +0000 UTC m=+0.420527969 container attach a90ad5d27e7c126ff0975eb66bbe0ff644bbd26e36a35d712e4e0373beb10e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:05:24.630 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 21 03:05:24 smithi165 podman[68067]: 2024-09-21 03:05:24.350138179 +0000 UTC m=+0.535431398 container died a90ad5d27e7c126ff0975eb66bbe0ff644bbd26e36a35d712e4e0373beb10e2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T03:05:24.918 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:05:24.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:24.955 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:05:24.955 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:05:24.956 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:24.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:24 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:24.523+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:25.257 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 systemd[1]: Stopping Ceph osd.11 for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:25.257 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.255+0000 7f2e8dd7b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.255+0000 7f2e8dd7b640 -1 osd.11 89 *** Got signal Terminated *** 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.255+0000 7f2e8dd7b640 -1 osd.11 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:25.518 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:25 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:25.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:26.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:26 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:26.550+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:27.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:27.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:27.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:27.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:27.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:27.978 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:27.979 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:27 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:27.516+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:28.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:28 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:28.469+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T03:04:00.632296+0000 front 2024-09-21T03:04:00.632435+0000 (oldest deadline 2024-09-21T03:04:26.432956+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-21T03:04:10.335016+0000 front 2024-09-21T03:04:10.335049+0000 (oldest deadline 2024-09-21T03:04:35.535729+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-21T03:04:16.036903+0000 front 2024-09-21T03:04:16.036850+0000 (oldest deadline 2024-09-21T03:04:41.936880+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-21T03:04:32.939626+0000 front 2024-09-21T03:04:32.939537+0000 (oldest deadline 2024-09-21T03:04:56.439467+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-21T03:04:41.640902+0000 front 2024-09-21T03:04:41.640968+0000 (oldest deadline 2024-09-21T03:05:05.140935+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T03:04:49.142564+0000 front 2024-09-21T03:04:49.142538+0000 (oldest deadline 2024-09-21T03:05:13.842395+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-21T03:04:56.643977+0000 front 2024-09-21T03:04:56.644075+0000 (oldest deadline 2024-09-21T03:05:20.743881+0000) 2024-09-21T03:05:29.728 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:29 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11[58859]: 2024-09-21T03:05:29.470+0000 7f2e8a366640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-09-21T03:05:05.345040+0000 front 2024-09-21T03:05:05.345005+0000 (oldest deadline 2024-09-21T03:05:28.845021+0000) 2024-09-21T03:05:30.932 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:30 smithi178 podman[68517]: 2024-09-21 03:05:30.569027327 +0000 UTC m=+5.411543440 container died 032ecd819b44d2837634b6d63584ccfbc84d696800efce2b740cb8e1808b8f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:05:31.228 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68517]: 2024-09-21 03:05:31.013835224 +0000 UTC m=+5.856351336 container cleanup 032ecd819b44d2837634b6d63584ccfbc84d696800efce2b740cb8e1808b8f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:05:31.228 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 bash[68517]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:05:31.682 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68533]: 2024-09-21 03:05:31.331605677 +0000 UTC m=+0.762610917 container remove 032ecd819b44d2837634b6d63584ccfbc84d696800efce2b740cb8e1808b8f39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:05:31.977 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68623]: 2024-09-21 03:05:31.584491154 +0000 UTC m=+0.015734989 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:05:31.977 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68623]: 2024-09-21 03:05:31.699309328 +0000 UTC m=+0.130553154 container create 6569c3644c454e05961ea77ee0e37b7dc03e9a33c064ac8195ead1c6b51e6869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:05:31.977 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68623]: 2024-09-21 03:05:31.901522545 +0000 UTC m=+0.332766370 container init 6569c3644c454e05961ea77ee0e37b7dc03e9a33c064ac8195ead1c6b51e6869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T03:05:31.977 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68623]: 2024-09-21 03:05:31.904831802 +0000 UTC m=+0.336075635 container start 6569c3644c454e05961ea77ee0e37b7dc03e9a33c064ac8195ead1c6b51e6869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T03:05:31.977 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:31 smithi178 podman[68623]: 2024-09-21 03:05:31.976378397 +0000 UTC m=+0.407622239 container attach 6569c3644c454e05961ea77ee0e37b7dc03e9a33c064ac8195ead1c6b51e6869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True) 2024-09-21T03:05:32.228 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:32 smithi178 podman[68623]: 2024-09-21 03:05:32.089253815 +0000 UTC m=+0.520497651 container died 6569c3644c454e05961ea77ee0e37b7dc03e9a33c064ac8195ead1c6b51e6869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:05:32.648 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:05:32.714 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:32 smithi178 podman[68646]: 2024-09-21 03:05:32.621166138 +0000 UTC m=+0.521401337 container remove 6569c3644c454e05961ea77ee0e37b7dc03e9a33c064ac8195ead1c6b51e6869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:05:32.714 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:32 smithi178 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.11.service: Deactivated successfully. 2024-09-21T03:05:32.714 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:32 smithi178 systemd[1]: Stopped Ceph osd.11 for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T03:05:32.714 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 21 03:05:32 smithi178 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@osd.11.service: Consumed 12.534s CPU time. 2024-09-21T03:05:32.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:32.730 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:05:32.730 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:05:32.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:05:33.035 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:05:32 smithi031 systemd[1]: Stopping Ceph mds.a for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:33.334 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:05:33 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a[74267]: 2024-09-21T03:05:33.034+0000 7fe1ec7e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:33.334 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:05:33 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a[74267]: 2024-09-21T03:05:33.034+0000 7fe1ec7e4640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:05:35.252 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:05:34 smithi031 podman[110311]: 2024-09-21 03:05:34.973475453 +0000 UTC m=+2.041215440 container died f3a818248bedad7a3aa3e9312367b6b0ee2c90d40fd2ed8e6aba78c1f5c9293e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:05:35.584 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:05:35 smithi031 podman[110311]: 2024-09-21 03:05:35.325064936 +0000 UTC m=+2.392804913 container cleanup f3a818248bedad7a3aa3e9312367b6b0ee2c90d40fd2ed8e6aba78c1f5c9293e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:05:35.584 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 03:05:35 smithi031 bash[110311]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:05:35.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:05:35.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:35.787 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:05:35.787 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:05:35.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:05:36.122 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:35 smithi031 systemd[1]: Stopping Ceph mds.d for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:36.584 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-d[73141]: 2024-09-21T03:05:36.120+0000 7fd180093640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:36.584 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:36 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-d[73141]: 2024-09-21T03:05:36.120+0000 7fd180093640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:05:38.271 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:37 smithi031 podman[110435]: 2024-09-21 03:05:37.96765496 +0000 UTC m=+1.954729102 container died 5f616260a6d9c18f501463c4cd7ec6969b7e0505e0b1354214761833a6e04ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:05:38.272 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:38 smithi031 podman[110435]: 2024-09-21 03:05:38.270658309 +0000 UTC m=+2.257732437 container cleanup 5f616260a6d9c18f501463c4cd7ec6969b7e0505e0b1354214761833a6e04ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:05:38.584 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:38 smithi031 bash[110435]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:05:38.584 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 03:05:38 smithi031 podman[110448]: 2024-09-21 03:05:38.488229591 +0000 UTC m=+0.520862325 container remove 5f616260a6d9c18f501463c4cd7ec6969b7e0505e0b1354214761833a6e04ba7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:38.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:05:38.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:38.673 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:05:38.673 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:05:38.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:05:39.000 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:05:38 smithi031 systemd[1]: Stopping Ceph mds.g for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:39.334 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:05:38 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g[73903]: 2024-09-21T03:05:38.998+0000 7fe12b3a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:39.334 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:05:38 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g[73903]: 2024-09-21T03:05:38.998+0000 7fe12b3a7640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:05:43.241 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:05:42 smithi031 podman[110555]: 2024-09-21 03:05:42.978591899 +0000 UTC m=+4.084118412 container died 49d3dbaf1837b15d1c710d452251c94111f1175e28239c3b81ef433165700a4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:43.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:05:43 smithi031 podman[110555]: 2024-09-21 03:05:43.340274709 +0000 UTC m=+4.445801230 container cleanup 49d3dbaf1837b15d1c710d452251c94111f1175e28239c3b81ef433165700a4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:43.584 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 03:05:43 smithi031 bash[110555]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:05:43.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:05:43.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:43.814 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:05:43.814 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:05:43.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:05:44.145 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 03:05:43 smithi031 systemd[1]: Stopping Ceph mds.j for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:44.584 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 03:05:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j[72764]: 2024-09-21T03:05:44.143+0000 7fd8b0981640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:44.584 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 03:05:44 smithi031 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j[72764]: 2024-09-21T03:05:44.144+0000 7fd8b0981640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:05:46.834 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 03:05:46 smithi031 podman[110676]: 2024-09-21 03:05:46.539794408 +0000 UTC m=+2.504710349 container died 1ad677833a1885cdc7162fff5647042fa2978429ff48003730b0742442f40d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T03:05:47.279 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 03:05:46 smithi031 podman[110676]: 2024-09-21 03:05:46.952241746 +0000 UTC m=+2.917157680 container cleanup 1ad677833a1885cdc7162fff5647042fa2978429ff48003730b0742442f40d84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T03:05:47.280 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 03:05:46 smithi031 bash[110676]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:05:47.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:05:47.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:47.467 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:05:47.468 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:05:47.468 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:05:47.800 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:05:47 smithi165 systemd[1]: Stopping Ceph mds.b for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:48.191 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:05:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: 2024-09-21T03:05:47.798+0000 7f5fc34c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:48.191 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:05:47 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b[63767]: 2024-09-21T03:05:47.798+0000 7f5fc34c2640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:05:51.266 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:05:50 smithi165 podman[68170]: 2024-09-21 03:05:50.98034565 +0000 UTC m=+3.316327315 container died 49ce0d5da4b8b51b53caf19c9fe9f29e9367f2e2c6c2fdc3bae5ca3ffb91c616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T03:05:51.575 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:05:51 smithi165 podman[68170]: 2024-09-21 03:05:51.356483681 +0000 UTC m=+3.692465333 container cleanup 49ce0d5da4b8b51b53caf19c9fe9f29e9367f2e2c6c2fdc3bae5ca3ffb91c616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T03:05:51.575 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 21 03:05:51 smithi165 bash[68170]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:05:51.707 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:05:51.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:51.742 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:05:51.742 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:05:51.742 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:05:51.941 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:05:51 smithi165 systemd[1]: Stopping Ceph mds.e for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:52.441 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:05:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e[63212]: 2024-09-21T03:05:52.051+0000 7fcb30583640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:52.441 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:05:52 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e[63212]: 2024-09-21T03:05:52.051+0000 7fcb30583640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:05:55.287 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:05:55 smithi165 podman[68300]: 2024-09-21 03:05:55.002555099 +0000 UTC m=+3.045773637 container died 34ee203c202183ac7c48112688cc74f553e90e0974e57e849ebbba0c7c659984 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:05:55.604 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:05:55 smithi165 podman[68300]: 2024-09-21 03:05:55.36085883 +0000 UTC m=+3.404077363 container cleanup 34ee203c202183ac7c48112688cc74f553e90e0974e57e849ebbba0c7c659984 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T03:05:55.604 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 21 03:05:55 smithi165 bash[68300]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:05:55.742 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:05:55.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:55.775 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:05:55.775 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:05:55.775 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:05:55.941 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:55 smithi165 systemd[1]: Stopping Ceph mds.h for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:05:56.441 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h[63497]: 2024-09-21T03:05:56.088+0000 7f6aac263640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:05:56.441 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:56 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h[63497]: 2024-09-21T03:05:56.088+0000 7f6aac263640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:05:59.248 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:58 smithi165 podman[68420]: 2024-09-21 03:05:58.997278654 +0000 UTC m=+3.008252774 container died 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:05:59.507 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:59 smithi165 podman[68420]: 2024-09-21 03:05:59.322351317 +0000 UTC m=+3.333325427 container cleanup 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:05:59.508 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:59 smithi165 bash[68420]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:05:59.756 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:05:59.777 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:59 smithi165 podman[68431]: 2024-09-21 03:05:59.507013944 +0000 UTC m=+0.510756904 container remove 4d0682e3266a9e057e8bbadc2bbf351a377ae4eacac1eba177c351412847517e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-h, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:05:59.777 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:59 smithi165 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.h.service: Deactivated successfully. 2024-09-21T03:05:59.777 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:59 smithi165 systemd[1]: Stopped Ceph mds.h for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T03:05:59.777 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 21 03:05:59 smithi165 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.h.service: Consumed 1.945s CPU time. 2024-09-21T03:05:59.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:05:59.789 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:05:59.789 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:05:59.789 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:06:00.044 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:05:59 smithi165 systemd[1]: Stopping Ceph mds.k for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:06:00.441 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k[62931]: 2024-09-21T03:06:00.133+0000 7fbdb7f42640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:06:00.441 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:00 smithi165 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k[62931]: 2024-09-21T03:06:00.133+0000 7fbdb7f42640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:06:03.278 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:02 smithi165 podman[68536]: 2024-09-21 03:06:02.981016889 +0000 UTC m=+2.974474843 container stop 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T03:06:03.278 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 podman[68536]: 2024-09-21 03:06:03.007109103 +0000 UTC m=+3.000567054 container died 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T03:06:03.528 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 podman[68536]: 2024-09-21 03:06:03.351707801 +0000 UTC m=+3.345165752 container cleanup 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:06:03.528 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 bash[68536]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:06:03.764 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:06:03.785 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 podman[68549]: 2024-09-21 03:06:03.527747499 +0000 UTC m=+0.539431978 container remove 392f53cb314353651c8c8477d6183a4f0fb064584c831ac4a41ba79836312bdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:06:03.785 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.k.service: Deactivated successfully. 2024-09-21T03:06:03.785 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 systemd[1]: Stopped Ceph mds.k for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T03:06:03.785 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 21 03:06:03 smithi165 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.k.service: Consumed 2.199s CPU time. 2024-09-21T03:06:03.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:06:03.796 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:06:03.796 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:06:03.796 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:06:04.104 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:06:03 smithi178 systemd[1]: Stopping Ceph mds.c for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:06:04.367 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:06:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c[63941]: 2024-09-21T03:06:04.102+0000 7f96713b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:06:04.367 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:06:04 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c[63941]: 2024-09-21T03:06:04.102+0000 7f96713b5640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:06:07.271 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:06:06 smithi178 podman[68725]: 2024-09-21 03:06:06.990247394 +0000 UTC m=+2.997612968 container died abf086330aad1b541846e81e1044cc9bf87460a869d5bd2fd46bacac7dfec62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T03:06:07.596 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:06:07 smithi178 podman[68725]: 2024-09-21 03:06:07.361125114 +0000 UTC m=+3.368490685 container cleanup abf086330aad1b541846e81e1044cc9bf87460a869d5bd2fd46bacac7dfec62b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T03:06:07.597 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 21 03:06:07 smithi178 bash[68725]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:06:07.744 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:06:07.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:06:07.780 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:06:07.780 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:06:07.780 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:06:08.124 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:06:07 smithi178 systemd[1]: Stopping Ceph mds.f for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:06:08.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:06:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f[63673]: 2024-09-21T03:06:08.123+0000 7ffb61d59640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:06:08.478 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:06:08 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f[63673]: 2024-09-21T03:06:08.123+0000 7ffb61d59640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:06:11.273 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:06:11 smithi178 podman[68856]: 2024-09-21 03:06:11.001166458 +0000 UTC m=+2.991855294 container died 3de6889f53ff89eaac1143c8aa72ad8944b584b73b5e4325e6a80c98ca400de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:06:11.624 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:06:11 smithi178 podman[68856]: 2024-09-21 03:06:11.355746639 +0000 UTC m=+3.346435529 container cleanup 3de6889f53ff89eaac1143c8aa72ad8944b584b73b5e4325e6a80c98ca400de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:06:11.624 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 21 03:06:11 smithi178 bash[68856]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:06:11.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:06:11.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:06:11.816 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:06:11.816 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:06:11.816 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:06:12.144 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:06:11 smithi178 systemd[1]: Stopping Ceph mds.i for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:06:12.478 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:06:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i[64221]: 2024-09-21T03:06:12.142+0000 7f90e2a3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:06:12.478 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:06:12 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i[64221]: 2024-09-21T03:06:12.142+0000 7f90e2a3d640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:06:15.228 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:06:14 smithi178 podman[68977]: 2024-09-21 03:06:14.992582276 +0000 UTC m=+2.950565811 container died 17789cc7aef87995d915eb06482f4f947c781650ae6f820db3ac3627ae2b9d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:06:15.702 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:06:15 smithi178 podman[68977]: 2024-09-21 03:06:15.484350179 +0000 UTC m=+3.442333708 container cleanup 17789cc7aef87995d915eb06482f4f947c781650ae6f820db3ac3627ae2b9d12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:06:15.702 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 21 03:06:15 smithi178 bash[68977]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:06:15.859 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:06:15.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:06:15.892 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:06:15.892 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:06:15.892 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:06:16.205 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:15 smithi178 systemd[1]: Stopping Ceph mds.l for fdd37b14-77c2-11ef-baf5-efdc52797490... 2024-09-21T03:06:16.478 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l[63406]: 2024-09-21T03:06:16.204+0000 7f40286f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:06:16.478 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:16 smithi178 ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l[63406]: 2024-09-21T03:06:16.204+0000 7f40286f6640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:06:19.255 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 podman[69099]: 2024-09-21 03:06:19.001598926 +0000 UTC m=+2.895498346 container died 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T03:06:19.522 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 podman[69099]: 2024-09-21 03:06:19.328962918 +0000 UTC m=+3.222862329 container cleanup 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:06:19.523 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 bash[69099]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:06:19.756 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:06:19.777 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 podman[69111]: 2024-09-21 03:06:19.521446481 +0000 UTC m=+0.517087244 container remove 4031152f76c10c555453db82ba6ad46d4d54c442e9fb7d500adfbbf452f74cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fdd37b14-77c2-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:06:19.777 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.l.service: Deactivated successfully. 2024-09-21T03:06:19.777 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 systemd[1]: Stopped Ceph mds.l for fdd37b14-77c2-11ef-baf5-efdc52797490. 2024-09-21T03:06:19.777 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 21 03:06:19 smithi178 systemd[1]: ceph-fdd37b14-77c2-11ef-baf5-efdc52797490@mds.l.service: Consumed 2.325s CPU time. 2024-09-21T03:06:19.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:06:19.788 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:06:19.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:06:20.001 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T03:06:22.713 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:06:22.917 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T03:06:25.475 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:06:25.677 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T03:06:28.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:06:28.194 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:06:28.227 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:06:28.253 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:06:28.255 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi031/crash 2024-09-21T03:06:28.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:06:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:06:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:06:28.299 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi165/crash 2024-09-21T03:06:28.300 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:06:28.338 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:06:28.338 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:06:28.340 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi178/crash 2024-09-21T03:06:28.341 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:06:28.380 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:06:28.380 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:06:28.381 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:06:28.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:06:28.424 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:06:28.424 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:06:28.467 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:06:28.469 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:06:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:06:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:: No such file or directory 2024-09-21T03:06:28.493 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:06:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:06:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:06:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:06:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:06:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:06:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:06:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:06:28.499 INFO:teuthology.orchestra.run.smithi031.stderr: 83.9% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:06:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.g.log: 90.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:: No such file or directory 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:06:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.11.log: 80.8%gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr: 90.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.l.log: 87.4% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:06:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:06:28.506 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:06:28.506 INFO:teuthology.orchestra.run.smithi165.stderr:: No such file or directory 2024-09-21T03:06:28.506 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:06:28.506 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log: 90.2% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log: 80.9% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:06:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:06:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:06:28.509 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:06:28.510 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:06:28.510 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:06:28.511 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log: 91.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:06:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:06:28.513 INFO:teuthology.orchestra.run.smithi165.stderr: 92.6% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:06:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:06:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:06:28.514 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:06:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:06:28.514 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:06:28.515 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.h.log: 87.4% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:06:28.516 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:06:28.517 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.z.log: 92.6% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:06:28.517 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:06:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.d.log: /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.x.log: 95.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:06:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:06:28.527 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.j.log: 95.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:06:28.528 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:06:28.530 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:06:28.531 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:06:28.543 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:06:28.546 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:06:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:06:28.565 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:06:28.583 INFO:teuthology.orchestra.run.smithi178.stderr: 94.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:06:28.584 INFO:teuthology.orchestra.run.smithi031.stderr: 94.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:06:28.585 INFO:teuthology.orchestra.run.smithi031.stderr: 94.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:06:28.587 INFO:teuthology.orchestra.run.smithi165.stderr: 94.5% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:06:28.588 INFO:teuthology.orchestra.run.smithi165.stderr: 93.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:06:28.589 INFO:teuthology.orchestra.run.smithi178.stderr: 93.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:06:28.614 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:06:28.653 INFO:teuthology.orchestra.run.smithi178.stderr: 93.9% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:06:28.779 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:06:28.878 INFO:teuthology.orchestra.run.smithi165.stderr: 92.9% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:06:28.946 INFO:teuthology.orchestra.run.smithi165.stderr: 92.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:06:28.989 INFO:teuthology.orchestra.run.smithi178.stderr: 91.8% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:06:29.455 INFO:teuthology.orchestra.run.smithi031.stderr: 90.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:06:30.252 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:06:30.317 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:06:30.340 INFO:teuthology.orchestra.run.smithi178.stderr: 92.8% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:06:30.408 INFO:teuthology.orchestra.run.smithi031.stderr: 92.5% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T03:06:30.754 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:06:30.785 INFO:teuthology.orchestra.run.smithi165.stderr: 92.3% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:06:30.879 INFO:teuthology.orchestra.run.smithi031.stderr: 92.4% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T03:06:30.963 INFO:teuthology.orchestra.run.smithi178.stderr: 92.1% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:06:30.965 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-21T03:06:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m2.480s 2024-09-21T03:06:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m8.846s 2024-09-21T03:06:30.965 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.314s 2024-09-21T03:06:31.031 INFO:teuthology.orchestra.run.smithi165.stderr: 92.5% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:06:31.187 INFO:teuthology.orchestra.run.smithi165.stderr: 92.7% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:06:31.189 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-21T03:06:31.189 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m2.705s 2024-09-21T03:06:31.189 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m10.037s 2024-09-21T03:06:31.189 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m0.336s 2024-09-21T03:06:31.241 INFO:teuthology.orchestra.run.smithi031.stderr: 91.6% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:06:31.394 INFO:teuthology.orchestra.run.smithi031.stderr: 92.6% -- replaced with /var/log/ceph/fdd37b14-77c2-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T03:06:31.397 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-21T03:06:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m2.915s 2024-09-21T03:06:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m10.966s 2024-09-21T03:06:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.382s 2024-09-21T03:06:31.397 INFO:tasks.cephadm:Archiving logs... 2024-09-21T03:06:31.397 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi031/log 2024-09-21T03:06:31.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:06:32.975 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi165/log 2024-09-21T03:06:32.976 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:06:34.438 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi178/log 2024-09-21T03:06:34.439 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:06:35.734 INFO:tasks.cephadm:Removing cluster... 2024-09-21T03:06:35.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --force 2024-09-21T03:06:35.949 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T03:06:36.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --force 2024-09-21T03:06:36.430 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T03:06:36.699 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fdd37b14-77c2-11ef-baf5-efdc52797490 --force 2024-09-21T03:06:36.893 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: fdd37b14-77c2-11ef-baf5-efdc52797490 2024-09-21T03:06:37.158 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T03:06:37.158 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:06:37.178 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:06:37.194 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:06:37.209 INFO:tasks.cephadm:Teardown complete 2024-09-21T03:06:37.210 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:06:37.220 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:06:37.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:06:37.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:06:37.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:06:37.339 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:06:37.349 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:06:37.349 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:06:37.352 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:06:37.354 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:06:37.366 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:06:37.366 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:06:37.368 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:06:37.370 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:06:37.370 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-09-21T03:06:37.370 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -65us[ -73us] +/- 40ms 2024-09-21T03:06:37.370 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -42us[ -50us] +/- 50ms 2024-09-21T03:06:37.371 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +177us[ +169us] +/- 74ms 2024-09-21T03:06:37.371 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:06:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:06:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-21T03:06:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -60us[ -44us] +/- 40ms 2024-09-21T03:06:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -30us[ -30us] +/- 50ms 2024-09-21T03:06:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +185us[ +185us] +/- 75ms 2024-09-21T03:06:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:06:37.373 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:06:37.373 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-21T03:06:37.373 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -73us[ -85us] +/- 40ms 2024-09-21T03:06:37.374 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -46us[ -59us] +/- 50ms 2024-09-21T03:06:37.374 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +177us[ +165us] +/- 75ms 2024-09-21T03:06:37.374 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:06:37.374 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:06:37.384 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:06:37.385 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:06:37.418 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:06:37.506 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:06:37.571 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:06:37.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:06:37.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:06:37.667 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:06:37.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:06:37.694 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:06:37.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:06:37.722 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:06:37.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:06:37.742 INFO:teuthology.task.internal:Duration was 1861.968474 seconds 2024-09-21T03:06:37.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:06:37.751 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:06:37.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:06:37.754 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:06:37.756 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:06:37.792 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:06:37.794 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:06:37.800 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:06:38.231 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:06:38.231 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T03:06:38.231 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:06:38.255 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-09-21T03:06:38.255 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:06:38.293 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-09-21T03:06:38.294 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:06:38.322 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:06:38.322 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:06:38.325 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:06:38.335 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:06:38.388 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:06:38.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:06:38.395 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:06:38.401 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:06:39.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:06:39.682 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:06:39.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:06:39.708 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:06:39.738 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:06:39.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:06:39.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:06:39.779 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:06:39.781 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:06:39.800 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-21T03:06:39.806 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-09-21T03:06:39.831 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-09-21T03:06:39.859 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:06:39.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:06:39.874 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:06:39.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:06:39.890 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:06:39.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:06:39.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:06:39.924 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:06:39.925 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi031 2024-09-21T03:06:39.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:06:39.984 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi165 2024-09-21T03:06:39.984 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:06:40.021 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913435/remote/smithi178 2024-09-21T03:06:40.021 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:06:40.057 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:06:40.057 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:06:40.060 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:06:40.062 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:06:40.108 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:06:40.125 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:06:40.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:06:40.135 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:06:40.136 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:06:40.138 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:06:40.140 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:06:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:06 /home/ubuntu/cephtest 2024-09-21T03:06:40.154 INFO:teuthology.orchestra.run.smithi165.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:06 /home/ubuntu/cephtest 2024-09-21T03:06:40.163 INFO:teuthology.orchestra.run.smithi178.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:06 /home/ubuntu/cephtest 2024-09-21T03:06:40.164 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:06:40.177 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:06:40.200 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} duration: 1861.9684743881226 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T03:06:40.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:06:40.286 INFO:teuthology.run:pass